Traditionals

some traditional machine learning algorithms

Survey Papers / Repos

Resources

Tasks

Supervised

  • Linear Regression

y=ax+bL(y,y^)=(yβˆ’y^)2y=ax+b\\ L(y,\hat{y}) = (y-\hat{y})^2
  • Logistic Regression

y=11+eβˆ’(ax+b)L(y,y^)=βˆ’y^log⁑yβˆ’(1βˆ’y^)log⁑(1βˆ’y)y=\frac{1}{1+e^{-(ax+b)}} \\ L(y,\hat{y}) = -\hat{y}\log y - (1 - \hat{y}) \log (1-y)
  • Naive Bayes

P(A∣B)=P(B∣A)P(A)P(B)P(A|B) = \frac{P(B|A)P(A)}{P(B)}
  • Support Vector Machine (SVM)

    • Training process: Lagrange -> Dual Problem -> SMO

    min⁑12∣∣w∣∣2s.t.Β y(i)(wTx(i)+b)β‰₯1,i=1,...,m\min \frac{1}{2} ||w||^2 \\ \text{s.t.}~y^{(i)}(w^{T}x^{(i)}+b) \geq 1, i=1,...,m
  • K Nearest Neighbor (kNN)

  • Expectation-Maximization (EM)

  • Linear Discrimant Analysis (LDA)

  • Decision Tree

  • Random Forest

  • Gradient Boosting Tree (GBDT)

Semi-supervised

Weakly-supervised

Unsupervised

  • Clustering

    • K-means

    • Mean-shift

    • DBSCAN

  • Principal Component Analysis (PCA)

  • Latent Dirichlet allocation (LDA) Topic Modeling

Others

Ensemble

  • K-Fold Cross Validation

  • Bagging

  • Boosting

Metrics

​

True Samples

False Samples

Predict True

True Positive

False Positive [Type I Error]

Predict False

False Negative [Type II Error]

True Negative

  • Precision and Recall

    • ​Precision=TPTP+FP\text{Precision} = \frac{\text{TP}}{\text{TP} +\text{FP}}​

    • ​Recall=TPTP+FN\text{Recall} = \frac{\text{TP}}{\text{TP}+\text{FN}}​

  • F1 Score

    • ​F1Β score=2β‹…Precisionβ‹…RecallPrecision+Recall\text{F1 score} = 2 \cdot\frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} +\text{Recall}}​

  • Receiver Operating Characteristic (ROC)

    • ​TPR=TPTP+FN\text{TPR} = \frac{\text{TP}}{\text{TP}+\text{FN}}​

    • ​FPR=FPFP+TN\text{FPR} = \frac{\text{FP}}{\text{FP}+\text{TN}}​

  • Area Under ROC (AUC)

  • Confusion Matrix

Reference

​