📝
📝
📝
📝
Tzu-Heng's wiki
gitbooklzhbrian.me
Search
⌃K
Links

Traditionals

some traditional machine learning algorithms

Survey Papers / Repos

Resources

Tasks

Supervised

  • Linear Regression
y=ax+bL(y,y^)=(y−y^)2y=ax+b\\ L(y,\hat{y}) = (y-\hat{y})^2
  • Logistic Regression
y=11+e−(ax+b)L(y,y^)=−y^log⁡y−(1−y^)log⁡(1−y)y=\frac{1}{1+e^{-(ax+b)}} \\ L(y,\hat{y}) = -\hat{y}\log y - (1 - \hat{y}) \log (1-y)
  • Naive Bayes
P(A∣B)=P(B∣A)P(A)P(B)P(A|B) = \frac{P(B|A)P(A)}{P(B)}
  • Support Vector Machine (SVM)
    • Training process: Lagrange -> Dual Problem -> SMO
    min⁡12∣∣w∣∣2s.t. y(i)(wTx(i)+b)≥1,i=1,...,m\min \frac{1}{2} ||w||^2 \\ \text{s.t.}~y^{(i)}(w^{T}x^{(i)}+b) \geq 1, i=1,...,m
  • K Nearest Neighbor (kNN)
  • Expectation-Maximization (EM)
  • Linear Discrimant Analysis (LDA)
  • Decision Tree
  • Random Forest
  • Gradient Boosting Tree (GBDT)

Semi-supervised

Weakly-supervised

Unsupervised

  • Clustering
    • K-means
    • Mean-shift
    • DBSCAN
  • Principal Component Analysis (PCA)
  • Latent Dirichlet allocation (LDA) Topic Modeling

Others

Ensemble

  • K-Fold Cross Validation
  • Bagging
  • Boosting

Metrics

Text
True Samples
False Samples
Predict True
True Positive
False Positive [Type I Error]
Predict False
False Negative [Type II Error]
True Negative
  • Precision and Recall
    • ​
      Precision=TPTP+FP\text{Precision} = \frac{\text{TP}}{\text{TP} +\text{FP}}
      ​
    • ​
      Recall=TPTP+FN\text{Recall} = \frac{\text{TP}}{\text{TP}+\text{FN}}
      ​
  • F1 Score
    • ​
      F1 score=2⋅Precision⋅RecallPrecision+Recall\text{F1 score} = 2 \cdot\frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} +\text{Recall}}
      ​
  • Receiver Operating Characteristic (ROC)
    • ​
      TPR=TPTP+FN\text{TPR} = \frac{\text{TP}}{\text{TP}+\text{FN}}
      ​
    • ​
      FPR=FPFP+TN\text{FPR} = \frac{\text{FP}}{\text{FP}+\text{TN}}
      ​
  • Area Under ROC (AUC)
  • Confusion Matrix

Reference

Previous
Tzu-Heng's wiki
Next - Machine Learning
Deep Learning
Last modified 1yr ago