2. Analysing Quality Of Supervised Algorithms

Metrics for Classification Problems

Accuracy alone may not be a sufficient metric, so we look at different factors such as:

• Precision

  Precision: proportion of true positives over the true and false positives $\frac{\text{TP}}{\text{TP }+\text{ FP}}$

  Link to original

• Sensitivity

  Sensitivity (or recall): proportion of true positives over true positives and false negatives $\frac{\text{TP}}{\text{TP }+\text{ FN}}$

  Link to original

• F1 Score

  F1 Score: harmonic mean between sensitivity and precision $F_1=\frac{2}{{\text{recall}}^{-1}+{\text{precision}}^{-1}}$

  The general weighted form looks like this: $F_{\beta }=\frac{1+{\beta }^2}{{\beta }^2{\text{precision}}^{-1}+{\text{recall}}^{-1}}$

  Link to original

Metrics for Regression Problems

Assume our regression is predicting real numbers. Call predictions ${\hat{y}}_1,\dots ,{\hat{y}}_n$ and true values $y_1,\dots ,y_n$. Then define the following:

• Mean Absolute Error

  Mean Absolute Error: $\frac{1}{n}{\sum }_{i=1}^n|{\hat{y}}_i-y_i|$

  Link to original

• Mean Squared Error

  Mean Squared Error: $\frac{1}{n}{\sum }_{i=1}^n({\hat{y}}_i-y_i{)}^2$

  Link to original

• Root-Mean-Squared-Error

  Root-Mean-Squared-Error: $\sqrt{\frac{1}{n}{\sum }_{i=1}^n({\hat{y}}_i-y_i{)}^2}$

  Link to original