Model evaluation

Description:

• https://www.youtube.com/watch?v=LbX4X71-TFI

	label = +	label = -
predict +	tp	fp
predict -	fn	tn

• Accuracy: % of correct predictions
  • $\text{accuracy}=\frac{tp+tn}{tp+fp+tn+fn}$
• Precision Metric: % of positive predictions that are correct
  • $\text{precision}=\frac{tp}{tp+fp}$
• Recall Metric: % of correct positive labels that are found
  • $\text{recall}=\frac{tp}{tp+fn}$
• F Metric
  • measure assess the trade-off between precision and recall
  • $F=\frac{1}{\alpha \frac{1}{P}+(1-\alpha )\frac{1}{R}}=\frac{({\beta }^2+1)PR}{{\beta }^{2}P+R}$
  • Usually, prefer balanced F-1 measure
    • $\beta =1$, that is $\alpha =1/2$
    • Harmonic mean $F=\frac{2PR}{P+R}$
• F_1 score $=2\times \frac{Precision\times Recall}{Precision+Recall}=\frac{2tp}{2tp+fp+fn}$

Symmetric vs asymmetric metrics:

Symmetric metrics	Asymmetric metrics
Treat all classes the same	Measures a model’s performance w.r.t. a class
Accuracy	F1, recall, precision, ROC