Generalisation Error
The generalisation error is the error a model has on unseen data. We rarely have a model for unseen data, and we only have samples to deal with, however we can minimise the empirical error which we have control over.
Link to original
Underfitting
Underfitting is when the model fails to capture the complexity of the training data (or otherwise fails to find a pattern in the data). Refer to left plot.
Link to original
Overfitting
Overfitting is when the model fits too much to the training data and fails to generalise to unseen data. (see graph for degree 29)
Link to original
Example Training Evaluations
We can plot the model complexity against the error to try to figure out what fits best.
See Underfitting and Overfitting Demo.ipynb notebook.