With supervised learning, we train a system to understand a particular problem using a subset of collected data. The trained function (model) learns to predict the output from the training data input. Once trained, it can be used to provide answers to new (unseen) input data.
There are two types of machine learning applications:
- Classification: sorting items into specific classes and categories; predicting an item belongs in a specific class or category
- Regression: predicting an outcome (number); number can then be fed into another application or system
Classification
Given a collection of records (training set), each record containing a set of attributes, one of the attributes being a class. We find a model for class attribute as a function of the values of other attributes.
Previously unseen records should be assigned a class as accurately as possible. We typically separate data sets into training and test sets (validation).
Example of algorithms are:
- Decision Trees
- K-Nearest Neighbour
- Neural Networks
- Support Vector Machines
- Naïve Bayes
Regression
Regression models are a statistical model, we assume a relationship between the input variables and the single output variable , so we can predict from . Hence need to establish correlations between variables.
Correlation is the degree of relationship between two variables:
- Direction (positive: increase as other increase, negative: decrease as other increase)
- Strength (how well the data fits on a line)
However correlation does not mean causation. Once we have collected some data and observed a significant correlation, then we can use this relationship to predict new outcomes.
There are may techniques such as:
- linear regression
- logistic regression
- polynomial regression
- ridge regressino
- lasso regresion
- ElasticNet regression