Gain Ratio

Gain Ratio is a new metric that incorporates another metric split information that penalises features with lots of values (we ask what the value of the feature is):

$$\begin{array}{rl}\text{SplitInformation}(S,A)& =-\sum _i\frac{|S_i|}{|S|}{\log }_2\frac{|S_i|}{|S|}\\ \text{GainRatio}(S,A)& =\frac{\text{Gain}(S,A)}{\text{SplitInformation(S,A)}}\end{array}$$

We now use gain ratio as the new way of picking the best feature.