LGT

Gini Index

	A	B	Total
M	6	4	10
F	4	6	10
	10	10

1. $\text{GINI}(M)=1-\left(\begin{array}{c}{\frac{6}{10}}^2+{\frac{4}{10}}^2\end{array}\right)=0.48$
2. $\text{GINI}(F)=1-\left(\begin{array}{c}{\frac{4}{10}}^2+{\frac{6}{10}}^2\end{array}\right)=0.48$
3. ${\text{GINI}}_{\text{split}}(\text{Gender})=\frac{10}{20}\cdot 0.48+\frac{10}{20}\cdot 0.48=0.48$

	A	B	Total
High	1	3	4
Medium	8	0	8
Low	1	7	8
	10	10

4. $\text{GINI}(\text{High})=1-\left(\begin{array}{c}{\frac{1}{4}}^2+{\frac{3}{4}}^2\end{array}\right)=0.375$
5. $\text{GINI}(\text{Medium})=1-\left(\begin{array}{c}{\frac{8}{8}}^2\end{array}\right)=0$
6. $\text{GINI}(\text{Low})=1-\left(\begin{array}{c}{\frac{1}{8}}^2+{\frac{7}{8}}^2\end{array}\right)=0.21875$
7. ${\text{GINI}}_{\text{split}}(\text{Income})=\frac{4}{20}\cdot 0.375+\frac{8}{20}\cdot 0+\frac{8}{20}\cdot 0.21875=0.1625$

	A	B	Total
Excellent	3	5	8
Fair	3	2	5
Poor	4	3	7
	10	10

8. $\text{GINI}(\text{Excellent})=1-\left(\begin{array}{c}{\frac{3}{8}}^2+{\frac{5}{8}}^2\end{array}\right)=0.46875$
9. $\text{GINI}(\text{Fair})=1-\left(\begin{array}{c}{\frac{3}{5}}^2+{\frac{2}{5}}^2\end{array}\right)=0.48$
10. $\text{GINI}(\text{Poor})=1-\left(\begin{array}{c}{\frac{4}{7}}^2+{\frac{3}{7}}^2\end{array}\right)=0.48979$
11. ${\text{GINI}}_{\text{split}}(\text{Credit Rating})=\frac{8}{20}\cdot 0.46875+\frac{5}{20}\cdot 0.48+\frac{7}{20}\cdot 0.490=0.4789285$

Neural Network

TYPO: bottom most should be 4/3

Hidden layer uses ReLU activation function. Output uses identity function.

$$\text{ReLU}(x)=max(0,x)$$

$MSE=\frac{1}{n}{\sum }_{i=1}^n({\hat{y}}_i-y_i{)}^2$

1. $x_n=(1,2,3)$ $h_1=0.5\cdot 1+2\cdot 2-1\cdot 3=1.5$ $h_2=1\cdot 1=1$ $\hat{y}=2\cdot 1.5-2\cdot 2=1$
2. $x_n=(1,2,3)$ $h_1=1\cdot 3+3\cdot 1=6$ $h_2=1\cdot -1-2\cdot 0.5+3\cdot \frac{4}{3}=2$ $\hat{y}=-\frac{1}{3}\cdot 6+3\cdot 2=4$
3. $x_n=(5,-2,0)$ $h_1=0.5\cdot 5-2\cdot 2=-1.5$ (ReLU redefines as $0$) $h_2=5\cdot 1-2\cdot 0=5$ $\hat{y}=-5\cdot 2=-10$
4. $x_n=(5,-2,0)$ $h_1=3\cdot 5=15$ $h_2=-1\cdot 5-2\cdot -0.5=-4$ (ReLU redefines as $0$) $\hat{y}=15\cdot -\frac{1}{3}=-5$
5. $MSE=\frac{1}{2}\left(\begin{array}{c}(1-5{)}^2+(-10-(-3){)}^2\end{array}\right)=32.5$
6. $MSE=\frac{1}{2}\left(\begin{array}{c}(4-5{)}^2+(-5-(-3){)}^2\end{array}\right)=2.5$
7. $x_n=(1,1,0)$ $h_1=3$ $h_2=-1+-0.5=-1.5$ (ReLU redefines as $0$) $\hat{y}=3\cdot -\frac{1}{3}=-1$