A Bayesian spam filter uses information about previously seen e-mail messages to guess whether an incoming e-mail is spam. It looks for occurrences of particular words in messages.
For a word , the probability that appears in a spam message is estimating by determining:
- the number of times appears in a message from a large set of messages known to be spam
- the number of times appears in a large set of messages that are known not to be spam
Sometimes spam filters are not perfect:
- A false negative occurs when a spam filter fails to identify a spam message as spam.
- A false positive occurs when a spam filter identifies a message as spam that is not spam.
title: Example
Suppose we have found the word "Bitcoin" occurs in $250$ of $2000$ messages known to be spam, and in $5$ out of $1000$ messages known not to be spam.
Estimate the probability that an incoming message containing the word "Bitcoin" is spam, assuming that it is equally likely that an incoming message is or is not spam.
Let our threshold for spam be $0.9$, will we reject the message?
Let $S$ be the event that an incoming message is spam, and $R$ be the event that the message contains the word "Bitcoin". Then:
- $p(R | S) = \frac{250}{2000} = 0.125$
- $p(R | \bar S) = \frac{5}{1000} = 0.005$
- $p(S) = p(\bar S) = 0.5$
$$
\begin{aligned}
p(S | R) &= \frac{0.125 \cdot 0.5}{0.125 \cdot 0.5 + 0.005 \cdot 0.5} \\
&= \frac{0.125}{0.13} \\
&\approx 0.962
\end{aligned}
$$
As $0.962 > 0.9$, we reject the message as spam.