it‘s all about conditional probability and combined probability.
P(A|B)=P(AB) / P(B)
=> P(A|B)=P(A)P(B|A) / P(B)
=> P(A|B)=P(A)P(B|A) / (P(A)P(B|A)+P(A’)P(B|A’))
joint probability
assumption : events are independent .
E1 = p(s|w1)*p(s|w2)*p(s)
E2 = (1-p(s|w1))*(1-p(s|w2))*(1-p(s))
P(S|w1,w2) = P(S)P(S|w1)P(S|w2) / ( P(S)P(S|w1)P(S|w2) + P(~S)P(~S|w1)P(~S|w2) )
bayesian inference
后验概率 ＝ 先验概率 ｘ 调整因子
usage example
1.spam mail filter
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
p(spam|word) = p(spam)p(word|spam) / p(spam)p(word|spam)+p(healthy)p(word|healthy) p(spam|word1,word2) = p(spam)p(word1,word2|spam) / (p(spam)p(word1,word2|spam) + p(healthy)p(word1,word2|healthy)) = p(spam)p(word1|spam)p(word2|spam) / p(spam)p(word1|spam)p(word2|spam)+p(healthy)p(word1|healthy)p(word2|healthy) #so call "naive",ignore relationship between words = (p(spam|word1)p(spam|word2)p(word1)p(word2) / p(spam)) / ((p(spam|word1)p(spam|word2)p(word1)p(word2) / p(spam) ) + (p(healthy|word1)p(healthy|word2)p(word1)p(word2) / p(healthy))) = p(spam|word1)p(spam|word2) / (p(spam|word1)p(spam|word2) + p(healthy|word1)p(healthy|word2)) # prior probability : p(spam) = p(healthy) = 0.5 = p(spam|word1)p(spam|word2) / (p(spam|word1)p(spam|word2) + (1-p(spam|word1))(1-p(spam|word2))) = p1p2 / (p1p2 + (1-p1)(1-p2)) #p1=p(spam|word1) p2=p(spam|word2) |
reference:
http://www.ruanyifeng.com/blog/2011/08/bayesian_inference_part_one.html