Naive Bayes

A simple yet surprisingly effective probabilistic classifier built on Bayes' theorem.

Bayes' Theorem

P(y | X) = P(X | y) · P(y) / P(X)

We want the probability of class y given features X. Bayes flips this into terms we can estimate from data.

The "Naive" Assumption

Naive Bayes assumes all features are conditionally independent given the class. This rarely holds in practice, but the model still works remarkably well.

P(X | y) = ∏ P(xᵢ | y)

Variants

Gaussian NB: features are continuous, normally distributed
Multinomial NB: for word counts (text classification)
Bernoulli NB: for binary features

Python Implementation

from sklearn.naive_bayes import MultinomialNB
from sklearn.feature_extraction.text import CountVectorizer

texts = ["free money now", "meeting at noon", "win cash prize", "project update"]
labels = [1, 0, 1, 0]  # 1 = spam

vec = CountVectorizer()
X = vec.fit_transform(texts)

nb = MultinomialNB()
nb.fit(X, labels)
print(nb.predict(vec.transform(["free prize"])))

Best use case

Naive Bayes is the classic algorithm for text classification — spam filters, sentiment analysis, document categorization. Fast to train, scales linearly with data.