Today we will begin discussing Naive Bayes, a Classification algorithm that uses probability.

According to the author, these estimates are based on probabilistic methods, or methods concerned with describing uncertainty. They use the data from past events to extrapolate to future events.

Naive Bayes has been used successful for working with email.

Spam Filtering

• spam filtering
• SpamBayes
• Spam Filtering with Naive Bayes - Which Naive Bayes?
• A Bayesian Approach to Filtering Junk E-Mail

Naive Bayes has been used successful for working with email.

Prioritizing

Folderizing

Naive Bayes has been used successful for Text Classification.

• text classification
• text classification
• text classification

Naive Bayes has been used for Sentiment Analysis.

• Citius

Naive Bayes has been used for Intrusion Detection.

• Network Intrusion Detection

Naive Bayes has been used Medical Diagnosis

• Diagnosis
• Diagnosis

• Coursera Stanford Videos on Graphical Models

• event
• trial
• mutually exclusive
• Venn Diagrams
• joint probability
• independent events
• dependent events
• conditional probability

• Bayes' Theorem
• prior probability
• likelihood
• posterior probability

\(P(spam|Viagra) = \frac{P(Viagra|spam)P(spam)}{P(Viagra)}\)

prior: \(P(spam)\)

likelihood: \(P(Viagra|spam) = L(spam)\)

posterior: \(P(spam|Viagra)\)

The classification is done using the posterior probability.

The class with the highest probability is the classification for that observation/example.

The naive Bayes (NB) algorithm describes a simple application using Bayes' theorem for classification.

NB is the de facto standard for much text classification.

See page 97/95 for the Strengths and Weaknesses.

The naive Bayes algorithm is named naive because it makes a couple of "naive" assumptions about the data.

1. It assumes that all of the features in the dataset are equally important.
2. It assumes that all of the features in the dataset are independent.

Naive Bayes assumes class-conditional independence, which means the events are independent so long as they are conditioned on the same class value.

Features need to be in categories

• discretize
• bin
• cut points

(Reminder: All variables/features need to be numeric for kNN.)

Or use a different algorithm from a different package.

Next time we will work with the example in the book - filtering mobile phone (SMS) spam with the naive Bayes algorithm.