---
title: "Naive Bayes"
author: "Prof. Eric A. Suess"
date: "February 14, 2024"
format: 
  revealjs:
    self-contained: true
---


## Introduction

Today we will begin discussing **Naive Bayes**, a **Classification** algorithm that uses **probability**.

According to the author, these estimates are based on probabilistic methods, or methods concerned with describing uncertainty.  They use the data from past events to extrapolate to future events.

## Naive Bayes

Naive Bayes has been used successful for working with email.

**Spam Filtering**

- [spam filtering](http://en.wikipedia.org/wiki/Naive_Bayes_spam_filtering)
- [SpamBayes](http://spambayes.sourceforge.net/)
- [Spam Filtering with Naive Bayes - Which Naive Bayes?](http://www.aueb.gr/users/ion/docs/ceas2006_paper.pdf)
- [A Bayesian Approach to Filtering Junk E-Mail](ftp://ftp.research.microsoft.com/pub/ejh/junkfilter.pdf)

## Naive Bayes

Naive Bayes has been used successful for working with email.


**Prioritizing**

**Folderizing**


## Naive Bayes

Naive Bayes has been used successful for Text Classification.

- [text classification](http://blog.datumbox.com/machine-learning-tutorial-the-naive-bayes-text-classifier/)
- [text classification](http://arxiv.org/pdf/1410.5329.pdf)
- [text classification](https://courses.cs.washington.edu/courses/csep573/11wi/lectures/20-textcat.pdf)

## Naive Bayes

Naive Bayes has been used for Sentiment Analysis.

- [Citius](http://alt.qcri.org/semeval2014/cdrom/pdf/SemEval2014026.pdf)

## Naive Bayes

Naive Bayes has been used for Intrusion Detection.

- [Network Intrusion Detection](http://www.softcomputing.net/ias10_panda.pdf)


## Naive Bayes

Naive Bayes has been used Medical Diagnosis

- [Diagnosis](http://scialert.net/fulltext/?doi=itj.2012.1166.1174&org=11)
- [Diagnosis](http://www.ijarcce.com/upload/2014/may/IJARCCE9E%20%20a%20rupali%20%20Heart%20Disease%20Prediction.pdf)

## Naive Bayes

- [Coursera Stanford Videos on Graphical Models](https://class.coursera.org/pgm-003/lecture)

## Probability

- event
- trial
- mutually exclusive
- Venn Diagrams
- joint probability
- independent events
- dependent events
- conditional probability

## Probability

- Bayes' Theorem
- prior probability
- likelihood
- posterior probability

## Bayes Theorem

$P(spam|Viagra) = \frac{P(Viagra|spam)P(spam)}{P(Viagra)}$

**prior:** $P(spam)$

**likelihood:** $P(Viagra|spam) = L(spam)$

**posterior:** $P(spam|Viagra)$

The classification is done using the posterior probability.  

The class with the *highest probability* is the *classification* for that observation/example.

## The naive Bayes algorithm

The **naive Bayes (NB)** algorithm describes a simple application using Bayes' theorem for classification.

NB is the de facto standard for much *text classification*.

See page 97/95 for the Strengths and Weaknesses.

## Why naive?

The naive Bayes algorithm is named naive because it makes a couple of "naive" assumptions about the data.

1. It assumes that all of the features in the dataset are **equally important**.
2. It assumes that all of the features in the dataset are **independent**.

## The naive Bayes classification

Naive Bayes assumes *class-conditional independence*, which means the events are independent so long as they are conditioned on the same class value.

## Using numeric features with naive Bayes

Features need to be in **categories**

- discretize
- bin
- cut points

(Reminder: All variables/features need to be **numeric** for kNN.)

Or use a different algorithm from a different package.

## Example

Next time we will work with the example in the book - filtering mobile phone (SMS) spam with the naive Bayes algorithm.