**Probability for Learning**

Probability for classification and modeling concepts.

Bayesian probability

- Notion of probability interpreted as partial belief

Bayesian Estimation

- It calculate the validity of a proposition

- Based on prior estimate of its probability

- and New relevant evidence

**Bayes Theorem**

Goal: To determine the most probable hypothesis, given the data D plus any initial knowledge about the prior probabilities of the various hypotheses in H.

**Bayes Rule :**

P(h|D) = P(D|h)P(h)/P(D)

P(h) = prior probability of hypothesis h

P(D) = prior probability of training data D

P(h|D) = probability of h given D (posterior density)

P(D|h) = probability of D given h (likelihood of D given h)

**An Example**

Does patient have cancer or not ?

A patient takes a lab test and the result comes back positive. The test returns a correct positive result in only 98% of the cases in which the disease is actually present, and a correct negative result in only 97% of the cases in which disease is not present. Furthermore, .008 of the entire population have this cancer.

Maximum A Posteriori (MAP) Hypothesis

P(h|D) = P(D|h)P(h)/P(D)

The Goal of Bayesian Learning: the most probable hypothesis given the training data (Maximum A Posteriori hypothesis)

**Compute ML Hypo**

**Bayes Optimal Classifier**

Question: Given new instance x, what is its most probable classification?

hMAP (x) is not the most probable classification!

Example: Let P(h1|D) = .4,

P(h2|D) = .3,

P(h3|D) = .3

Given new data x, we have h1(x)=+, h2(x) = -, h3 = -

What is the most probable classification of x?

Bayes optimal classification:

Where V is the set of all the values a classification can take and vj is one possible such classification.

Example:

## No comments:

## Post a Comment