Naive Bayes Classifiers

Xác suất điều kiện (Bayes)

$$P(y│X)=\frac{P(X│y)P(y)}{P(X)}, X = (x_1, x_2, x_3, …, x_n)$$

Categorical Naive Bayes

CategoricalNB implements the categorical naive Bayes algorithm for categorically distributed data. It assumes that each feature, which is described by the index i, has its own categorical distribution.

For each feature i in the training set X, CategoricalNB estimates a categorical distribution for each feature i of X conditioned on the class y. The index set of the samples is defined as J = {1, 2, ..., m}, with m as the number of samples.

The probability of category t in feature i given class c is estimated as:

$$P(x_i=t|y=c;\alpha)=\frac{N_{tic} + \alpha}{N_c+\alpha n_i}$$

where N_tic=j ∈ J|x_ij=t, y_j=c}| is the number of times category t appears in the samples x_i, which belong to class c

N_c = |{j ∈ J|y_j=c}| is the number of samples with class c

α is a smoothing parameter

n_i is the number of available categories of feature i.

Aqua,Small,Twisted,1
Blue,Small,Pointed,0
Dune,Large,Rounded,0
Dune,Small,Rounded,1
Cyan,Large,Rounded,0
Aqua,Small,Rounded,1
Aqua,Small,Rounded,0
Cyan,Small,Pointed,1
Cyan,Small,Pointed,1
Dune,Small,Rounded,1
Dune,Small,Rounded,0
Dune,Small,Rounded,1
Dune,Small,Rounded,1
Cyan,Small,Pointed,1
Dune,Small,Rounded,1
Dune,Large,Rounded,0
Cyan,Small,Twisted,1
Blue,Small,Rounded,0
Aqua,Small,Pointed,1
Aqua,Small,Pointed,1
Dune,Small,Twisted,0
Blue,Small,Rounded,0
Dune,Small,Rounded,0
Blue,Small,Twisted,0
Dune,Small,Rounded,0
Aqua,Large,Pointed,1
Dune,Large,Rounded,0
Dune,Small,Rounded,0
Dune,Small,Rounded,0
Cyan,Large,Rounded,0
Dune,Small,Twisted,0
Dune,Large,Twisted,0
Dune,Small,Rounded,0
Dune,Small,Rounded,0
Dune,Large,Rounded,0
Aqua,Large,Rounded,1
Aqua,Small,Rounded,0
Aqua,Small,Rounded,1
Dune,Small,Rounded,0
Blue,Small,Rounded,0

Dự đoán dòng dữ liệu (Cyan, Small, Twisted) thuộc lớp (0 hay 1)

Yêu cầu

Tạo thống kê theo yêu cầu sau

Dự đoán dòng dữ liệu (Cyan, Small, Twisted) thuộc lớp (0 hay 1)

Bước tính xác suất điều kiện

$$P(0│Cyan, Small, Twisted)=P(Cyan│0)*P(Small│0)*P(Twisted│0)*P(0)=2/24*17/24*4/24*24/40= 0.005903$$

$$P(1│Cyan, Small, Twisted)=P(Cyan│1)*P(Small│1)*P(Twisted│1)*P(1)=4/16*14/16*2/16*16/40 = 0.010938$$

Bước chuẩn hóa

$$P(0│Cyan, Small, Twisted)=\frac{0.005903}{0.005903+0.010938}=0.350515$$

$$P(1│Cyan, Small, Twisted)=\frac{0.010938}{0.005903+0.010938}=0.649485$$

Phân lớp

Nếu P(1|Cyan, Small, Twisted) > P(0|Cyan, Small, Twisted) nên (Cyan, Small, Twisted) thuộc lớp 1 và ngược lại

Làm trơn Laplace

$$P(x_i|y=c;\alpha)=\frac{N_{tic}+\alpha}{N_c+\alpha n_i}$$