Naive Bayes Classifier: Algorithm & Examples

Instructor: David Gloag

David has over 40 years of industry experience in software development and information technology and a bachelor of computer science

Classification helps us make sense of the world. In this lesson, we'll take a look at a specific method, the Naive Bayes Classifier. At the end of the lesson, you should have a good understanding of this interesting technique.

Making Sense of Our World

Our world is complex. There is no question about it. The items we use have many moving parts, the organizations we belong to have many rules, and the relationships we foster have many subtleties. Is it a wonder that we are constantly looking for ways to simplify, and understand, our world? Not really.

The inherent complexity dictates that we organize, and classify, various aspect of our lives. We do this so that we can quickly understand and adjust to the properties of new elements as they are introduced.

What Is Classification?

Classification is a grouping process. It organizes things together based on how similar they are to each other. For example, let's say we have 5 vehicles; a Honda, an Acura, a Viper, a Hummer, and a Jeep. Further, let's create groupings for these vehicles. There are a number of possibilities, but let's go for the simplest; cars, and trucks. The Honda, Acura, and Viper fall into the car classification. The Hummer and Jeep fall into the truck classification.

Why would we do this? Well, the classification tells us something about the vehicles, simply by belonging to the grouping or classification. Cars generally have a smoother ride than trucks, and trucks perform better off-road than cars, for example. This is common knowledge about the class. We don't have to know the details about each vehicle to get a general understanding.

What Is Probability?

Probability is the mathematical likelihood that something will occur. It is determined by taking the number of elements in a specific class, and dividing it by the total number of elements in all classes. Let's consider again the vehicle example.

The probability of a car is 3/5 or 0.6, similarly, the probability of a truck is 2/5 or 0.4. This is helpful in predicting the class of the next vehicle, but there is an inherent weakness. It doesn't take into account that the classification of the next vehicle affects the probabilities. If it's a car, then the probabilities change to 4/6 or 0.66, and 2/6 or 0.33 respectively. If it's a truck, then the probabilities change to 3/6 or 0.5, and 3/6 or 0.5 respectively.

How Does Naive Bayes Classifier Work?

The Naive Bayes Classifier is a technique in probability that also tries to group or classify new items. The main difference is that it not only considers the probability of the class, it takes into account the details about the new item.

In the vehicle example, the probability of a new item being a car is 0.6, and a truck is 0.4, so the new item is likely a car. But the Naive Bayes Classifier would also look at the details of the new vehicle, a Land Rover. It would see that the vehicle has off-road tires. Comparing this to the vehicles, it would see that 2 trucks have this characteristic, and 0 cars. Thus, the detail probabilities would be 0/3 or 0.0 for cars, and 1/1 for trucks.

To unlock this lesson you must be a Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use

Become a member and start learning now.
Become a Member  Back
What teachers are saying about
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account