Decision Trees, Naive Bayes and k-Nearest Neighbors Notes for GATE DA: Concepts, Formulas, Worked Examples & Practice

Decision Trees, Naive Bayes and k-Nearest Neighbors notes for GATE DA: 70 study cards covering concepts, formulas, shortcuts and exam traps, plus solved practice questions.

k-Nearest Neighbors Classification

"Tell me who your neighbors are, and I will tell you who you are."

Supervised Learning → Decision Trees, Naive Bayes & k-NN

What you will learn here

The core intuition behind instance-based, lazy learning.
Step-by-step distance calculation and majority voting.
Critical exam traps: choice of k, feature scaling, and decision boundaries.

The Core Intuition: Lazy Learning

k-Nearest Neighbors (k-NN) is an instance-based or lazy learning algorithm.

No explicit training phase: The algorithm does not learn a generalized function or estimate parameters. It merely stores the entire training dataset in memory.
Prediction is the real work: All computation is deferred until a prediction is required for a new, unseen query point.
The fundamental assumption: Data points that are close to each other in the feature space are likely to belong to the same class.

Measuring Closeness: Distance Metrics

To identify the "nearest" neighbors, we must quantify the distance between a query point $x_{q}$ and a training point $x_{i}$ .

1. Euclidean Distance

The straight-line distance between two points in an $n$ -dimensional space:

d (x_{q}, x_{i}) = j = 1 \sum n (x_{q, j} - x_{i, j})^{2}

2. Manhattan Distance

The sum of absolute differences along each dimension:

d (x_{q}, x_{i}) = j = 1 \sum n ∣ x_{q, j} - x_{i, j} ∣

*Note: Exams predominantly test Euclidean distance in 2D or 3D spaces.

The k-NN Classification Algorithm

To classify a new query point $x_{q}$ , follow these exact steps:

Calculate Distances:

Compute the distance (usually Euclidean) between $x_{q}$ and every point $x_{i}$ in the training dataset.

Sort Distances:

Arrange the computed distances in ascending order, keeping track of the corresponding class labels.

Select k Neighbors:

Pick the top $k$ points with the smallest distances.

Majority Vote:

Count the frequency of each class among these $k$ neighbors. Assign $x_{q}$ to the class with the highest frequency.

Tie-breaking rule: If there is a tie, common conventions include reducing $k$ by 1, using distance-weighted voting, or relying on the specific rule stated.

More notes in this unit

chapter

Decision Trees, Naive Bayes and k-Nearest Neighbors Notes for GATE DA: Concepts, Formulas, Worked Examples & Practice

Decision Trees, Naive Bayes and k-Nearest Neighbors notes for GATE DA: 70 study cards covering concepts, formulas, shortcuts and exam traps, plus solved pract

Free preview ends here

Login to view the complete notes

Creating an account is free. You get the rest of this chapter, step-by-step solutions, and a study plan built around the topics you are actually weak at.

Why MastersUp

Personalised first. High quality throughout.

Most platforms hand everyone the same content. Here the content moves with your performance, topic by topic.

Built around you, not around a syllabus PDF

Every answer you give moves your topic-level intelligence rate. The next question, the next revision card and tomorrow's plan all change with it.

Revision that hits your weak spots

We only revise topics you have actually attempted and are still below the safe bar on — never the same chapter on repeat.

Questions calibrated to the real exam

Each question carries a measured toughness. You are served a rung above your current level, so practice keeps stretching you.

Notes written for recall, not for volume

Full lesson cards for first study, curated short-note cards for the last mile — with derivations, traps and exam patterns marked.

One place for everything

Notes, chapter practice, previous-year questions, test series and full-length papers — all feeding one picture of your preparation.

Honest progress

No vanity streaks. Progress here means chapters mastered and accuracy that held up on harder questions.

Unlock the whole course

Full notes and short notes, the complete question bank with worked solutions, mock tests, full-length papers, and an adaptive plan that rebuilds itself as you improve.

View all plans Start free