Decision Trees, Naive Bayes and k-Nearest Neighbors Short Notes for GATE DA: Concepts, Formulas, Worked Examples & Practice

Decision Trees, Naive Bayes and k-Nearest Neighbors short notes for GATE DA: 6 study cards covering concepts, formulas, shortcuts and exam traps, plus solved practice questions.

k-NN Quick Revision Checklist

Type: Instance-based, lazy, non-parametric learning.
Mechanism: Store data $\to$ Calculate distances to query $\to$ Sort $\to$ Majority vote of $k$ nearest.
Distance: Euclidean is standard. Compare squared distances to save calculation time.
Hyperparameter $k$ :
- Small $k$ $\to$ High variance, overfitting, jagged boundary.
- Large $k$ $\to$ High bias, underfitting, smooth boundary.
- Use odd $k$ for binary classification to prevent ties.
Critical Preprocessing: Feature scaling (Standardization/Normalization) is mandatory due to scale sensitivity.
Complexity: Training is $O (1)$ , Prediction is $O (N \cdot d)$ where $N$ is samples and $d$ is dimensions.

Summary: k-NN Quick Recall

k-NN Quick Recall

Core Idea

Lazy, non-parametric. Predicts via majority vote of

k

nearest points.

Formula

\overset{y}{^} = ar g max_{c} \sum I (y_{i} = c)

Metrics

Euclidean (

p = 2

), Manhattan (

p = 1

). Always scale features.

Effect of k

Small

k

: noisy. Large

k

: smooth. Use odd

k

for binary.

Main Traps to Avoid

Voting ties (use odd $k$ or distance weighting).
High dimensionality (use feature selection/PCA).
Unscaled features (standardize before computing distance).

Quick Revision Checklist

Priors: $N_{k} / N$
Likelihoods (Discrete): $N_{k, j, v} / N_{k}$
Independence: Multiply individual feature likelihoods.
Parameter Count: $(C - 1) + \sum C (V_{j} - 1)$
- Shortcut: For 2 classes and binary features $\to 1 + 2 K$ .
Zero Frequency: Ruins prediction. Fix with Laplace smoothing.
Laplace Formula: Add $α$ to numerator, add $α \times V$ to denominator.

Quick Revision: Parameter Estimation

Discrete MLE: Count frequencies, divide by total.
Parameter Count: $(C - 1) + \sum C (V_{i} - 1)$ .
Continuous: Assume Gaussian, estimate mean and variance per class.
Zero Probability: Use Laplace smoothing. Add $α$ to numerator, add $α V_{i}$ to denominator.
Missing Data: Ignore instance only for the specific missing feature's conditional probability.

More short notes in this unit

chapter

Decision Trees, Naive Bayes and k-Nearest Neighbors Short Notes for GATE DA: Concepts, Formulas, Worked Examples & Practice

Decision Trees, Naive Bayes and k-Nearest Neighbors short notes for GATE DA: 6 study cards covering concepts, formulas, shortcuts and exam traps, plus solved

Free preview ends here

Login to view the complete short notes

Creating an account is free. You get the rest of this chapter, step-by-step solutions, and a study plan built around the topics you are actually weak at.

Why MastersUp

Personalised first. High quality throughout.

Most platforms hand everyone the same content. Here the content moves with your performance, topic by topic.

Built around you, not around a syllabus PDF

Every answer you give moves your topic-level intelligence rate. The next question, the next revision card and tomorrow's plan all change with it.

Revision that hits your weak spots

We only revise topics you have actually attempted and are still below the safe bar on — never the same chapter on repeat.

Questions calibrated to the real exam

Each question carries a measured toughness. You are served a rung above your current level, so practice keeps stretching you.

Notes written for recall, not for volume

Full lesson cards for first study, curated short-note cards for the last mile — with derivations, traps and exam patterns marked.

One place for everything

Notes, chapter practice, previous-year questions, test series and full-length papers — all feeding one picture of your preparation.

Honest progress

No vanity streaks. Progress here means chapters mastered and accuracy that held up on harder questions.

Unlock the whole course

Full notes and short notes, the complete question bank with worked solutions, mock tests, full-length papers, and an adaptive plan that rebuilds itself as you improve.

View all plans Start free