Linear Classifiers, Discriminant Analysis and Margin-Based Methods Short Notes for GATE DA: Concepts, Formulas, Worked Examples & Practice

Linear Classifiers, Discriminant Analysis and Margin-Based Methods short notes for GATE DA: 6 study cards covering concepts, formulas, shortcuts and exam traps, plus solved practice questions.

Exam Readiness: Fisher Discriminant Checklist

Optimal Weight Vector: $w^{*} \propto S_{W}^{- 1} (μ_{1} - μ_{2})$
Objective: Maximize $J (w) = \frac{w ^{T} S _{B} w}{w ^{T} S _{W} w}$ (Rayleigh Quotient)
Supervised vs Unsupervised: FLD uses labels ( $S_{B}, S_{W}$ ); PCA ignores labels ( $S_{T}$ ).
Dimensionality Limit: For $C$ classes, maximum meaningful projection dimensions = $C - 1$ .
Singularity Trap: If $S_{W}$ is not invertible (e.g., $N < d$ ), use regularization: $S_{W} \leftarrow S_{W} + ϵ I$ .
Decision Boundary: The threshold $w_{0}$ is typically chosen at the midpoint of the projected means: $w_{0} = - \frac{1}{2} w^{* T} (μ_{1} + μ_{2})$ (assuming equal priors and costs).

Exam Readiness: Fisher Optimization Checklist

Objective Function: Maximize $J (u) = \frac{u ^{T} S _{B} u}{u ^{T} S _{W} u}$ (Rayleigh Quotient).
Core Derivation: Setting $\nabla_{u} J (u) = 0$ yields the generalized eigenvalue problem $S_{B} u = λ S_{W} u$ .
Binary Closed-Form: $u^{*} \propto S_{W}^{- 1} (μ_{1} - μ_{2})$ .
Decision Threshold: $w_{0} = \frac{1}{2} u^{* T} (μ_{1} + μ_{2})$ (assuming equal priors/costs).
Singularity Trap: If $N < d$ , $S_{W}$ is singular. Fix via regularization: $S_{W} \leftarrow S_{W} + ϵ I$ .
Exam Shortcut: If asked for the equation satisfied by the optimal $u^{*}$ and max value $λ$ , immediately select $S_{B} u^{*} = λ S_{W} u^{*}$ .

Quick Recap: Perceptron Tracing Essentials

Augmented form:

\tilde{w} = [w, b]^{T}

\tilde{x} = [x, 1]^{T}

Mistake condition:

y_{i} (\tilde{w}^{T} \tilde{x}_{i}) \leq 0

(includes

s = 0

)

Update rule:

\tilde{w} \leftarrow \tilde{w} + y_{i} \tilde{x}_{i}

Convergence: Guaranteed in finite steps only if data is linearly separable.

Order dependence: Shuffling the dataset changes the update trajectory and final

\tilde{w}

Visual Check: Use the Convex Hull Test. If hulls of

+ 1

and

- 1

classes intersect, the data is not separable, and the algorithm will cycle indefinitely.

Next: Support Vector Machines, Margins and Support Vectors

Quick Recap: Perceptron Essentials

Quick Recap

✓ Separability:

y_{i} (w^{T} x_{i} + b) > 0

for all

i

✓ Update:

w \leftarrow w + y_{i} x_{i}

(only on mistakes)

✓ Convergence: Guaranteed if separable. Bound:

(R / γ)^{2}

✗ Failure: Non-separable data (e.g., XOR)

Next: Fisher Discriminant and Support Vector Machines

More short notes in this unit

chapter

Linear Classifiers, Discriminant Analysis and Margin-Based Methods Short Notes for GATE DA: Concepts, Formulas, Worked Examples & Practice

Linear Classifiers, Discriminant Analysis and Margin-Based Methods short notes for GATE DA: 6 study cards covering concepts, formulas, shortcuts and exam trap

Free preview ends here

Login to view the complete short notes

Creating an account is free. You get the rest of this chapter, step-by-step solutions, and a study plan built around the topics you are actually weak at.

Why MastersUp

Personalised first. High quality throughout.

Most platforms hand everyone the same content. Here the content moves with your performance, topic by topic.

Built around you, not around a syllabus PDF

Every answer you give moves your topic-level intelligence rate. The next question, the next revision card and tomorrow's plan all change with it.

Revision that hits your weak spots

We only revise topics you have actually attempted and are still below the safe bar on — never the same chapter on repeat.

Questions calibrated to the real exam

Each question carries a measured toughness. You are served a rung above your current level, so practice keeps stretching you.

Notes written for recall, not for volume

Full lesson cards for first study, curated short-note cards for the last mile — with derivations, traps and exam patterns marked.

One place for everything

Notes, chapter practice, previous-year questions, test series and full-length papers — all feeding one picture of your preparation.

Honest progress

No vanity streaks. Progress here means chapters mastered and accuracy that held up on harder questions.

Unlock the whole course

Full notes and short notes, the complete question bank with worked solutions, mock tests, full-length papers, and an adaptive plan that rebuilds itself as you improve.

View all plans Start free