Orthogonality, Projections and Linear Systems Short Notes for GATE DA

Orthogonality, Projections and Linear Systems short notes for GATE DA: 2 study cards covering concepts, formulas, shortcuts and exam traps, plus solved practice questions.

Summary: Projection Matrix Master Sheet

Core Properties

P^{2} = P

Idempotent

P^{T} = P

Symmetric

Eigenvalues $λ \in {0, 1}$
$Trace (P) = Rank (P) = dim (C (P))$

Subspaces

$C (P) = U$ (Target subspace)
$N (P) = U^{⊥}$ (Orthogonal complement)
$R^{n} = C (P) \oplus N (P)$

Formulas

Line (vector $a$ ):

P = \frac{a a ^{T}}{a ^{T} a}

Subspace (matrix $A$ ):

P = A (A^{T} A)^{- 1} A^{T}

Complement: Projection onto

U^{⊥}

P^{⊥} = I - P

Summary: Consistency Master Sheet

Quick Summary

Consistency Master Sheet

Consistency Checklist

Consistent: $Rank (A) = Rank ([A ∣ b])$
Inconsistent: $Rank (A) < Rank ([A ∣ b])$

Counting Solutions (If Consistent)

Unique: $Rank (A) = n$
Infinite: $Rank (A) < n$ (Free variables = $n - Rank (A)$ )

Structure of Infinite Solutions

x = x_{p} + x_{h}

(Particular solution + Null space basis)

Golden Rules

$det (A) = 0$ does not mean no solution; it means not unique. Check $[A ∣ b]$ .
Free variables depend on $n$ (columns), not $m$ (rows).

Orthogonality, Projections and Linear Systems: Solved Questions with Step-by-Step Explanations (10 Problems)

Question 1 · Linear Algebra MCQ

A linear system $A x = b$ is consistent if and only if:

A.
$Rank (A) = Rank ([A ∣ b])$
B.
$Rank (A) < Rank ([A ∣ b])$
C.
$det (A) \neq = 0$
D.
$b = 0$

Correct Answer:

Step-by-Step Solution

Key idea: This is a direct-recall question about the fundamental condition for consistency.

Step 1: Consistency means there exists at least one solution $x$ .

Step 2: Geometrically, this means $b$ lies in the column space of $A$ .

Step 3: Algebraically, appending $b$ to $A$ to form the augmented matrix $[A ∣ b]$ adds a new column.

Step 4: If $b$ is already in the column space of $A$ , it does not increase the dimension of the column space. Thus, the rank remains unchanged.

Step 5: If $b$ is NOT in the column space, it adds a new independent direction, increasing the rank by 1.

Step 6: Therefore, consistency is equivalent to $Rank (A) = Rank ([A ∣ b])$ .

Answer: A

Question 2 · Linear Algebra MCQ

Which of the following pairs of algebraic properties uniquely defines an orthogonal projection matrix $P$ ?

A.
$P^{2} = I$ and $P^{T} = P$
B.
$P^{2} = P$ and $P^{T} = - P$
C.
$P^{2} = P$ and $P^{T} = P$
D.
$P^{2} = I$ and $P^{T} = - P$

Correct Answer:

Step-by-Step Solution

Key idea: This is a direct recall question about the defining algebraic properties of orthogonal projection matrices.

Step 1: Recall the geometric meaning. An orthogonal projection drops a perpendicular to a target subspace.

Step 2: Algebraically, projecting a vector twice is the same as projecting it once, because the first projection already lands in the subspace. This gives idempotence: $P^{2} = P$ .

Step 3: The projection is "orthogonal", meaning the error vector is perpendicular to the subspace. This geometric requirement translates to the matrix being symmetric: $P^{T} = P$ .

Answer: $P^{2} = P$ and $P^{T} = P$ (Option C).

Question 3 · Linear Algebra MCQ

An orthogonal projection matrix $P \in R^{n \times n}$ has rank $k$ . What are its eigenvalues, and what is its trace?

A.
Eigenvalues are $0$ and $1$ ; Trace is $k$
B.
Eigenvalues are $- 1$ and $1$ ; Trace is $0$
C.
Eigenvalues are $0$ and $1$ ; Trace is $n$
D.
Eigenvalues are $1$ and $k$ ; Trace is $k$

Correct Answer:

Step-by-Step Solution

Key idea: This question tests the spectral properties (eigenvalues and trace) of projection matrices.

Step 1: Since $P^{2} = P$ , any eigenvalue $λ$ must satisfy $λ^{2} = λ$ . Thus, the only possible eigenvalues are $λ \in {0, 1}$ .

Step 2: The rank of $P$ is the dimension of its column space, which equals the number of non-zero eigenvalues. So there are exactly $k$ eigenvalues equal to $1$ , and the remaining $n - k$ eigenvalues are $0$ .

Step 3: The trace is the sum of the eigenvalues: $k \times 1 + (n - k) \times 0 = k$ .

Answer: Eigenvalues are $0$ and $1$ ; Trace is $k$ (Option A).

Question 4 · Linear Algebra MCQ

A consistent system of linear equations with $n$ unknowns has infinitely many solutions if:

A.
$Rank (A) = n$
B.
$Rank (A) < n$
C.
$Rank (A) > n$
D.
$Rank (A) = 0$

Correct Answer:

Step-by-Step Solution

Key idea: This is a direct-recall question linking rank to the number of solutions.

Step 1: For a consistent system, the number of solutions is determined by the number of free variables.

Step 2: Number of free variables $= n - Rank (A)$ .

Step 3: If there are no free variables ( $n - Rank (A) = 0 ⟹ Rank (A) = n$ ), the solution is unique.

Step 4: If there is at least one free variable ( $n - Rank (A) \geq 1 ⟹ Rank (A) < n$ ), the system has infinitely many solutions.

Step 5: $Rank (A) > n$ is impossible since rank cannot exceed the number of columns.

Answer: B

Question 5 · Linear Algebra MCQ

For the system of linear equations $x + y = 2$ and $2 x + 2 y = k$ to have infinitely many solutions, what must be the value of the parameter $k$ ?

A.
4
B.
2
C.
0
D.
Any real number

Correct Answer:

Step-by-Step Solution

Key idea: This is a direct-formula application question involving a parameter that determines consistency.

Step 1: Write the augmented matrix for the system:

$[1212 ∣ ∣ 2 k]$

Step 2: Perform row reduction to echelon form. Apply the operation $R_{2} \to R_{2} - 2 R_{1}$ :

$[1010 ∣ ∣ 2 k - 4]$

Step 3: Analyze the second row. It represents the equation $0 x + 0 y = k - 4$ , or simply $0 = k - 4$ .

Step 4: For the system to have infinitely many solutions, it must first be consistent. This requires the second row to not be a contradiction. Thus, we must have $k - 4 = 0$ , which gives $k = 4$ .

Step 5: If $k = 4$ , the second row becomes $0 = 0$ , leaving one independent equation with two variables, resulting in infinitely many solutions.

Answer: A

Question 6 · Linear Algebra MCQ

The system of equations $x + 2 y = 3$ and $2 x + 4 y = k$ has infinitely many solutions. What is the value of $k$ ?

A.
3
B.
5
C.
6
D.
8

Correct Answer:

Step-by-Step Solution

Key idea: This is a direct-formula application question involving a parameter that determines consistency.

Step 1: Write the augmented matrix for the system:

$[1224 ∣ ∣ 3 k]$

Step 2: Perform row reduction to echelon form. Apply the operation $R_{2} \to R_{2} - 2 R_{1}$ :

$[1020 ∣ ∣ 3 k - 6]$

Step 3: Analyze the second row. It represents the equation $0 x + 0 y = k - 6$ , or simply $0 = k - 6$ .

Step 4: For the system to have infinitely many solutions, it must first be consistent. This requires the second row to not be a contradiction. Thus, we must have $k - 6 = 0$ , which gives $k = 6$ .

Step 5: If $k = 6$ , the second row becomes $0 = 0$ , leaving one independent equation with two variables, resulting in infinitely many solutions.

Answer: C

Question 7 · Linear Algebra MCQ

Let $P$ be the orthogonal projection matrix onto a subspace $U \subseteq R^{n}$ . Which of the following correctly describes the null space of $P$ ?

A.
The subspace $U$ itself
B.
The orthogonal complement $U^{⊥}$
C.
The entire space $R^{n}$
D.
The trivial subspace ${0}$

Correct Answer:

Step-by-Step Solution

Key idea: This question tests the fundamental subspaces of a projection matrix.

Step 1: The column space $C (P)$ is the set of all possible outputs of $P$ . Since $P$ projects onto $U$ , $C (P) = U$ .

Step 2: The null space $N (P)$ is the set of vectors that map to $0$ . Geometrically, these are the vectors that are completely "flattened" or removed by the projection.

Step 3: The vectors removed are exactly those orthogonal to the target subspace $U$ . Thus, $N (P) = U^{⊥}$ .

Answer: The orthogonal complement $U^{⊥}$ (Option B).

Question 8 · Linear Algebra MCQ

Let $a = (21)$ . What is the orthogonal projection matrix $P$ onto the line spanned by $a$ ?

A.
$egin{pmatrix} 1/5 & 2/5 \ 2/5 & 4/5 nd{pmatrix}$
B.
$egin{pmatrix} 4/5 & 2/5 \ 2/5 & 1/5 nd{pmatrix}$
C.
$egin{pmatrix} 4/5 & 1/5 \ 1/5 & 1/5 nd{pmatrix}$
D.
$egin{pmatrix} 2/5 & 4/5 \ 4/5 & 1/5 nd{pmatrix}$

Correct Answer:

Step-by-Step Solution

Key idea: This question requires direct substitution into the formula for the projection matrix onto a 1D subspace (a line).

Step 1: The formula for the projection matrix onto the line spanned by $a$ is $P = \frac{a a ^{T}}{a ^{T} a}$ .

Step 2: Calculate the denominator: $a^{T} a = 2^{2} + 1^{2} = 4 + 1 = 5$ .

Step 3: Calculate the numerator: $a a^{T} = (21) (21) = (4221)$ .

Step 4: Divide by the denominator: $P = \frac{1}{5} (4221) = (4/5 2/5 2/5 1/5)$ .

Answer: Option B.

Question 9 · Linear Algebra NAT

Let $P$ be the orthogonal projection matrix onto the line spanned by the vector $a = (34)$ in $R^{2}$ . What is the trace of $P$ ?

Correct Answer:

Step-by-Step Solution

Key idea: This question tests the spectral property that the trace of a projection matrix equals its rank.

Step 1: The matrix $P$ is a projection onto a line. A line in $R^{2}$ is a 1-dimensional subspace.

Step 2: The rank of a projection matrix is equal to the dimension of its column space, which is the target subspace. Thus, $rank (P) = 1$ .

Step 3: The eigenvalues of any projection matrix are strictly $0$ and $1$ . The number of $1$ s equals the rank. So, $P$ has one eigenvalue of $1$ and one eigenvalue of $0$ .

Step 4: The trace is the sum of the eigenvalues: $1 + 0 = 1$ .

Answer: 1

Question 10 · Linear Algebra MCQ

Let $A$ be an $m \times n$ matrix with full column rank. Which of the following expressions gives the orthogonal projection matrix onto the column space of $A$ ?

A.
$A (A^{T} A)^{- 1} A^{T}$
B.
$(A^{T} A)^{- 1} A^{T}$
C.
$A^{T} (A A^{T})^{- 1} A$
D.
$A (A^{T} A) A^{T}$

Correct Answer:

Step-by-Step Solution

Key idea: This question tests the standard formula for constructing a projection matrix onto a general subspace defined by the columns of a matrix $A$ .

Step 1: The projection of a vector $b$ onto the column space of $A$ is given by $p = A x$ , where $x$ solves the normal equations $A^{T} A x = A^{T} b$ .

Step 2: Since $A$ has full column rank, $A^{T} A$ is invertible. Thus, $x = (A^{T} A)^{- 1} A^{T} b$ .

Step 3: Substitute $x$ back into $p$ :

$p = A ((A^{T} A)^{- 1} A^{T} b) = (A (A^{T} A)^{- 1} A^{T}) b$

Step 4: The matrix that maps $b$ to $p$ is the projection matrix $P = A (A^{T} A)^{- 1} A^{T}$ .

Answer: Option A.