Random Variables

Overview

In our preceding discussions, we established the foundational principles of probability theory by examining sample spaces and events. However, to perform a rigorous quantitative analysis of random phenomena, we must bridge the gap between abstract outcomes and numerical values. This chapter introduces the concept of a random variable, a fundamental construct that assigns a numerical value to every possible outcome of a random experiment. The formalization of this concept is a cornerstone of modern statistics and machine learning, allowing us to apply the powerful tools of mathematical analysis to uncertain events.

We shall begin by defining discrete and continuous random variables and exploring their associated probability distributions. Subsequently, we will investigate the essential characteristics of these distributions through measures of central tendency and dispersion. We will learn to compute and interpret quantities such as the expected value ($E[X]$) and variance ($\operatorname{Var}(X)$), which provide concise summaries of a variable's behavior. The chapter then progresses to the study of relationships between multiple random variables, introducing covariance and correlation as measures of linear dependence. A firm grasp of these concepts is indispensable for understanding feature interactions in data analysis and machine learning models, a topic of significant importance for the GATE examination.

Finally, we culminate our study with an examination of conditional expectation and variance. This advanced topic addresses how our knowledge or assumptions about one event or variable can alter our expectations about another. This principle forms the basis for many predictive models and inferential techniques encountered in the Data Science and AI syllabus. A thorough understanding of the material presented herein is therefore critical for solving complex problems and building a robust theoretical foundation for subsequent topics.

---

Chapter Contents

| # | Topic | What You'll Learn |
|---|-------|-------------------|
| 1 | Definition of Random Variables | Formalizing numerical outcomes of random experiments. |
| 2 | Measures of Central Tendency and Dispersion | Quantifying the center and spread of distributions. |
| 3 | Correlation and Covariance | Analyzing linear dependence between random variables. |
| 4 | Conditional Expectation and Variance | Updating expectations with new information provided. |

---

Learning Objectives

---

We now turn our attention to Definition of Random Variables...
## Part 1: Definition of Random Variables

Introduction

In our study of probability, we are often concerned not with the specific outcomes of an experiment, but rather with some numerical property associated with those outcomes. For instance, in an experiment involving the tossing of two coins, we might be more interested in the number of heads that appear than in the exact sequence of heads and tails. This need to associate a numerical value with each outcome of a random experiment leads us to the fundamental concept of a random variable.

A random variable provides a means of mapping the, often non-numerical, outcomes in a sample space to a set of real numbers. This transformation is crucial as it allows us to apply the powerful tools of calculus and mathematical analysis to the study of probability and statistics. We can analyze distributions, calculate expected values, and determine variances, all of which are central to data analysis and inference. Understanding the formal definition and classification of random variables is the first essential step in this direction.

We use a capital letter, $X$, to represent the random variable as a function, and a lowercase letter, $x$, to represent a specific value that the random variable can take. The set of all possible values of $X$ is called the range or support of the random variable.

---

Key Concepts

The most fundamental classification of random variables is based on the nature of the values they can assume. This leads to two primary types: discrete and continuous random variables.

#
## 1. Discrete Random Variables

A random variable is said to be discrete if its range is finite or countably infinite. This means that the variable can only take on a specific, separated set of values. There are "gaps" between the possible values.

Consider the experiment of rolling a standard six-sided die. The sample space is $S = \{1, 2, 3, 4, 5, 6\}$. If we define a random variable $X$ as the outcome of the roll, then $X$ can take values from the set $\{1, 2, 3, 4, 5, 6\}$. Since this set is finite, $X$ is a discrete random variable. Similarly, the number of defective items in a batch of 100 is a discrete random variable, as it can take integer values from 0 to 100.

Sample Space (S)

HH
HT
TH
TT

Real Numbers (ℝ)

2

1

0

X = No. of Heads

Worked Example:

Problem: An experiment consists of tossing two fair coins. Let the random variable $X$ be defined as the number of heads observed. Determine the sample space $S$ and the set of possible values for $X$.

Solution:

Step 1: Define the sample space $S$.
The sample space consists of all possible outcomes of tossing two coins. Let H denote Heads and T denote Tails.

Step 2: Apply the function $X$ to each outcome in $S$.
The random variable $X$ counts the number of heads in each outcome.

For the outcome $HH$, the number of heads is 2. So, $X(HH) = 2$.

For the outcome $HT$, the number of heads is 1. So, $X(HT) = 1$.

For the outcome $TH$, the number of heads is 1. So, $X(TH) = 1$.

For the outcome $TT$, the number of heads is 0. So, $X(TT) = 0$.

Step 3: List the set of all possible values for $X$.
The range of the random variable $X$ is the set of all unique numerical values it can take.

Answer: The sample space is $S = \{HH, HT, TH, TT\}$ and the random variable $X$ can take values from the set $\{0, 1, 2\}$. Since this set is finite, $X$ is a discrete random variable.

---

#
## 2. Continuous Random Variables

A random variable is said to be continuous if its range is an interval or a collection of intervals on the real number line. This means the variable can take on any value within a given range, and there are uncountably infinite possible values.

For example, if we define a random variable $Y$ as the height of a randomly selected student, $Y$ can take any value within a certain range, say $[150 \text{ cm}, 190 \text{ cm}]$. It is not restricted to integer values; a height of $175.342$ cm is perfectly possible. Other examples include temperature, weight, and time. For a continuous random variable, the probability of it taking any single specific value is zero, i.e., $P(Y=y) = 0$. We instead focus on the probability that the variable falls within a certain interval.

---

Problem-Solving Strategies

---

---

Common Mistakes

---

Practice Questions

:::question type="MCQ" question="Which of the following is an example of a discrete random variable?" options=["The height of a building", "The time taken to complete a race", "The number of defective items in a shipment of 1000 items", "The temperature of a room in Celsius"] answer="The number of defective items in a shipment of 1000 items" hint="A discrete variable's values can be counted. Which of the options represents a counted quantity rather than a measured one?" solution="Let's analyze the options.

• The height of a building is a measurement and can take any value within a range, making it continuous.


• The time taken to complete a race is also a measurement and is continuous.


• The number of defective items is a count (0, 1, 2, ..., 1000). This set of values is finite and countable. Therefore, it is a discrete random variable.


• The temperature of a room is a measurement and is continuous.

Answer: \boxed{\text{The number of defective items in a shipment of 1000 items}}"
:::

:::question type="NAT" question="A box contains 4 red and 3 green balls. An experiment consists of drawing 2 balls from the box without replacement. Let the random variable Y be the number of green balls drawn. What is the number of distinct values that Y can take?" answer="3" hint="Consider the minimum and maximum number of green balls you can possibly draw in this experiment." solution="Let Y be the number of green balls drawn.
The experiment involves drawing 2 balls from a total of 7.

• We could draw 0 green balls (meaning 2 red balls are drawn). This is possible since there are 4 red balls. So, Y can be 0.


• We could draw 1 green ball and 1 red ball. This is possible. So, Y can be 1.


• We could draw 2 green balls. This is possible since there are 3 green balls. So, Y can be 2.


• We cannot draw 3 green balls, as we are only drawing 2 balls in total.

The set of possible values for Y is $\{0, 1, 2\}$.
The number of distinct values is 3.
Answer: \boxed{3}"
:::

:::question type="MSQ" question="Let S be the sample space of a random experiment. Let X be a random variable defined as a function $X: S \to \mathbb{R}$. Which of the following statements are ALWAYS true?" options=["X is always a discrete random variable.", "The range of X is a subset of the real numbers.", "If the sample space S is finite, then X must be a discrete random variable.", "If X is a continuous random variable, the sample space S must be infinite."] answer="The range of X is a subset of the real numbers.,If the sample space S is finite, then X must be a discrete random variable." hint="Review the fundamental definition of a random variable and the properties of discrete vs. continuous variables." solution="Let's evaluate each statement:

'X is always a discrete random variable.' This is false, as continuous random variables (e.g., height, temperature) exist.

'The range of X is a subset of the real numbers.' This is true by the formal definition of a random variable, which maps outcomes from the sample space $S$ to the set of real numbers $\mathbb{R}$.

'If the sample space S is finite, then X must be a discrete random variable.' This is true. A function defined on a finite set can only produce a finite number of distinct outputs. A random variable with a finite range is, by definition, discrete.

'If X is a continuous random variable, the sample space S must be infinite.' This is true. A continuous random variable has an uncountable range (an interval). A function cannot map a finite or countably infinite domain to an uncountable range. Therefore, the sample space $S$ must be infinite.

Based on the fundamental definitions, statements 2 and 3 are the most direct and fundamental consequences of the definition of a random variable.

Answer: \boxed{\text{The range of X is a subset of the real numbers.,If the sample space S is finite, then X must be a discrete random variable.}}"
:::

---

Summary

---

What's Next?

---

---

Part 2: Measures of Central Tendency and Dispersion

Introduction

In the study of random variables, it is often insufficient to know only the full probability distribution. For many applications in data analysis and statistical inference, we require concise numerical summaries that describe the essential features of the distribution. These summaries are broadly categorized into two types: measures of central tendency and measures of dispersion.

Measures of central tendency aim to identify a single value that represents the "center" or "typical" value of a random variable. The most common of these is the mean, or expected value, which provides a long-run average of the outcomes. Measures of dispersion, conversely, quantify the variability or spread of the random variable's possible values around this central point. The primary measures here are the variance and its square root, the standard deviation. A thorough understanding of these measures is not merely a procedural exercise; it is fundamental to interpreting probabilistic models and making informed decisions based on data. In the context of the GATE examination, questions frequently test the direct calculation of these measures as well as their known properties for standard probability distributions.

---

---

Measures of Central Tendency

These measures provide a single value that attempts to describe a set of data by identifying the central position within that set of data.

1. Mean (Expected Value)

The most important measure of central tendency is the mean, also known as the expected value. For a random variable $X$, its expected value, denoted as $E[X]$ or $\mu_X$, represents the weighted average of all possible values that $X$ can take, where the weights are the corresponding probabilities.

For a discrete random variable $X$ with a set of possible values $S$ and a probability mass function (PMF) $p(x)$, the expected value is the sum of each value multiplied by its probability.

For a continuous random variable $X$ with a probability density function (PDF) $f(x)$, the expected value is found by integrating the product of $x$ and $f(x)$ over the entire range of $X$.

Worked Example:

Problem: A discrete random variable $X$ has the following probability mass function: $P(X=1) = 0.2$, $P(X=2) = 0.5$, and $P(X=3) = 0.3$. Calculate the mean of $X$.

Solution:

Step 1: Identify the values of $x$ and their corresponding probabilities $p(x)$.
The values are $x_1=1$, $x_2=2$, $x_3=3$.
The probabilities are $p(1)=0.2$, $p(2)=0.5$, $p(3)=0.3$.

Step 2: Apply the formula for the expected value of a discrete random variable.

Step 3: Substitute the values and compute the sum.

Step 4: Calculate the final result.

Answer: The mean of the random variable $X$ is $2.1$.

2. Median

The median is the value that separates the higher half from the lower half of a probability distribution. For a continuous random variable $X$ with cumulative distribution function (CDF) $F_X(x)$, the median $m$ is the value such that $P(X \le m) = 0.5$.

We observe that the median is less sensitive to extreme values (outliers) than the mean, a property known as robustness.

3. Mode

The mode of a random variable is the value at which its PMF or PDF takes its maximum value. A distribution can have one mode (unimodal), two modes (bimodal), or more (multimodal).

---

Measures of Dispersion

While central tendency tells us about the location of a distribution, measures of dispersion describe its spread or variability.

1. Variance

Variance is the most common measure of dispersion. It quantifies the spread of a random variable's values around its mean. Specifically, it is the expected value of the squared deviation from the mean.

While the definition is intuitive, a more practical formula, often called the computational formula, is used for calculations. We can derive it as follows:

By linearity of expectation,

Since $\mu$ is a constant, $E[2\mu X] = 2\mu E[X] = 2\mu^2$ and $E[\mu^2] = \mu^2$.

2. Standard Deviation

The standard deviation is simply the positive square root of the variance. Its primary advantage is that it is expressed in the same units as the random variable, making it more interpretable than the variance.

Worked Example:

Problem: For the discrete random variable $X$ from the previous example with $P(X=1) = 0.2$, $P(X=2) = 0.5$, and $P(X=3) = 0.3$, calculate the variance and standard deviation. We already found that $E[X] = 2.1$.

Solution:

Step 1: Calculate $E[X^2]$ using its definition for a discrete random variable.

Step 2: Substitute the values.

Step 3: Apply the computational formula for variance.

Step 4: Substitute the calculated values of $E[X^2]$ and $E[X]$.

Step 5: Calculate the standard deviation by taking the square root of the variance.

Answer: The variance is $0.49$ and the standard deviation is $0.7$.

---

Mean and Variance of Standard Distributions

For the GATE exam, it is imperative to know the mean and variance of several standard probability distributions by heart. Direct application of these formulas can save significant time.

| Distribution | Parameters | Mean ($E[X]$) | Variance ($Var(X)$) |
| :--- | :--- | :--- | :--- |
| Binomial | $n$ (trials), $p$ (success prob.) | $np$ | $np(1-p)$ |
| Poisson | $\lambda$ (rate) | $\lambda$ | $\lambda$ |
| Uniform (Continuous) | $a$ (min), $b$ (max) | $\frac{a+b}{2}$ | $\frac{(b-a)^2}{12}$ |
| Exponential | $\lambda$ (rate) | $\frac{1}{\lambda}$ | $\frac{1}{\lambda^2}$ |
| Normal | $\mu$ (mean), $\sigma^2$ (variance) | $\mu$ | $\sigma^2$ |
| Standard Normal | $\mu=0$, $\sigma^2=1$ | $0$ | $1$ |

---

Problem-Solving Strategies

A strategic approach is essential for solving problems under time constraints.

---

Common Mistakes

Awareness of common pitfalls can prevent the loss of valuable marks.

---

Practice Questions

:::question type="MCQ" question="A random variable $X$ has the probability mass function $P(X=0) = 1/3$, $P(X=1) = 1/2$, and $P(X=2) = 1/6$. What is the variance of the random variable $Y = 2X - 3$?" options=["17/36","17/9","34/9","5/6"] answer="17/9" hint="First, find the variance of $X$ using the computational formula $Var(X) = E[X^2] - (E[X])^2$. Then, use the property $Var(aX+b) = a^2Var(X)$." solution="
Step 1: Calculate the mean of $X$, $E[X]$.

Step 2: Calculate $E[X^2]$.

Step 3: Calculate the variance of $X$, $Var(X)$.

Step 4: Calculate the variance of $Y = 2X - 3$.

Result: The variance of $Y$ is $17/9$.
"
:::

:::question type="NAT" question="A continuous random variable $X$ has a probability density function given by $f(x) = \frac{3}{8}x^2$ for $0 \le x \le 2$, and $f(x)=0$ otherwise. Calculate the mean of $X$." answer="1.5" hint="Use the formula for the expected value of a continuous random variable: $E[X] = \int x \cdot f(x) \,dx$ over the defined range." solution="
Step 1: Set up the integral for the expected value, $E[X]$.

Step 2: Substitute the given PDF and adjust the integration limits.

Step 3: Simplify the integrand.

Step 4: Evaluate the integral.

Step 5: Substitute the limits of integration.

Result:

" :::

:::question type="MSQ" question="Let $X$ be a random variable with mean $E[X] = 10$ and variance $Var(X) = 4$. Let a new random variable be defined as $Y = 5 - 2X$. Which of the following statements is/are correct?" options=["The mean of Y is -15.","The variance of Y is -8.","The variance of Y is 16.","The standard deviation of Y is 4."] answer="The mean of Y is -15.,The variance of Y is 16.,The standard deviation of Y is 4." hint="Apply the properties of expectation and variance for a linear transformation. $E[aX+b] = aE[X]+b$ and $Var(aX+b) = a^2Var(X)$." solution="
Let us analyze each statement.
The transformation is $Y = 5 - 2X$. Here, $a = -2$ and $b = 5$.

Statement 1: The mean of Y is -15.
Using the property of expectation:

Substituting $E[X] = 10$:

Thus, this statement is correct.

Statement 2: The variance of Y is -8.
Variance can never be negative. Thus, this statement is incorrect.

Statement 3: The variance of Y is 16.
Using the property of variance:

Substituting $Var(X) = 4$:

Thus, this statement is correct.

Statement 4: The standard deviation of Y is 4.
The standard deviation is the square root of the variance.

Thus, this statement is correct.

Therefore, the correct options are: "The mean of Y is -15.", "The variance of Y is 16.", and "The standard deviation of Y is 4."
"
:::

:::question type="NAT" question="The number of defects on a semiconductor wafer follows a Poisson distribution with a mean of 2 defects per wafer. What is the standard deviation of the number of defects per wafer?" answer="1.414" hint="Recall the key property of a Poisson distribution regarding its mean and variance." solution="
Step 1: Identify the distribution and its parameter.
The problem states that the number of defects follows a Poisson distribution. The mean is given as 2 defects per wafer. For a Poisson distribution, the parameter $\lambda$ is equal to the mean.
So, $\lambda = 2$.

Step 2: Recall the formula for the variance of a Poisson distribution.
For a Poisson random variable $X$ with parameter $\lambda$, the variance is given by:

Step 3: Calculate the variance.

Step 4: Calculate the standard deviation.
The standard deviation $\sigma$ is the square root of the variance.

Step 5: Compute the numerical value.

Result: The standard deviation, rounded to three decimal places, is 1.414.
"
:::

:::question type="MCQ" question="A fair six-sided die is rolled 10 times. Let $X$ be the number of times the number '4' appears. What is the variance of $X$?" options=["50/36","5/3","25/18","5/6"] answer="25/18" hint="Recognize that this scenario describes a Binomial experiment. Identify the parameters $n$ and $p$, and then use the formula for the variance of a Binomial distribution." solution="
Step 1: Identify the type of random variable.
The experiment consists of a fixed number of independent trials ($n=10$). Each trial has two outcomes: success (rolling a '4') or failure (not rolling a '4'). The probability of success is constant for each trial. This is the definition of a Binomial experiment.

Step 2: Determine the parameters of the Binomial distribution.
The number of trials is $n = 10$.
The probability of success, $p$, is the probability of rolling a '4' on a fair die, which is $p = 1/6$.
The probability of failure is $q = 1 - p = 1 - 1/6 = 5/6$.

Step 3: Apply the formula for the variance of a Binomial random variable.
The variance of a Binomial distribution is given by $Var(X) = np(1-p)$, or $npq$.

Step 4: Substitute the parameters and calculate the variance.

Step 5: Simplify the fraction.

Result: The variance of $X$ is $25/18$.
"
:::

---

Summary

A firm grasp of central tendency and dispersion is non-negotiable for success in the Probability and Statistics section of the GATE exam. These concepts form the bedrock upon which more complex statistical ideas are built.

---

What's Next?

Mastery of these fundamental measures prepares you for more advanced topics in probability theory.

---

---

Part 3: Correlation and Covariance

Introduction

In our study of random variables, we have thus far focused primarily on the properties of a single variable, such as its mean and variance. The variance, in particular, quantifies the spread or dispersion of a variable's distribution around its mean. However, in many practical applications, we are interested in understanding the relationship between two or more random variables. Do they tend to move in the same direction, in opposite directions, or is there no discernible pattern to their joint behavior?

This chapter introduces two fundamental statistical measures that address this question: covariance and correlation. Covariance provides a measure of the joint variability of two random variables, indicating the direction of their linear relationship. Correlation, a standardized version of covariance, goes a step further by also quantifying the strength of this linear association. A firm grasp of these concepts is indispensable, as they form the bedrock of more advanced topics such as regression analysis and portfolio theory, and are frequently tested in the GATE examination.

---

Key Concepts

1. Understanding and Calculating Covariance

The definition of covariance, $E[(X - \mu_X)(Y - \mu_Y)]$, provides significant intuition. Consider the term $(X - \mu_X)(Y - \mu_Y)$. If $X$ and $Y$ tend to be simultaneously above their means or simultaneously below their means, this product will be positive on average. This results in a positive covariance, suggesting a positive linear relationship. Conversely, if one variable tends to be above its mean when the other is below its mean, the product will be negative on average, yielding a negative covariance. If there is no consistent linear pattern, the positive and negative products will tend to cancel out, resulting in a covariance near zero.

Positive Covariance







Negative Covariance







Zero Covariance

While the definitional formula is useful for conceptual understanding, a more practical formula is often used for computation, especially with discrete random variables. We can expand the definition:

This leads to the widely used computational formula.

Worked Example:

Problem: A fair six-sided die is rolled. Let $X$ be a random variable that is 1 if the outcome is even and 0 if it is odd. Let $Y$ be a random variable that is 1 if the outcome is greater than 3, and 0 otherwise. Calculate the covariance between $X$ and $Y$.

Solution:

Step 1: Define the sample space and probabilities.
The sample space is $S = \{1, 2, 3, 4, 5, 6\}$, with each outcome having a probability of $1/6$.

Step 2: Determine the values of $X$ and $Y$ for each outcome and calculate $E[X]$ and $E[Y]$.

• For $X$: $X=1$ for $\{2, 4, 6\}$; $X=0$ for $\{1, 3, 5\}$.


• For $Y$: $Y=1$ for $\{4, 5, 6\}$; $Y=0$ for $\{1, 2, 3\}$.

$P(X=1) = 3/6 = 1/2$, $P(X=0) = 3/6 = 1/2$.

$P(Y=1) = 3/6 = 1/2$, $P(Y=0) = 3/6 = 1/2$.

Step 3: Determine the value of the product $XY$ for each outcome and calculate $E[XY]$.
We need to find the outcomes where $XY=1$. This occurs only when both $X=1$ (even) and $Y=1$ (greater than 3). The outcomes satisfying this are $\{4, 6\}$.
Thus, $P(XY=1) = P(\{4, 6\}) = 2/6 = 1/3$. For all other outcomes, $XY=0$.

Step 4: Apply the computational formula for covariance.

Answer: The covariance between $X$ and $Y$ is $1/12$.

---

2. Properties of Covariance

Understanding the properties of covariance is critical for simplifying complex expressions, a common requirement in GATE problems.

Symmetry: $Cov(X, Y) = Cov(Y, X)$

Covariance with a Constant: The covariance of a random variable with a constant is always zero.

Relationship with Variance: The covariance of a random variable with itself is its variance.

Effect of Scaling and Shifting (Linear Transformations): For constants $a, b, c, d$:

Notice that the additive constants $b$ and $d$ do not affect the covariance, as they do not change the spread of the variables.

Bilaterality:

Worked Example:

Problem: Let $X$ be a random variable with $Var(X) = 9$. Let $Y = 5 - 2X$. Calculate $Cov(X, Y)$.

Solution:

Step 1: Identify the relationship between $X$ and $Y$.
We are given $Y$ as a linear transformation of $X$: $Y = -2X + 5$.

Step 2: Apply the property of covariance under linear transformations.
We need to find $Cov(X, Y) = Cov(X, -2X + 5)$.
Let $a=1, b=0, c=-2, d=5$. The general property is $Cov(aX + b, cX + d) = ac \cdot Cov(X, X)$.

Step 3: Use the relationship between covariance and variance.
We know that $Cov(X, X) = Var(X)$.

Step 4: Substitute the given value of $Var(X)$.
We are given $Var(X) = 9$.

Answer: The covariance between $X$ and $Y$ is $-18$.

---

#
## 3. Correlation Coefficient

A significant limitation of covariance is that its magnitude is scale-dependent. If we change the units of $X$ from meters to centimeters, the covariance will increase by a factor of 100, even though the underlying relationship between the variables has not changed. To overcome this, we use the correlation coefficient, which is a normalized measure.

The correlation coefficient, $\rho$, is a dimensionless quantity that always lies in the range $[-1, 1]$.

• $\rho = +1$: Perfect positive linear relationship.


• $\rho = -1$: Perfect negative linear relationship.


• $\rho = 0$: No linear relationship.


• Values between 0 and 1 indicate the strength of a positive linear relationship.


• Values between -1 and 0 indicate the strength of a negative linear relationship.

---

#
## 4. Variance of Sums of Random Variables

Covariance plays a crucial role in determining the variance of a sum or difference of random variables.

Let us derive the formula for $Var(X+Y)$:

By linearity of expectation:

This simplifies to:

Similarly, for the difference:

A special and very important case arises when $X$ and $Y$ are independent. If two variables are independent, their covariance is zero. (The converse is not always true). For independent variables, the formulas simplify significantly:

---

Problem-Solving Strategies

---

Common Mistakes

---

Practice Questions

:::question type="MCQ" question="Let $W$ be a random variable with $E[W]=0$ and $Var(W)=1$. A new random variable $V$ is defined as $V = 3 - 4W$. Given that $E[(V - E[V])^2] = 16$ and $E[(V - E[V])W] = -4$, which of the following statements is consistent?" options=["The definition of $V$ is consistent with the given expected values.","The definition of $V$ is inconsistent because the variance is incorrect.","The definition of $V$ is inconsistent because the covariance term is incorrect.","The data is insufficient to determine consistency."] answer="The definition of $V$ is consistent with the given expected values." hint="Use the properties of expectation, variance, and covariance on the linear transformation $V = 3 - 4W$ and check if the results match the given values." solution="
Step 1: Calculate the expected value of $V$ from its definition.

Since $E[W]=0$, we have $E[V] = 3$.

Step 2: Calculate the variance of $V$ from its definition.
The given term $E[(V - E[V])^2]$ is the definition of $Var(V)$. So, we are given $Var(V)=16$.
Let's calculate $Var(V)$ from the transformation:

Since $Var(W)=1$, we have $Var(V) = 16 \cdot 1 = 16$. This matches the given information.

Step 3: Calculate the covariance term from its definition.
The given term $E[(V - E[V])W]$ is a form of covariance. Since $E[W]=0$, this is $E[(V - E[V])(W - E[W])] = Cov(V, W)$. So, we are given $Cov(V, W) = -4$.
Let's calculate $Cov(V, W)$ from the transformation:

Using properties, $Cov(-4W, W) = -4 \cdot Cov(W, W) = -4 \cdot Var(W)$.
Since $Var(W)=1$, we have $Cov(V, W) = -4 \cdot 1 = -4$. This also matches the given information.

Answer: \boxed{The definition of $V$ is consistent with the given expected values.}
"
:::

:::question type="NAT" question="Two balls are drawn with replacement from an urn containing 3 red balls and 2 blue balls. Let the random variable $X$ be the number of red balls drawn, and let the random variable $Y$ be the number of blue balls drawn. The value of the covariance of $X$ and $Y$, $Cov(X, Y)$, is ______ (rounded off to two decimal places)." answer="-0.48" hint="Notice that $X+Y=2$ is a constant. Use this relationship and the properties of variance and covariance." solution="
Step 1: Establish the relationship between $X$ and $Y$.
Since two balls are drawn in total, the number of red balls ($X$) plus the number of blue balls ($Y$) must equal 2.

Step 2: Use the property of variance of a constant.
The variance of a constant is zero.

Step 3: Expand $Var(X+Y)$ using the formula involving covariance.

Step 4: Equate the expressions from Step 2 and Step 3.

This implies:

Step 5: Calculate $Var(X)$ and $Var(Y)$.
The drawing of each ball is a Bernoulli trial. Let a "success" be drawing a red ball. The probability of success is $p = 3/5 = 0.6$. Since we draw $n=2$ balls with replacement, $X$ follows a binomial distribution $B(n=2, p=0.6)$.
The variance of a binomial distribution is $np(1-p)$.

Similarly, $Y$ is the number of blue balls, so it follows $B(n=2, p=2/5=0.4)$.

Step 6: Substitute the variances into the equation for covariance.

Answer: \boxed{-0.48}
"
:::

:::question type="MSQ" question="Let $X$ and $Y$ be two random variables such that $Var(X)=16$, $Var(Y)=25$, and the correlation coefficient $\rho(X, Y)=-0.5$. Which of the following statements are correct?" options=["$Cov(X, Y) = -10$","$Var(X+Y) = 21$","$Var(X-Y) = 61$","$Cov(3X, 2-Y) = 30$"] answer="A,B,C" hint="Calculate each value using the fundamental formulas for covariance, correlation, and variance of sums/differences." solution="
Let's evaluate each option.

Option A: $Cov(X, Y)$
The formula for correlation is $\rho(X, Y) = \frac{Cov(X, Y)}{\sqrt{Var(X)Var(Y)}}$.

So, statement A is correct.

Option B: $Var(X+Y)$
The formula is $Var(X+Y) = Var(X) + Var(Y) + 2Cov(X, Y)$.

So, statement B is correct.

Option C: $Var(X-Y)$
The formula is $Var(X-Y) = Var(X) + Var(Y) - 2Cov(X, Y)$.

So, statement C is correct.

Option D: $Cov(3X, 2-Y)$
Using the property $Cov(aX+b, cY+d) = ac \cdot Cov(X, Y)$.
Here $a=3, b=0, c=-1, d=2$.

So, statement D is correct.

Answer: \boxed{A, B, C}
"
:::

---

Summary

---

---

What's Next?

---

---

Part 4: Conditional Expectation and Variance

Introduction

In our study of random variables, we often seek to understand the relationship between them. While concepts like covariance and correlation provide a measure of linear association, they do not capture the full picture. Conditional expectation offers a more powerful and nuanced tool. It allows us to determine the expected value of one random variable, given that we have observed the outcome of another. This concept is fundamental to prediction and estimation, forming the bedrock of modern statistical modeling and machine learning.

Consider a scenario where we wish to predict a student's final exam score ($Y$) based on their mid-term score ($X$). The unconditional expectation, $E[Y]$, gives us the average final score over all students. However, a much more accurate prediction can be made if we know the student's mid-term score, say $X=x$. The value we would then be interested in is the conditional expectation, $E[Y|X=x]$, which represents the average final score specifically for students who scored $x$ on the mid-term.

This chapter will formally define conditional expectation and its counterpart, conditional variance. We will explore their properties, most notably the Law of Total Expectation and the Law of Total Variance, which provide elegant methods for decomposing complex problems. Furthermore, we shall see how these concepts can be applied to solve intricate problems involving sequences of random events, a common feature in GATE questions.

---

Key Concepts

1. Conditional Expectation for Discrete Random Variables

When dealing with discrete random variables, the conditional expectation is computed as a weighted average, where the weights are given by the conditional probability mass function (PMF).

First, we must define the conditional PMF of $Y$ given $X=x$.

Here, $p_{X,Y}(x,y)$ is the joint PMF of $X$ and $Y$, and $p_X(x)$ is the marginal PMF of $X$, provided that $p_X(x) > 0$.

With this conditional PMF, we can now define the conditional expectation.

Worked Example:

Problem: The joint PMF of two discrete random variables $X$ and $Y$ is given by the following table:

| | Y=0 | Y=1 | Y=2 |
| :--- | :--- | :--- | :--- |
| X=0 | 0.1 | 0.2 | 0.1 |
| X=1 | 0.3 | 0.1 | 0.2 |

Calculate the conditional expectation $E[Y|X=1]$.

Solution:

Step 1: Find the marginal PMF of $X$, $p_X(x)$, specifically for $x=1$. We sum the probabilities across the row for $X=1$.

Step 2: Determine the conditional PMF of $Y$ given $X=1$ for each possible value of $Y$.

For $Y=0$:

For $Y=1$:

For $Y=2$:

(As a check, we note that $0.5 + 1/6 + 1/3 = 3/6 + 1/6 + 2/6 = 1$, as required for a valid PMF).

Step 3: Apply the formula for conditional expectation.

Step 4: Compute the final value.

Answer: $E[Y|X=1] = \frac{5}{6} \approx 0.833$

---

2. Conditional Expectation for Continuous Random Variables

The logic for continuous variables is analogous to the discrete case, with sums replaced by integrals and PMFs by probability density functions (PDFs).

The conditional PDF of $Y$ given $X=x$ is defined as:

where $f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y) \, dx$ is the marginal PDF of $X$, assuming $f_X(x) > 0$.

Worked Example:

Problem: Let the joint PDF of random variables $X$ and $Y$ be

Calculate $E[X|Y=1]$.

Solution:

Step 1: Find the marginal PDF of $Y$, $f_Y(y)$, by integrating the joint PDF with respect to $x$.

This is valid for $0 < y < 2$.

Step 2: Evaluate the marginal PDF at the given condition, $y=1$.

Step 3: Determine the conditional PDF $f_{X|Y}(x|y=1)$.

This conditional PDF is valid for $0 < x < 1$.

Step 4: Apply the formula for conditional expectation.

Step 5: Compute the integral.

Step 6: Simplify to find the final answer.

Answer: $E[X|Y=1] = \frac{5}{9}$

---

3. Properties of Conditional Expectation

The true power of conditional expectation is revealed through its properties, which simplify complex calculations and provide deep theoretical insights.

The Law of Total Expectation (Tower Property)

This is arguably the most important property of conditional expectation and is frequently tested in GATE. It states that the expected value of the conditional expectation of $Y$ given $X$ is simply the expected value of $Y$.

Let us demonstrate this for the continuous case.

This elegant result confirms the property.

Other Key Properties

Linearity: $E[aY + bZ | X] = aE[Y|X] + bE[Z|X]$ for constants $a, b$.

Taking Out What Is Known: For any function $g$, $E[g(X)Y | X] = g(X)E[Y|X]$. This is because, given $X$, the value of $g(X)$ is no longer random; it is a known constant. A special case is $E[g(X)|X] = g(X)$.

Independence: If $X$ and $Y$ are independent, then $E[Y|X] = E[Y]$. Knowing the value of $X$ provides no new information about $Y$.

---

---

#
## 4. Conditional Variance and The Law of Total Variance

Similar to expectation, we can define the variance of a random variable conditional on the value of another.

This leads to a decomposition formula for variance, analogous to the Law of Total Expectation.

---

#
## 5. Application: Recurrence Relations for Expected Values

A powerful application of conditional expectation is in solving problems involving sequences of trials, such as finding the expected number of steps to reach a certain state. This technique was implicitly tested in GATE. The core idea is to condition on the outcome of the first step.

Let $E$ be the expected number of trials until a target event occurs. We can write:

Worked Example:

Problem: A fair coin is flipped repeatedly. What is the expected number of flips required to see the pattern HT (Heads followed by Tails)?

Solution:

Step 1: Define the states and the expected values from each state.
Let $E$ be the expected number of flips from the start.
Let $E_H$ be the expected number of additional flips required, given that we have just seen a Head.

Step 2: Set up the equation for $E$ by conditioning on the first flip.
The first flip is either H (with probability 1/2) or T (with probability 1/2).
If the first flip is T, we have wasted one flip and are back to the start. The expected number of additional flips is $E$. Total flips: $1+E$.
If the first flip is H, we have used one flip and are now in state H. The expected number of additional flips is $E_H$. Total flips: $1+E_H$.

Step 3: Set up the equation for $E_H$ by conditioning on the next flip.
From state H, the next flip is either T (probability 1/2) or H (probability 1/2).
If the next flip is T, we have achieved the pattern HT. The process stops. Total additional flips: 1.
If the next flip is H, we have wasted a flip but are still in a state where the last flip was H. The expected number of additional flips is still $E_H$. Total additional flips: $1+E_H$.

Step 4: Solve the system of linear equations. First, solve for $E_H$.

Step 5: Substitute the value of $E_H$ back into the equation for $E$.

Answer: The expected number of flips is $\boxed{4}$.

---

Problem-Solving Strategies

---

Common Mistakes

---

Practice Questions

:::question type="MCQ" question="Let $X$ be a random variable and $g(X) = X^2$. Which of the following is always equal to $E[g(X)Y | X=x]$?" options=["$g(x) E[Y]$","$g(x) E[Y|X=x]$","$E[Y] E[g(X)|X=x]$","$g(E[X|Y=y])E[Y]$"] answer="$g(x) E[Y|X=x]$" hint="Recall the 'taking out what is known' property of conditional expectation. When we condition on $X=x$, any function of $x$ is treated as a constant." solution="The property of conditional expectation states that for any function $g$, $E[g(X)Y | X] = g(X)E[Y|X]$. When we evaluate this at a specific point $X=x$, the random variable $g(X)$ becomes the constant $g(x)$.
Therefore, we have:

In this specific case, $g(x) = x^2$. So, $E[X^2 Y | X=x] = x^2 E[Y|X=x]$.
Answer: \boxed{g(x) E[Y|X=x]}"
:::

:::question type="NAT" question="The joint PDF of two random variables $X$ and $Y$ is given by $f(x,y) = 8xy$ for $0 < y < x < 1$, and 0 otherwise. Calculate the value of $E[Y|X=0.5]$. (Round off to 2 decimal places)." answer="0.33" hint="First, find the marginal PDF $f_X(x)$. Then find the conditional PDF $f_{Y|X}(y|x)$. Finally, integrate $y \cdot f_{Y|X}(y|0.5)$ over the appropriate range of $y$." solution="
Step 1: Find the marginal PDF $f_X(x)$.
The limits for $y$ are from $0$ to $x$.

This is for $0 < x < 1$.

Step 2: Find the conditional PDF $f_{Y|X}(y|x)$.

This is for $0 < y < x$.

Step 3: Calculate $E[Y|X=0.5]$.
First, find the conditional PDF for $x=0.5$.

The range for $y$ is $0 < y < 0.5$.

Step 4: Integrate to find the conditional expectation.

Result:
The value is approximately 0.3333... Rounding to 2 decimal places gives 0.33.
Answer: \boxed{0.33}"
:::

:::question type="MSQ" question="Let $X$ and $Y$ be random variables. Which of the following statements are always true?" options=["$E[E[Y|X]] = E[Y]$","$Var(Y) = Var(E[Y|X]) + Var(Var(Y|X))$","$E[X+Y|X] = X + E[Y|X]$","$If X and Y are independent,$E[XY|X] = X E[Y]$"] answer="A,C,D" hint="Carefully check the standard properties of conditional expectation and variance. Pay close attention to the Law of Total Expectation and Law of Total Variance." solution="

• Option A: This is the Law of Total Expectation (or Tower Property), which is a fundamental property and is always true.


• Option B: This is an incorrect statement of the Law of Total Variance. The correct law is $Var(Y) = Var(E[Y|X]) + E[Var(Y|X)]$. The second term should be the expectation of the conditional variance, not the variance of the conditional variance. So, this statement is false.


• Option C: Using the linearity property and the 'taking out what is known' property: $E[X+Y|X] = E[X|X] + E[Y|X]$. Since $E[X|X]=X$, the statement simplifies to $X + E[Y|X]$. This is correct.


• Option D: Using the 'taking out what is known' property, we get $E[XY|X] = X E[Y|X]$. If $X$ and $Y$ are independent, then $E[Y|X] = E[Y]$. Substituting this in, we get $E[XY|X] = X E[Y]$. This statement is correct.

Therefore, options A, C, and D are always true.
Answer: \boxed{A, C, D}"
:::

:::question type="NAT" question="A coin has a probability $p=2/3$ of landing Heads (H). It is flipped repeatedly until the pattern Heads-Tails (HT) appears for the first time. What is the expected number of flips required?" answer="4.5" hint="Set up recurrence relations. Let $E$ be the expected flips from the start, and $E_H$ be the expected additional flips after one Head. Solve the system of two linear equations in terms of $p$." solution="
Step 1: Define states.
Let $E$ be the expected number of flips from the start.
Let $E_H$ be the expected additional flips, given the last flip was H.
Let $p=2/3$ be the probability of Heads (H) and $q=1-p=1/3$ be the probability of Tails (T).

Step 2: Formulate the equation for $E$.
From the start, the first flip is either H (with probability $p$) or T (with probability $q$).

• If T: We use 1 flip and are back to the start. Total flips: $1+E$.


• If H: We use 1 flip and move to state H. Total flips: $1+E_H$.

Step 3: Formulate the equation for $E_H$.
From state H, the next flip is either T (with probability $q$) or H (with probability $p$).

• If T: We use 1 flip and the pattern HT is achieved. The process stops. Total additional flips: 1.


• If H: We use 1 flip and are still in state H (the last flip was H). Total additional flips: $1+E_H$.

Step 4: Solve for $E_H$.

Since $1-p=q$:

Step 5: Substitute $E_H$ into the equation for $E$.

Since $1-q=p$ and $q+p=1$:

Step 6: Calculate the value using $p=2/3$ and $q=1/3$.

Result:
The expected number of flips is 4.5.
Answer: \boxed{4.5}"
:::

---

Summary

---

---

What's Next?

---

Chapter Summary

---

Chapter Review Questions

:::question type="MCQ" question="Let $X$ and $Y$ be two random variables with $E[X]=2$, $E[Y]=3$, $\operatorname{Var}(X)=1$, and $\operatorname{Var}(Y)=4$. If the correlation coefficient between them is $\rho_{XY} = 0.5$, what is the variance of the random variable $Z = 2X - Y$?" options=["4","8","12","16"] answer="A" hint="Recall the formula for the variance of a linear combination of two random variables: $\operatorname{Var}(aX + bY) = a^2\operatorname{Var}(X) + b^2\operatorname{Var}(Y) + 2ab\operatorname{Cov}(X, Y)$. You will first need to find the covariance from the given correlation coefficient." solution="
We are asked to find the variance of $Z = 2X - Y$. The general formula for the variance of a linear combination $aX + bY$ is:

In our case, $a=2$ and $b=-1$. The formula becomes:


We are given $\operatorname{Var}(X)=1$ and $\operatorname{Var}(Y)=4$. To find $\operatorname{Cov}(X, Y)$, we use the definition of the correlation coefficient:

where $\sigma_X = \sqrt{\operatorname{Var}(X)}$ and $\sigma_Y = \sqrt{\operatorname{Var}(Y)}$.
Given $\operatorname{Var}(X)=1$, we have $\sigma_X = \sqrt{1} = 1$.
Given $\operatorname{Var}(Y)=4$, we have $\sigma_Y = \sqrt{4} = 2$.
Now, we can find the covariance:

Finally, we substitute all the values back into the variance formula for $Z$:


Therefore, the variance of $Z$ is $\boxed{4}$.
"
:::

:::question type="NAT" question="A continuous random variable $X$ is uniformly distributed over the interval $[1, 3]$. The value of the conditional expectation $E[X | X > 2]$ is ____." answer="2.5" hint="First, find the conditional PDF of $X$ given the event $A = \{X > 2\}$. The formula is $f_{X|A}(x) = f_X(x) / P(A)$ for $x$ in the conditioned range." solution="
Let $X \sim U(1, 3)$. The probability density function (PDF) of $X$ is:

We want to find the conditional expectation $E[X | X > 2]$. Let the event be $A = \{X > 2\}$.
First, we calculate the probability of this event:

The conditional PDF of $X$ given $A$ is defined as:

For $2 < x \le 3$, the conditional PDF is:

And $f_{X|A}(x) = 0$ otherwise. This means that given $X>2$, $X$ is uniformly distributed on the interval $(2, 3]$.
Now, we can compute the conditional expectation:


The value of the conditional expectation is $\boxed{2.5}$.
"
:::

:::question type="MCQ" question="A fair six-sided die is rolled. Let $X$ be the random variable for the outcome. What is the expected value of the random variable $Y = (X-3.5)^2$?" options=["2.917","3.5","1.708","2.5"] answer="A" hint="The value $E[(X-\mu)^2]$ is, by definition, the variance of $X$. First, calculate the mean (expected value) of the die roll." solution="
The random variable $X$ represents the outcome of a fair six-sided die roll. The possible values for $X$ are $\{1, 2, 3, 4, 5, 6\}$, each with probability $P(X=x) = 1/6$.

First, we calculate the expected value of $X$, denoted $\mu_X$ or $E[X]$:

The question asks for the value of $E[Y] = E[(X-3.5)^2]$.
We recognize that $3.5$ is the mean of $X$. Therefore, we are being asked to calculate $E[(X - E[X])^2]$. This is the definition of the variance of $X$, $\operatorname{Var}(X)$.

We can calculate $\operatorname{Var}(X)$ by first finding $E[X^2]$:


Now, we can find the variance:


As a decimal, $\frac{35}{12} \approx 2.9166...$.
Rounding to three decimal places, the answer is $\boxed{2.917}$.
"
:::

:::question type="NAT" question="Two discrete random variables $X$ and $Y$ have a joint PMF given by $P(X=1, Y=0)=0.1$, $P(X=1, Y=1)=0.4$, $P(X=2, Y=0)=0.3$, and $P(X=2, Y=1)=0.2$. The covariance, $\operatorname{Cov}(X, Y)$, is ____." answer="-0.1" hint="Use the formula $\operatorname{Cov}(X, Y) = E[XY] - E[X]E[Y]$. You will need to calculate the marginal distributions to find $E[X]$ and $E[Y]$ first." solution="
To find the covariance $\operatorname{Cov}(X, Y)$, we use the formula $\operatorname{Cov}(X, Y) = E[XY] - E[X]E[Y]$. We must compute each term separately.

1. Calculate marginal PMFs and expectations $E[X]$ and $E[Y]$:
The marginal PMF of $X$ is found by summing over the values of $Y$:
$P(X=1) = P(X=1, Y=0) + P(X=1, Y=1) = 0.1 + 0.4 = 0.5$
$P(X=2) = P(X=2, Y=0) + P(X=2, Y=1) = 0.3 + 0.2 = 0.5$
The expected value of $X$ is:
$E[X] = 1 \cdot P(X=1) + 2 \cdot P(X=2) = 1(0.5) + 2(0.5) = 0.5 + 1.0 = 1.5$.

The marginal PMF of $Y$ is found by summing over the values of $X$:
$P(Y=0) = P(X=1, Y=0) + P(X=2, Y=0) = 0.1 + 0.3 = 0.4$
$P(Y=1) = P(X=1, Y=1) + P(X=2, Y=1) = 0.4 + 0.2 = 0.6$
The expected value of $Y$ is:
$E[Y] = 0 \cdot P(Y=0) + 1 \cdot P(Y=1) = 0(0.4) + 1(0.6) = 0.6$.

2. Calculate $E[XY]$:
The expectation of the product $XY$ is calculated using the joint PMF:

The only terms that are non-zero are when both $x \ne 0$ and $y \ne 0$. In this case, that is when $y=1$.

3. Calculate Covariance:
Now we substitute the computed values into the covariance formula:

The covariance is $\boxed{-0.1}$.
"
:::

---

What's Next?