Random Variables and Distributions

Overview

Welcome to the foundational chapter on Random Variables and Distributions, a cornerstone of your CMI Masters in Data Science curriculum. This chapter is absolutely critical, as it lays the theoretical and practical groundwork for understanding and applying nearly every statistical and machine learning concept you will encounter. Without a firm grasp of random variables and their distributions, topics like hypothesis testing, regression analysis, and even advanced deep learning architectures become abstract and difficult to interpret effectively.

In the CMI context, mastering this material is not just about theoretical understanding; it's about developing the intuition and analytical tools to tackle real-world data challenges. You'll learn how to mathematically model uncertainty, quantify variability, and make informed decisions based on probabilistic outcomes. This chapter directly addresses core competencies required for the CMI exams, ensuring you can correctly identify, apply, and interpret different probabilistic models crucial for data analysis and predictive modeling.

By the end of this chapter, you will possess the essential framework for reasoning about data generation processes, understanding the behavior of estimators, and interpreting the output of complex algorithms. This knowledge is indispensable for building robust data science solutions and effectively communicating insights, making it a high-yield area for your CMI success.

---

Chapter Contents

| # | Topic | What You'll Learn |
|---|-------|-------------------|
| 1 | Random Variables | Quantify outcomes of random experiments numerically. |
| 2 | Distribution Functions | Describe probability of variable taking values. |
| 3 | Expectation and Variance | Measure central tendency and data spread. |
| 4 | Standard Distributions | Explore common, well-understood probabilistic models. |

---

Learning Objectives

---

Now let's begin with Random Variables...
## Part 1: Random Variables

Introduction

In the realm of probability and statistics, a random variable serves as a fundamental concept, bridging the gap between abstract outcomes of a random experiment and numerical values. It is a function that assigns a real number to each outcome in the sample space of a random experiment. This transformation allows us to apply mathematical tools, such as algebra and calculus, to analyze the probabilities associated with these numerical outcomes.

Understanding random variables is crucial for CMI as it forms the bedrock for analyzing data, modeling uncertainty, and making predictions. In data science, almost every piece of data collected or generated can be viewed as a realization of one or more random variables, from the success rate of an algorithm to the error in a measurement. This unit will rigorously define random variables, explore their types, and detail methods for characterizing their behavior through probability distributions.

---

Key Concepts

1. Types of Random Variables

Random variables are primarily classified into two types based on their range:

* Discrete Random Variables: A random variable is discrete if its range $R_X$ is a finite or countably infinite set of real numbers. These variables typically arise from counting processes.
* Examples: The number of heads in three coin flips ($R_X = \{0, 1, 2, 3\}$), the number of customers arriving at a store in an hour ($R_X = \{0, 1, 2, \dots\}$).

* Continuous Random Variables: A random variable is continuous if its range $R_X$ is an uncountable infinite set, typically an interval or a collection of intervals on the real line. These variables usually arise from measurements.
* Examples: The height of a student, the time it takes for a process to complete, the temperature of a room.

For the purpose of this chapter and the CMI exam, we will primarily focus on discrete random variables, as they are frequently encountered and directly relevant to the provided PYQ.

---

2. Probability Mass Function (PMF)

For a discrete random variable, its probability distribution is described by a Probability Mass Function (PMF). The PMF specifies the probability that the random variable takes on each of its possible values.

Worked Example:

Problem: A fair coin is flipped three times. Let $X$ be the random variable representing the number of heads obtained. Determine the PMF of $X$.

Solution:

Step 1: Identify the sample space and the values of $X$.

The sample space $\Omega$ consists of $2^3 = 8$ equally likely outcomes:
$\Omega = \{HHH, HHT, HTH, THH, HTT, THT, TTH, TTT\}$

The random variable $X$ maps each outcome to the number of heads:
$X(HHH) = 3$
$X(HHT) = 2$
$X(HTH) = 2$
$X(THH) = 2$
$X(HTT) = 1$
$X(THT) = 1$
$X(TTH) = 1$
$X(TTT) = 0$

The range of $X$ is $R_X = \{0, 1, 2, 3\}$.

Step 2: Calculate the probability for each value in $R_X$.

Since each outcome in $\Omega$ has a probability of $1/8$:

For $X=0$: Only $TTT$ maps to $0$.
$p_X(0) = P(X=0) = P(\{TTT\}) = \frac{1}{8}$

For $X=1$: $HTT, THT, TTH$ map to $1$.
$p_X(1) = P(X=1) = P(\{HTT, THT, TTH\}) = \frac{3}{8}$

For $X=2$: $HHT, HTH, THH$ map to $2$.
$p_X(2) = P(X=2) = P(\{HHT, HTH, THH\}) = \frac{3}{8}$

For $X=3$: Only $HHH$ maps to $3$.
$p_X(3) = P(X=3) = P(\{HHH\}) = \frac{1}{8}$

Step 3: Verify the properties of a PMF.

All $p_X(x) \ge 0$.

Answer: The PMF of $X$ is:
$p_X(0) = 1/8$
$p_X(1) = 3/8$
$p_X(2) = 3/8$
$p_X(3) = 1/8$
and $p_X(x) = 0$ for $x \notin \{0, 1, 2, 3\}$.

---

3. Cumulative Distribution Function (CDF)

The CDF provides a cumulative view of the probabilities, indicating the probability that a random variable $X$ takes on a value less than or equal to a given value $x$.

Worked Example:

Problem: Using the PMF from the previous example ($X$ = number of heads in 3 coin flips), find the CDF of $X$.

Solution:

Step 1: Recall the PMF values.
$p_X(0) = 1/8$
$p_X(1) = 3/8$
$p_X(2) = 3/8$
$p_X(3) = 1/8$

Step 2: Calculate $F_X(x)$ for different intervals of $x$.

For $x < 0$:
$F_X(x) = P(X \le x) = 0$ (since $X$ cannot be negative)

For $0 \le x < 1$:
$F_X(x) = P(X \le x) = p_X(0) = \frac{1}{8}$

For $1 \le x < 2$:
$F_X(x) = P(X \le x) = p_X(0) + p_X(1) = \frac{1}{8} + \frac{3}{8} = \frac{4}{8} = \frac{1}{2}$

For $2 \le x < 3$:
$F_X(x) = P(X \le x) = p_X(0) + p_X(1) + p_X(2) = \frac{1}{8} + \frac{3}{8} + \frac{3}{8} = \frac{7}{8}$

For $x \ge 3$:
$F_X(x) = P(X \le x) = p_X(0) + p_X(1) + p_X(2) + p_X(3) = \frac{1}{8} + \frac{3}{8} + \frac{3}{8} + \frac{1}{8} = \frac{8}{8} = 1$

Answer: The CDF of $X$ is:

---

4. Functions of Random Variables

Often, we are interested in a new random variable $Y$ that is a function of an existing random variable $X$, i.e., $Y = g(X)$. To find the PMF of $Y$, we need to identify the possible values of $Y$ and sum the probabilities of all $X$ values that map to each $Y$ value. This concept is directly tested in the provided CMI PYQ.

Worked Example:

Problem: Let $X$ be a discrete random variable with PMF given by:
$p_X(1) = 0.2$, $p_X(2) = 0.3$, $p_X(3) = 0.3$, $p_X(4) = 0.2$.
Let $Y = (X-2)^2$. Find the PMF of $Y$.

Solution:

Step 1: Identify the range of $X$ and its PMF.
$R_X = \{1, 2, 3, 4\}$
$p_X(1) = 0.2$
$p_X(2) = 0.3$
$p_X(3) = 0.3$
$p_X(4) = 0.2$

Step 2: Determine the possible values of $Y = (X-2)^2$ by applying the function $g(x) = (x-2)^2$ to each value in $R_X$.

For $x=1$: $y = (1-2)^2 = (-1)^2 = 1$
For $x=2$: $y = (2-2)^2 = (0)^2 = 0$
For $x=3$: $y = (3-2)^2 = (1)^2 = 1$
For $x=4$: $y = (4-2)^2 = (2)^2 = 4$

The range of $Y$ is $R_Y = \{0, 1, 4\}$.

Step 3: Calculate the PMF of $Y$ for each value in $R_Y$.

For $Y=0$: Only $X=2$ maps to $Y=0$.
$p_Y(0) = P(Y=0) = P(X=2) = p_X(2) = 0.3$

For $Y=1$: $X=1$ and $X=3$ map to $Y=1$.
$p_Y(1) = P(Y=1) = P(X=1 \text{ or } X=3) = p_X(1) + p_X(3) = 0.2 + 0.3 = 0.5$

For $Y=4$: Only $X=4$ maps to $Y=4$.
$p_Y(4) = P(Y=4) = P(X=4) = p_X(4) = 0.2$

Step 4: Verify the PMF properties.
All $p_Y(y) \ge 0$.

Answer: The PMF of $Y$ is:
$p_Y(0) = 0.3$
$p_Y(1) = 0.5$
$p_Y(4) = 0.2$
and $p_Y(y) = 0$ for $y \notin \{0, 1, 4\}$.

---

5. Expected Value of a Discrete Random Variable

The expected value (or mean) of a random variable is a measure of its central tendency, representing the average value we would expect to observe if the experiment were repeated many times.

Properties of Expected Value:

$\operatorname{E}[c] = c$ for any constant $c$.

$\operatorname{E}[aX + b] = a\operatorname{E}[X] + b$ for constants $a, b$. (Linearity of Expectation)

$\operatorname{E}[X+Y] = \operatorname{E}[X] + \operatorname{E}[Y]$ (for any random variables $X, Y$, not necessarily independent).

Worked Example:

Problem: For the random variable $X$ (number of heads in 3 coin flips) with PMF: $p_X(0) = 1/8$, $p_X(1) = 3/8$, $p_X(2) = 3/8$, $p_X(3) = 1/8$, calculate $\operatorname{E}[X]$ and $\operatorname{E}[2X+1]$.

Solution:

Step 1: Calculate $\operatorname{E}[X]$.

Step 2: Calculate $\operatorname{E}[2X+1]$ using linearity of expectation.

Answer: $\boxed{\operatorname{E}[X] = 1.5 \text{ and } \operatorname{E}[2X+1] = 4}$

---

6. Variance of a Discrete Random Variable

The variance measures the spread or dispersion of the values of a random variable around its mean. A higher variance indicates greater variability.

Properties of Variance:

$\operatorname{Var}(c) = 0$ for any constant $c$.

$\operatorname{Var}(aX + b) = a^2 \operatorname{Var}(X)$ for constants $a, b$. Note that the constant $b$ does not affect variance.

$\operatorname{Var}(X) \ge 0$.

Worked Example:

Problem: For the random variable $X$ (number of heads in 3 coin flips) with PMF: $p_X(0) = 1/8$, $p_X(1) = 3/8$, $p_X(2) = 3/8$, $p_X(3) = 1/8$, and $\operatorname{E}[X] = 1.5$, calculate $\operatorname{Var}(X)$ and $\operatorname{Var}(2X+1)$.

Solution:

Step 1: Calculate $\operatorname{E}[X^2]$.

Step 2: Calculate $\operatorname{Var}(X)$ using the computational formula.

Step 3: Calculate $\operatorname{Var}(2X+1)$ using properties of variance.

Answer: $\boxed{\operatorname{Var}(X) = 0.75 \text{ and } \operatorname{Var}(2X+1) = 3}$

---

7. Joint Probability and Independence of Random Variables

When dealing with multiple random variables, we often need to understand their joint behavior.

From a joint PMF, we can derive the marginal PMFs for $X$ and $Y$:

---

8. Uniform Discrete Distribution

A random variable follows a uniform discrete distribution if each value in its finite range has an equal probability of being observed. This is directly stated in the PYQ.

Worked Example:

Problem: Let $X$ be a random variable sampled uniformly at random from the set $S = \{0, 1, 2, 3, 4\}$.
a) What is the PMF of $X$?
b) Calculate $\operatorname{E}[X]$.

Solution:

Step 1: Identify the size of the set $S$.
The set $S$ has $N=5$ elements.

Step 2: Determine the PMF.
Since $X$ is sampled uniformly, each element has a probability of $1/N$.

a) The PMF of $X$ is:
$p_X(x) = \frac{1}{5}$ for $x \in \{0, 1, 2, 3, 4\}$
and $p_X(x) = 0$ otherwise.

Step 3: Calculate $\operatorname{E}[X]$.

Answer: $\boxed{\text{a) } p_X(x) = 1/5 \text{ for } x \in \{0,1,2,3,4\}. \text{ b) } \operatorname{E}[X] = 2.}$

---

Problem-Solving Strategies

---

Common Mistakes

---

Practice Questions

:::question type="MCQ" question="Let $X$ be a discrete random variable with PMF $p_X(x)$ given by $p_X(1)=0.1$, $p_X(2)=0.3$, $p_X(3)=0.4$, $p_X(4)=0.2$. Let $Y = |X-2|$. Which of the following is the correct PMF for $Y$?" options=["$p_Y(0)=0.3, p_Y(1)=0.5, p_Y(2)=0.2$","$p_Y(0)=0.1, p_Y(1)=0.3, p_Y(2)=0.4, p_Y(3)=0.2$","$p_Y(0)=0.3, p_Y(1)=0.3, p_Y(2)=0.4$","$p_Y(0)=0.3, p_Y(1)=0.6, p_Y(2)=0.1$"] answer="$p_Y(0)=0.3, p_Y(1)=0.5, p_Y(2)=0.2$" hint="Map each value of $X$ to $Y$ and sum probabilities for repeated $Y$ values." solution="
Step 1: Determine the values of $Y = |X-2|$ for each $x \in R_X$.

• If $X=1$, $Y = |1-2| = |-1| = 1$.


• If $X=2$, $Y = |2-2| = |0| = 0$.


• If $X=3$, $Y = |3-2| = |1| = 1$.


• If $X=4$, $Y = |4-2| = |2| = 2$.

Step 2: Identify the range of $Y$, which is $R_Y = \{0, 1, 2\}$.

Step 3: Calculate the PMF for $Y$.

• $p_Y(0) = P(Y=0) = P(X=2) = p_X(2) = 0.3$.


• $p_Y(1) = P(Y=1) = P(X=1 \text{ or } X=3) = p_X(1) + p_X(3) = 0.1 + 0.4 = 0.5$.


• $p_Y(2) = P(Y=2) = P(X=4) = p_X(4) = 0.2$.

Step 4: Verify that the probabilities sum to 1: $0.3 + 0.5 + 0.2 = 1.0$.
Answer: $\boxed{p_Y(0)=0.3, p_Y(1)=0.5, p_Y(2)=0.2}$
"
:::

:::question type="NAT" question="A discrete random variable $X$ has the following PMF: $p_X(x) = c(x+1)$ for $x \in \{0, 1, 2\}$, and $0$ otherwise. Calculate the value of $\operatorname{E}[X^2]$. (Enter your answer as a decimal rounded to two decimal places.)" answer="2.33" hint="First find the constant $c$ by ensuring the sum of probabilities is 1. Then calculate $\operatorname{E}[X^2]$." solution="
Step 1: Find the constant $c$.
The sum of probabilities must be 1:

Step 2: Write out the full PMF.
$p_X(0) = \frac{1}{6}(0+1) = \frac{1}{6}$
$p_X(1) = \frac{1}{6}(1+1) = \frac{2}{6}$
$p_X(2) = \frac{1}{6}(2+1) = \frac{3}{6}$

Step 3: Calculate $\operatorname{E}[X^2]$.

Step 4: Convert to decimal rounded to two places.
$7/3 \approx 2.3333...$
Rounded to two decimal places, $\operatorname{E}[X^2] = 2.33$.
Answer: $\boxed{2.33}$
"
:::

:::question type="MSQ" question="Let $X$ be a random variable representing the outcome of rolling a fair six-sided die, so $R_X = \{1, 2, 3, 4, 5, 6\}$. Let $Y = X \pmod 3$. Which of the following statements is/are true?" options=["$P(Y=0) = 1/3$","$X$ and $Y$ are independent","$\operatorname{E}[Y] = 1$","$\operatorname{Var}(Y) = 2/3$"] answer="A,C,D" hint="Calculate the PMF of $Y$ first. Then evaluate independence, expected value, and variance." solution="
Step 1: Determine the PMF of $X$. Since it's a fair die, $p_X(x) = 1/6$ for $x \in \{1, 2, 3, 4, 5, 6\}$.

Step 2: Determine the PMF of $Y = X \pmod 3$.

• If $X=1$, $Y = 1 \pmod 3 = 1$.


• If $X=2$, $Y = 2 \pmod 3 = 2$.


• If $X=3$, $Y = 3 \pmod 3 = 0$.


• If $X=4$, $Y = 4 \pmod 3 = 1$.


• If $X=5$, $Y = 5 \pmod 3 = 2$.


• If $X=6$, $Y = 6 \pmod 3 = 0$.

The range of $Y$ is $R_Y = \{0, 1, 2\}$.

Step 3: Calculate $p_Y(y)$.

• $P(Y=0) = P(X=3 \text{ or } X=6) = p_X(3) + p_X(6) = 1/6 + 1/6 = 2/6 = 1/3$. (Statement A is TRUE)


• $P(Y=1) = P(X=1 \text{ or } X=4) = p_X(1) + p_X(4) = 1/6 + 1/6 = 2/6 = 1/3$.


• $P(Y=2) = P(X=2 \text{ or } X=5) = p_X(2) + p_X(5) = 1/6 + 1/6 = 2/6 = 1/3$.

Step 4: Evaluate the statements.

Statement A: $P(Y=0) = 1/3$. This is TRUE from our calculation above.

Statement B: $X$ and $Y$ are independent.
Since $Y$ is a function of $X$ ($Y = g(X)$), they are generally dependent. For example, if we know $X=1$, then $Y$ must be $1 \pmod 3 = 1$. This means $P(Y=1 | X=1) = 1 \ne P(Y=1) = 1/3$. Thus, $X$ and $Y$ are dependent. (Statement B is FALSE)

Statement C: $\operatorname{E}[Y] = 1$.

(Statement C is TRUE)

Statement D: $\operatorname{Var}(Y) = 2/3$.
First, calculate $\operatorname{E}[Y^2]$.

Now, calculate $\operatorname{Var}(Y)$.

(Statement D is TRUE)
Answer: $\boxed{\text{A, C, D}}$
"
:::

:::question type="SUB" question="Let $X$ be a discrete random variable with PMF $p_X(x) = \frac{1}{2^x}$ for $x \in \{1, 2, 3, \dots\}$, and $0$ otherwise.
a) Prove that this is a valid PMF.
b) Derive the expression for the CDF, $F_X(x)$.
c) Calculate $\operatorname{E}[X]$. " answer="a) $\sum p_X(x) = 1$. b) $F_X(x) = 1 - \frac{1}{2^{\lfloor x \rfloor}}$. c) $\operatorname{E}[X] = 2$." hint="For part a), use the sum of a geometric series. For part b), use the definition of CDF. For part c), use the formula for $\operatorname{E}[X]$ and the sum $\sum_{k=1}^{\infty} k r^k = \frac{r}{(1-r)^2}$ for $|r|<1$." solution="
Part a) Prove that this is a valid PMF.

Step 1: Check non-negativity.
For $x \in \{1, 2, 3, \dots\}$, $p_X(x) = \frac{1}{2^x} > 0$. For other $x$, $p_X(x)=0$. So, $p_X(x) \ge 0$ for all $x$.

Step 2: Check if the sum of probabilities is 1.
We need to evaluate $\sum_{x=1}^{\infty} p_X(x)$.

This is a geometric series with first term $a = 1/2$ and common ratio $r = 1/2$.
The sum of an infinite geometric series is $\frac{a}{1-r}$ for $|r|<1$.

Since both conditions are met, $p_X(x)$ is a valid PMF.

Part b) Derive the expression for the CDF, $F_X(x)$.

Step 1: Define $F_X(x) = P(X \le x)$.

For $x < 1$:
$F_X(x) = 0$ (since $X$ cannot take values less than 1)

For $x \ge 1$:

This is a finite geometric series with $a=1/2$, $r=1/2$, and $n = \lfloor x \rfloor$ terms.
The sum of a finite geometric series is $a \frac{1-r^n}{1-r}$.

Thus, the CDF is:

Part c) Calculate $\operatorname{E}[X]$.

Step 1: Use the definition of expected value.

This is a known series sum. For a geometric series $\sum_{k=1}^{\infty} k r^k = \frac{r}{(1-r)^2}$ for $|r|<1$.
Here, $r = 1/2$.

Therefore, $\operatorname{E}[X] = 2$.
Answer: $\boxed{\text{a) } \sum p_X(x) = 1. \text{ b) } F_X(x) = 1 - \frac{1}{2^{\lfloor x \rfloor}}. \text{ c) } \operatorname{E}[X] = 2.}$
"
:::

---

Summary

---

What's Next?

---

---

Part 2: Distribution Functions

Introduction

Distribution functions are fundamental to probability theory and statistics, providing a comprehensive way to describe the behavior of random variables. In the context of the CMI Masters in Data Science, a deep understanding of these functions is crucial for modeling real-world phenomena, performing statistical inference, and building predictive models. This topic covers the essential concepts of how probabilities are distributed across the possible values of a random variable, whether discrete or continuous. Mastery of distribution functions allows us to quantify uncertainty, calculate probabilities of events, and characterize key aspects like the central tendency and spread of data, which are indispensable skills for any data scientist.

---

Key Concepts

1. Probability Mass Function (PMF)

The Probability Mass Function (PMF) is used to describe the probability distribution of a discrete random variable. It assigns a probability to each possible value that the random variable can take.

Worked Example:

Problem: Let $X$ be the number of heads in two coin tosses. Determine its PMF.

Solution:

Step 1: Identify the sample space and possible values of $X$.

The sample space for two coin tosses is $S = \{HH, HT, TH, TT\}$.
The possible values for $X$ (number of heads) are $0, 1, 2$.

Step 2: Calculate the probability for each value of $X$.

Step 3: Write down the PMF.

Answer: The PMF is $p_X(0)=1/4$, $p_X(1)=1/2$, $p_X(2)=1/4$.

---

2. Probability Density Function (PDF)

The Probability Density Function (PDF) is used to describe the probability distribution of a continuous random variable. Unlike the PMF, the PDF does not give the probability of a specific value, but rather the relative likelihood of the random variable taking on a given value. Probabilities for continuous random variables are calculated over intervals.

Worked Example:

Problem: Let $X$ be a continuous random variable with PDF $f(x) = cx(1-x)$ for $0 \le x \le 1$, and $0$ otherwise.
(a) Determine the value of $c$.
(b) Find the probability $P(X > 0.5)$.

Solution (a):

Step 1: Apply the normalization property of a PDF.

Step 2: Substitute the given PDF and integrate over its non-zero range.

Step 3: Simplify the integrand and perform the integration.

Step 4: Solve for $c$.

Answer (a): $c=6$.

Solution (b):

Step 1: Set up the integral for $P(X > 0.5)$ using the determined PDF.

Step 2: Substitute the PDF with the value of $c$.

Step 3: Perform the integration.

Answer (b): $P(X > 0.5) = 0.5$.

$x$
$f(x)$



0

0.5

1














$P(X > 0.5)$



Max $(0.5, 1.5)$

$f(x) = 6x(1-x)$

---

3. Cumulative Distribution Function (CDF)

The Cumulative Distribution Function (CDF) provides the probability that a random variable $X$ takes a value less than or equal to a given value $x$. It is defined for both discrete and continuous random variables.

Worked Example:

Problem: For the continuous random variable $X$ with PDF $f(x) = 6x(1-x)$ for $0 \le x \le 1$, and $0$ otherwise, find its CDF $F(x)$. Then, use the CDF to find $P(X > 0.5)$.

Solution:

Step 1: Define $F(x)$ for different ranges of $x$.

For $x < 0$:

For $0 \le x \le 1$:

For $x > 1$:

From the previous example, we know this integral equals 1.

Step 2: Combine the parts to write the full CDF.

Step 3: Use the CDF to find $P(X > 0.5)$.

Answer: The CDF is $F(x) = 3x^2 - 2x^3$ for $0 \le x \le 1$, and $P(X > 0.5) = 0.5$.

---

---

#
## 4. Expected Value (Mean)

The expected value, or mean, of a random variable is a measure of its central tendency. It represents the average value one would expect if the experiment were repeated many times.

Worked Example:

Problem: Find the expected value of $X$ for the continuous random variable with PDF $f(x) = 6x(1-x)$ for $0 \le x \le 1$.

Solution:

Step 1: Apply the formula for the expected value of a continuous random variable.

Step 2: Substitute the PDF and integrate over its non-zero range.

Step 3: Perform the integration.

Answer: $E[X] = 0.5$.

---

#
## 5. Variance

The variance measures the spread or dispersion of a random variable's values around its expected value. A higher variance indicates greater variability.

Worked Example:

Problem: Find the variance of $X$ for the continuous random variable with PDF $f(x) = 6x(1-x)$ for $0 \le x \le 1$. (We already found $E[X] = 0.5$.)

Solution:

Step 1: Calculate $E[X^2]$.

Step 2: Use the variance formula $\operatorname{Var}(X) = E[X^2] - (E[X])^2$.

We have $E[X^2] = 0.3$ and $E[X] = 0.5$.

Answer: $\operatorname{Var}(X) = 0.05$.

---

#
## 6. Properties of Expectation and Variance

These properties simplify calculations involving sums and linear transformations of random variables.

---

#
## 7. Central Limit Theorem (CLT) and Normal Approximation

The Central Limit Theorem (CLT) is one of the most powerful theorems in statistics. It explains why many natural phenomena follow a normal distribution, even if the individual components contributing to them do not.

Worked Example:

Problem: The time taken (in minutes) for a data scientist to complete a specific task is a random variable with mean $15$ minutes and standard deviation $4$ minutes. If a data scientist completes $100$ such tasks independently, what is the approximate probability that the total time taken for these $100$ tasks is less than $1450$ minutes?

Solution:

Step 1: Identify the given parameters for a single task $X_i$.

Mean $E[X_i] = \mu = 15$ minutes.
Standard deviation $\sigma = 4$ minutes.
Number of tasks $n = 100$.

Step 2: Define the total time $S_{100}$ and apply CLT.

The total time for $100$ tasks is $S_{100} = \sum_{i=1}^{100} X_i$.
By the Central Limit Theorem, for large $n$, $S_n$ is approximately normally distributed.

Calculate the mean of $S_{100}$:

Calculate the variance of $S_{100}$:

Calculate the standard deviation of $S_{100}$:

So, $S_{100} \sim N(1500, 1600)$ approximately.

Step 3: Standardize the random variable to use the Z-score.

We want to find $P(S_{100} < 1450)$.

Step 4: Look up the probability using the standard normal CDF (or Z-table).

Using a standard normal table or calculator, $P(Z < -1.25) \approx 0.1056$.

Answer: The approximate probability that the total time taken is less than $1450$ minutes is $0.1056$.

---

#
## 8. Standardization (Z-score)

Standardization transforms a random variable into a standard score (Z-score), which represents how many standard deviations an observation is from the mean. This is particularly useful for comparing values from different normal distributions or for using standard normal tables.

---

Problem-Solving Strategies

---

Common Mistakes

---

---

Practice Questions

:::question type="MCQ" question="Let $X$ be a continuous random variable with the probability density function given by:

What is the value of $k$ that makes $f(x)$ a valid PDF?" options=["$1/2$","$1$","$2$","$e$"] answer="$2$" hint="Use the property that the integral of a PDF over its entire range must equal 1." solution="Step 1: Apply the normalization condition for a PDF.

Step 2: Substitute the given PDF into the integral.

Step 3: Evaluate the integral.




Step 4: Solve for $k$.

Answer: \boxed{2}"
:::

:::question type="NAT" question="A discrete random variable $Y$ has the following Probability Mass Function:

What is the value of $c$ (rounded to two decimal places)?" answer="0.55" hint="The sum of all probabilities in a PMF must be equal to 1." solution="Step 1: Apply the normalization condition for a PMF.

Step 2: Sum the probabilities for each possible value of $Y$.



Step 3: Set the sum equal to 1 and solve for $c$.

Find a common denominator (6).



Step 4: Round to two decimal places.

Answer: \boxed{0.55}"
:::

:::question type="MSQ" question="Let $X$ be a continuous random variable with CDF given by:

Which of the following statements is/are true?" options=["The PDF of $X$ is $f(x) = 2x$ for $0 \le x < 1$.","$P(X \le 0.5) = 0.25$.","$P(0.2 < X < 0.8) = 0.6$.","$E[X] = 2/3$." ] answer="A,B,C,D" hint="Remember that $f(x) = dF(x)/dx$ for continuous random variables. Use the CDF to find probabilities. Calculate $E[X] = \int x f(x) dx$." solution="Statement A: The PDF of $X$ is $f(x) = 2x$ for $0 \le x < 1$.
To find the PDF from the CDF, differentiate the CDF:

This is valid for $0 \le x < 1$. For $x<0$ and $x \ge 1$, $f(x)=0$. So, statement A is true.

Statement B: $P(X \le 0.5) = 0.25$.
Using the CDF definition:

Since $0 \le 0.5 < 1$, we use $F(x) = x^2$:

So, statement B is true.

Statement C: $P(0.2 < X < 0.8) = 0.6$.
Using the CDF property $P(a < X < b) = F(b) - F(a)$:

So, statement C is true.

Statement D: $E[X] = 2/3$.
Using the PDF $f(x) = 2x$ for $0 \le x < 1$:

So, statement D is true.

All options A, B, C, D are true.
Answer: \boxed{A,B,C,D}"
:::

:::question type="SUB" question="A manufacturing process produces items whose weights are independent random variables with a mean of $10$ kg and a standard deviation of $2$ kg. A sample of $64$ items is taken from the production line.
(a) What is the probability that the average weight of the items in the sample is less than $9.5$ kg?
(b) What is the total expected weight of the $64$ items?" answer="$0.0228$, $640$ kg" hint="For part (a), use the Central Limit Theorem to approximate the distribution of the sample mean. For part (b), use the linearity of expectation for a sum of random variables." solution="(a) Probability that the average weight is less than $9.5$ kg:

Step 1: Identify parameters for a single item $X_i$.
Mean $E[X_i] = \mu = 10$ kg.
Standard deviation $\sigma = 2$ kg.
Sample size $n = 64$.

Step 2: Apply the Central Limit Theorem to the sample mean $\bar{X}_n$.
For large $n$, $\bar{X}_n$ is approximately normally distributed with:
Mean of sample mean: $E[\bar{X}_n] = \mu = 10$ kg.
Standard deviation of sample mean (standard error): $\sigma_{\bar{X}_n} = \frac{\sigma}{\sqrt{n}} = \frac{2}{\sqrt{64}} = \frac{2}{8} = 0.25$ kg.
So, $\bar{X}_{64} \sim N(10, (0.25)^2)$ approximately.

Step 3: Standardize the value $9.5$ kg.
We want to find $P(\bar{X}_{64} < 9.5)$.

Step 4: Look up the probability using the standard normal CDF.

Using a standard normal table or calculator, $P(Z < -2) \approx 0.0228$.

(b) Total expected weight of the $64$ items:

Step 1: Define the total weight $S_{64}$.
$S_{64} = \sum_{i=1}^{64} X_i$.

Step 2: Apply the linearity of expectation.

Since each $E[X_i] = \mu = 10$ kg:

Answer: (a) \boxed{0.0228}, (b) \boxed{640 \text{ kg}}"
:::

---

Summary

---

What's Next?

---

---

Part 3: Expectation and Variance

Introduction

Expectation and variance are fundamental concepts in probability theory, providing concise summaries of the central tendency and spread of a random variable's distribution. The expectation, or expected value, quantifies the "average" outcome of a random variable over a large number of trials. It represents the weighted average of all possible values a random variable can take, with weights given by their respective probabilities. The variance, on the other hand, measures the dispersion or spread of the random variable's values around its expected value. A low variance indicates that values tend to be close to the mean, while a high variance suggests that values are spread out over a wider range.

In the CMI examination, a deep understanding of expectation and variance is crucial. These concepts are extensively tested, often through complex scenarios involving multiple random variables, indicator functions, and various probability distributions. Mastery of linearity of expectation and the properties of variance is essential for efficiently solving problems that might otherwise appear intractable.

---

Key Concepts

#
## 1. Expectation of a Random Variable

The expectation, also known as the expected value or mean, of a random variable $X$ is denoted by $E[X]$ or $\mu$. It represents the long-run average value of the variable.

#
### 1.1. Discrete Random Variables

For a discrete random variable $X$ with probability mass function (PMF) $P(X=x)$, the expectation is calculated by summing the products of each possible value of $X$ and its corresponding probability.

Worked Example:

Problem: A fair six-sided die is rolled. Let $X$ be the number rolled. Calculate $E[X]$.

Solution:

Step 1: Identify the possible values of $X$ and their probabilities.
The possible values are $1, 2, 3, 4, 5, 6$. Since the die is fair, each outcome has a probability of $1/6$.

Step 2: Apply the formula for the expectation of a discrete random variable.

Step 3: Simplify the expression.

Answer: \boxed{3.5}

---

#
### 1.2. Continuous Random Variables

For a continuous random variable $X$ with probability density function (PDF) $f(x)$, the expectation is calculated by integrating the product of $x$ and its PDF over the entire range of possible values.

Worked Example:

Problem: Let $X$ be a continuous random variable with PDF $f(x) = 2x$ for $0 \le x \le 1$, and $f(x) = 0$ otherwise. Calculate $E[X]$.

Solution:

Step 1: Identify the PDF and its range.

Step 2: Apply the formula for the expectation of a continuous random variable.

Since $f(x)$ is non-zero only for $0 \le x \le 1$, the integral limits change.

Step 3: Evaluate the integral.

Answer: \boxed{\frac{2}{3}}

---

---

#

1.3. Properties of Expectation

Expectation has several important properties that simplify calculations, especially when dealing with sums or transformations of random variables.

Worked Example (Linearity of Expectation):

Problem: A box contains 10 balls: 3 red and 7 blue. Two balls are drawn without replacement. Let $X$ be the number of red balls drawn. Find $E[X]$.

Solution:

Step 1: Define indicator random variables.
Let $X_1$ be an indicator variable for the first ball drawn being red.
Let $X_2$ be an indicator variable for the second ball drawn being red.

Step 2: Express $X$ as a sum of indicator variables.
The total number of red balls drawn is $X = X_1 + X_2$.

Step 3: Calculate the expectation of each indicator variable.
For $X_1$:

For $X_2$:
The probability that the second ball is red can be found using the Law of Total Probability:

So, $E[X_2] = \frac{3}{10}$.

Step 4: Apply linearity of expectation.

Answer: $\frac{3}{5}$

---

#

2. Variance of a Random Variable

The variance of a random variable $X$, denoted by $V(X)$ or $\text{Var}(X)$ or $\sigma^2$, measures the spread or dispersion of its values around the mean. It is the expected value of the squared deviation from the mean.

Derivation of $V(X) = E[X^2] - (E[X])^2$:

Step 1: Start with the definition of variance.

Step 2: Expand the squared term inside the expectation. Let $\mu = E[X]$ for simplicity.

Step 3: Apply linearity of expectation.

Step 4: Use properties of expectation ($E[aX] = aE[X]$ and $E[c] = c$).

Step 5: Substitute $\mu = E[X]$ back into the expression.

Step 6: Simplify the expression.

---

Worked Example:

Problem: A fair six-sided die is rolled. Let $X$ be the number rolled. Calculate $V(X)$.

Solution:

Step 1: Recall $E[X]$ from the previous example.

Step 2: Calculate $E[X^2]$.
Using the formula $E[g(X)] = \sum_{x} g(x) P(X=x)$ with $g(X) = X^2$.

Step 3: Apply the computational formula for variance.

Step 4: Find a common denominator and simplify.

Answer: $\frac{35}{12}$

---

#

2.1. Properties of Variance

Variance also has several key properties.

---

#

3. Standard Deviation

The standard deviation is the square root of the variance and is denoted by $\sigma$. It has the same units as the random variable itself, making it more interpretable than variance in many contexts.

---

#

4. Indicator Random Variables

An indicator random variable is a special type of discrete random variable that takes on a value of $1$ if a particular event occurs and $0$ otherwise. They are incredibly powerful when used with the linearity of expectation, especially in counting problems.

Worked Example (Using Indicator Variables for Expectation):

Problem: In a group of $n$ people, what is the expected number of people who share the same birthday (ignoring leap years)?

Solution:

Step 1: Define indicator variables for each possible pair of people.
Let $N = \binom{n}{2}$ be the total number of pairs of people.
Let $I_{ij}$ be an indicator variable that people $i$ and $j$ share a birthday, for $1 \le i < j \le n$.

Step 2: Express the total number of shared birthdays ($X$) as a sum of indicator variables.

Step 3: Calculate the expectation of a single indicator variable.
Assuming each day of the year (365 days) is equally likely for a birthday.
The probability that two specific people share a birthday is $P(I_{ij}=1) = \frac{1}{365}$.

Step 4: Apply linearity of expectation.

Since there are $\binom{n}{2}$ such indicator variables, and each has the same expectation:

Answer: $\frac{n(n-1)}{730}$

---

#

5. Chebyshev's Inequality

Chebyshev's Inequality provides a bound on the probability that a random variable deviates from its mean by a certain amount. It is a powerful tool because it applies to any probability distribution for which the mean and variance exist, without requiring knowledge of the specific distribution shape.

Worked Example:

Problem: The average height of students in a university is $170$ cm with a standard deviation of $5$ cm. What is the minimum percentage of students whose height is between $160$ cm and $180$ cm?

Solution:

Step 1: Identify the given values.
$E[X] = 170$ cm
$\sigma_X = 5$ cm
We want to find $P(160 \le X \le 180)$.

Step 2: Rephrase the probability in terms of deviation from the mean.
The interval $[160, 180]$ is $170 \pm 10$. So, we are interested in $P(|X - 170| \le 10)$.

Step 3: Apply Chebyshev's Inequality for the complementary event.
Chebyshev's inequality gives an upper bound for $P(|X - E[X]| \ge k)$.
Here, $k = 10$.

First, calculate $V(X) = \sigma_X^2 = 5^2 = 25$.

Step 4: Find the probability for the desired interval.
The probability of being within the interval is $1 - P(|X - 170| \ge 10)$.

Step 5: Convert to percentage.

Answer: At least $75\%$ of students have heights between $160$ cm and $180$ cm.

---

---

Problem-Solving Strategies

---

Common Mistakes

---

Practice Questions

:::question type="NAT" question="A company manufactures light bulbs. The lifespan of a bulb, $X$, in years, has a probability density function $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$, where $\lambda = 0.5$. What is the expected lifespan of a bulb in years?" answer="2" hint="Recall the formula for the expectation of a continuous random variable and the properties of the exponential distribution." solution="Step 1: Identify the PDF and its parameter.
The PDF is $f(x) = 0.5 e^{-0.5x}$ for $x \ge 0$. This is an exponential distribution with rate parameter $\lambda = 0.5$.

Step 2: Apply the formula for the expectation of a continuous random variable.

This is the mean of an exponential distribution, which is $1/\lambda$.

Step 3: Calculate the expectation.

The expected lifespan is 2 years.
Answer: \boxed{2}"
:::

:::question type="MCQ" question="Let $X$ be a discrete random variable with $P(X=1) = 0.2$, $P(X=2) = 0.3$, and $P(X=3) = 0.5$. Which of the following statements about $V(X)$ is correct?" options=["$V(X) = 0.61$","$V(X) = 0.76$","$V(X) = 1.69$","$V(X) = 2.3$"] answer="$V(X) = 0.61$" hint="First calculate $E[X]$ and $E[X^2]$, then use the formula $V(X) = E[X^2] - (E[X])^2$." solution="Step 1: Calculate $E[X]$.

Step 2: Calculate $E[X^2]$.

Step 3: Calculate $V(X)$.

Answer: \boxed{0.61}"
:::

:::question type="SUB" question="A bag contains 5 red balls and 5 blue balls. Three balls are drawn without replacement. Let $Y$ be the number of blue balls drawn. Calculate $E[Y]$ using indicator random variables." answer="$E[Y] = 1.5$" hint="Define an indicator variable for each draw, then use linearity of expectation." solution="Step 1: Define indicator random variables.
Let $Y_1$, $Y_2$, $Y_3$ be indicator variables for the first, second, and third ball drawn being blue, respectively.

Step 2: Express $Y$ as a sum of indicator variables.

Step 3: Calculate the expectation of each indicator variable.
For $Y_1$:

For $Y_2$:
By symmetry, the probability that the second ball drawn is blue is the same as the first. Alternatively, using Law of Total Probability:

So,

For $Y_3$:
Similarly,

Step 4: Apply linearity of expectation.

Answer: \boxed{1.5}"
:::

:::question type="MSQ" question="Let $X$ be a random variable with $E[X]=5$ and $V(X)=4$. Which of the following statements are correct?" options=["$E[2X+3] = 13$","$V(2X+3) = 16$","$E[X^2] = 29$","$P(|X-5| \ge 4) \le 1/4$"] answer="A,B,C,D" hint="Apply the properties of expectation and variance, and Chebyshev's inequality." solution="Let's evaluate each option:

Option A: $E[2X+3] = 13$
Using linearity of expectation:

Given $E[X]=5$:

This statement is correct.

Option B: $V(2X+3) = 16$
Using properties of variance:

Given $V(X)=4$:

This statement is correct.

Option C: $E[X^2] = 29$
Using the computational formula for variance:

We are given $V(X)=4$ and $E[X]=5$.



This statement is correct.

Option D: $P(|X-5| \ge 4) \le 1/4$
Using Chebyshev's Inequality:

Here, $E[X]=5$, $V(X)=4$, and $k=4$.



This statement is correct.

All statements are correct.
Answer: \boxed{A,B,C,D}"
:::

:::question type="NAT" question="A discrete random variable $Y$ has $P(Y=0)=0.4$, $P(Y=1)=0.3$, and $P(Y=2)=0.3$. What is the standard deviation of $Y$ (round to two decimal places)?" answer="0.83" hint="Calculate $E[Y]$ and $E[Y^2]$, then $V(Y)$, and finally $\sigma_Y = \sqrt{V(Y)}$." solution="Step 1: Calculate $E[Y]$.

Step 2: Calculate $E[Y^2]$.

Step 3: Calculate $V(Y)$.

Step 4: Calculate the standard deviation $\sigma_Y$.

Rounding to two decimal places, $\sigma_Y = 0.83$.
Answer: \boxed{0.83}"
:::

---

Summary

---

What's Next?

---

---

Part 4: Standard Distributions

Introduction

Standard distributions are fundamental building blocks in probability theory and statistics, providing models for a wide array of random phenomena encountered in data science. Each distribution describes the probabilities of different outcomes for a specific type of random variable, characterized by its parameters. Understanding these distributions is crucial for modeling real-world data, performing statistical inference, and making informed decisions.

In the CMI exam, a deep understanding of standard discrete and continuous distributions is essential. This includes knowing their probability mass/density functions, cumulative distribution functions, expected values, variances, and how to apply them to calculate probabilities and estimate parameters in various scenarios. Mastery of these concepts forms the bedrock for advanced topics like hypothesis testing, regression analysis, and machine learning algorithms.

---

Key Concepts

1. Discrete Distributions

Discrete distributions model scenarios where the outcomes are countable.

1.1 Bernoulli Distribution

The Bernoulli distribution models a single trial with two possible outcomes: "success" (usually denoted by 1) or "failure" (usually denoted by 0).

Properties:

• Mean: $E[X] = p$


• Variance: $Var(X) = p(1-p)$

---

1.2 Binomial Distribution

The Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials.

Properties:

• Mean: $E[X] = np$


• Variance: $Var(X) = np(1-p)$

Worked Example:

Problem: A fair coin is tossed 10 times. What is the probability of getting exactly 7 heads?

Solution:

Step 1: Identify parameters for Binomial distribution.
Here, $n=10$ (number of tosses), $k=7$ (number of heads), and $p=0.5$ (probability of heads for a fair coin).

Step 2: Apply the Binomial PMF.

Step 3: Calculate the binomial coefficient and simplify.

Answer: $0.1171875$

---

1.3 Poisson Distribution

The Poisson distribution models the number of events occurring in a fixed interval of time or space, given a constant average rate ($\lambda$) of occurrence and that these events occur independently.

Properties:

• Mean: $E[X] = \lambda$


• Variance: $Var(X) = \lambda$

Worked Example:

Problem: A call center receives an average of 5 calls per hour. What is the probability of receiving exactly 3 calls in the next hour?

Solution:

Step 1: Identify the parameter for Poisson distribution.
Here, $\lambda = 5$ (average calls per hour), and $k=3$ (number of calls).

Step 2: Apply the Poisson PMF.

Step 3: Calculate the terms and simplify.

Answer: $0.1403729$

---

2. Continuous Distributions

Continuous distributions model scenarios where the outcomes can take any value within a range.

2.1 Uniform Distribution

The Uniform distribution assigns equal probability to all values within a specified interval $[a, b]$.

Properties:

• Mean: $E[X] = \frac{a+b}{2}$


• Variance: $Var(X) = \frac{(b-a)^2}{12}$

Worked Example:

Problem: A random variable $X$ is uniformly distributed between 0 and 10. What is the probability that $X$ is between 3 and 7?

Solution:

Step 1: Identify parameters.
Here, $a=0$, $b=10$. We want to find $P(3 < X < 7)$.

Step 2: Use the PDF or CDF. Using PDF:

Step 3: Evaluate the integral.

Answer: $0.4$

---

2.2 Exponential Distribution

The Exponential distribution models the time until an event occurs in a Poisson process, where events occur continuously and independently at a constant average rate. It is memoryless.

Properties:

• Mean: $E[X] = \frac{1}{\lambda}$


• Variance: $Var(X) = \frac{1}{\lambda^2}$


• Memoryless Property: $P(X > s+t | X > s) = P(X > t)$. The future waiting time does not depend on past waiting time.

Worked Example:

Problem: The lifespan of a certain electronic component follows an exponential distribution with a mean lifespan of 5 years. What is the probability that a component will last less than 3 years?

Solution:

Step 1: Determine the rate parameter $\lambda$.
Given mean $E[X] = 5$ years. For exponential distribution, $E[X] = 1/\lambda$.

Step 2: Use the CDF to find $P(X < 3)$.

Answer: $0.4512$

---

2.3 Normal (Gaussian) Distribution

The Normal distribution is arguably the most important distribution in statistics. It is symmetric, bell-shaped, and characterized by its mean ($\mu$) and standard deviation ($\sigma$). Many natural phenomena follow this distribution, and it is central to the Central Limit Theorem.

Properties:

• Mean: $E[X] = \mu$


• Variance: $Var(X) = \sigma^2$


• Median = Mode = Mean = $\mu$.


• The curve is symmetric about $\mu$.


• The total area under the curve is 1.

Worked Example (Probability Calculation):

Problem: The height of adult males in a city is normally distributed with a mean of 175 cm and a standard deviation of 7 cm. What is the probability that a randomly selected male is between 168 cm and 182 cm tall? (Use $\Phi(1)=0.8413$, $\Phi(-1)=0.1587$)

Solution:

Step 1: Identify parameters and values.
$\mu = 175$, $\sigma = 7$. We want to find $P(168 < X < 182)$.

Step 2: Standardize the values.
For $x_1 = 168$:

For $x_2 = 182$:

Step 3: Use the Standard Normal CDF ($\Phi$) to find the probability.

Answer: $0.6826$

$\mu$


$x_1$

$x_2$


$P(x_1 < X < x_2)$
$X$

Worked Example (CLT and Parameter Estimation):

Problem: Suppose the individual scores on an exam are normally distributed with an unknown mean $\mu$ and standard deviation $\sigma$. A candidate fails if they score below 35% and passes with distinction if they score above 80%. In a large group, 16% fail and 2% pass with distinction. Find $\mu$ and $\sigma$. (Use $\Phi(-1)=0.16$, $\Phi(2)=0.98$)

Solution:

Step 1: Set up equations based on the given probabilities and Z-scores.
Let $X$ be the score on the exam.
We are given $P(X < 35) = 0.16$.
Standardize $X=35$: $Z_1 = \frac{35 - \mu}{\sigma}$.
From the Z-table, $P(Z < -1) = 0.16$, so $Z_1 = -1$.

We are given $P(X > 80) = 0.02$.
This means $P(X \le 80) = 1 - 0.02 = 0.98$.
Standardize $X=80$: $Z_2 = \frac{80 - \mu}{\sigma}$.
From the Z-table, $P(Z \le 2) = 0.98$, so $Z_2 = 2$.

Step 2: Solve the system of linear equations.
From Equation 1:

From Equation 2:

Subtract Equation 3 from Equation 4:

Substitute $\sigma=15$ into Equation 3:

Answer: $\mu = 50$ and $\sigma = 15$.

---

2.4 Gamma Distribution

The Gamma distribution is a versatile continuous distribution that generalizes the exponential distribution. It is often used to model waiting times for multiple events or the sum of independent exponentially distributed random variables.

Properties:

• Mean: $E[X] = \frac{\alpha}{\beta}$


• Variance: $Var(X) = \frac{\alpha}{\beta^2}$


• If $\alpha=1$, the Gamma distribution reduces to the Exponential distribution with rate $\beta$.

Worked Example:

Problem: The lifespan of a device follows a Gamma distribution. Historical data suggests the mean lifespan is 6 years and the variance is 12 years$^2$. Find the parameters $\alpha$ and $\beta$ of this distribution.

Solution:

Step 1: Write down the equations for mean and variance in terms of $\alpha$ and $\beta$.
$E[X] = \frac{\alpha}{\beta} = 6$
$Var(X) = \frac{\alpha}{\beta^2} = 12$

Step 2: Solve the system of equations for $\alpha$ and $\beta$.
From the mean equation, $\alpha = 6\beta$.
Substitute this into the variance equation:

Now substitute $\beta = 0.5$ back into the equation for $\alpha$:

Answer: $\alpha = 3$ and $\beta = 0.5$.

---

2.5 Beta Distribution

The Beta distribution is a continuous probability distribution defined on the interval $[0, 1]$. It is particularly useful for modeling probabilities or proportions, as its values are naturally constrained within this range.

Properties:

• Mean: $E[X] = \frac{\alpha}{\alpha+\beta}$


• Variance: $Var(X) = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$


• Mode: $Mode(X) = \frac{\alpha-1}{\alpha+\beta-2}$ (for $\alpha > 1, \beta > 1$)

Worked Example:

Problem: The proportion of defective items produced by a machine follows a Beta distribution. From historical data, the mean proportion of defective items is 0.25 and the mode is 0.20. Find the parameters $\alpha$ and $\beta$ of this distribution.

Solution:

Step 1: Write down the equations for mean and mode in terms of $\alpha$ and $\beta$.
$E[X] = \frac{\alpha}{\alpha+\beta} = 0.25$
$Mode(X) = \frac{\alpha-1}{\alpha+\beta-2} = 0.20$

Step 2: Solve the system of equations.
From the mean equation:

Substitute $\beta = 3\alpha$ into the mode equation:

Substitute $\alpha=3$ back into Equation 1:

Answer: $\alpha = 3$ and $\beta = 9$.

---

Problem-Solving Strategies

---

Common Mistakes

---

Practice Questions

:::question type="MCQ" question="A call center receives calls at an average rate of 20 calls per hour. What is the probability that exactly 15 calls are received in a 30-minute interval?" options=["$\frac{e^{-10} 10^{15}}{15!}$","$\frac{e^{-20} 20^{15}}{15!}$","$\frac{e^{-10} 15^{10}}{10!}$","$\frac{e^{-20} 15^{20}}{20!}$"] answer="A" hint="Adjust the average rate to match the given time interval before applying the Poisson PMF." solution="Step 1: Determine the average rate for the given interval.
The average rate is 20 calls per hour. For a 30-minute interval (0.5 hours), the average rate $\lambda$ will be:

Step 2: Apply the Poisson Probability Mass Function (PMF).
The Poisson PMF is $P(X=k) = \frac{e^{-\lambda} \lambda^k}{k!}$.
Here, $\lambda = 10$ and $k = 15$.

The correct option is A." :::

:::question type="NAT" question="The scores on a standardized test are normally distributed with a mean of 600 and a standard deviation of 100. If a student scores 750, what is their Z-score? Report to two decimal places." answer="1.50" hint="Use the Z-score formula $Z = (X - \mu) / \sigma$." solution="Step 1: Identify the given values.
$X = 750$ (student's score)
$\mu = 600$ (mean score)
$\sigma = 100$ (standard deviation)

Step 2: Apply the Z-score formula.

The Z-score is 1.50."
:::

:::question type="MSQ" question="A quality control process inspects batches of 50 items. Each item has a 1% chance of being defective, independently. Which of the following statements are correct?" options=["The number of defective items in a batch follows a Binomial distribution.","The probability of finding exactly 1 defective item in a batch is $\binom{50}{1} (0.01)^1 (0.99)^{49}$.","The mean number of defective items in a batch is 0.5.","The Poisson approximation to this distribution would use $\lambda = 50$."] answer="A,B,C" hint="Identify the distribution type and its parameters. Check the conditions for Poisson approximation." solution="Statement A: The number of defective items in a fixed number of independent trials (50 items) with a constant probability of success (1% defect) follows a Binomial distribution. This is correct.

Statement B: For a Binomial distribution $B(n,p)$, $P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$. Here $n=50$, $p=0.01$, $k=1$. So $P(X=1) = \binom{50}{1} (0.01)^1 (0.99)^{49}$. This is correct.

Statement C: The mean of a Binomial distribution is $E[X] = np$. Here $E[X] = 50 \times 0.01 = 0.5$. This is correct.

Statement D: The Poisson approximation to a Binomial distribution uses $\lambda = np$. Here $\lambda = 50 \times 0.01 = 0.5$. The statement says $\lambda=50$, which is incorrect. This is incorrect.

Therefore, statements A, B, and C are correct."
:::

:::question type="SUB" question="The time (in minutes) a customer spends waiting for a service representative follows an exponential distribution. If 80% of customers wait longer than 5 minutes, what is the average waiting time (in minutes)? Report to two decimal places." answer="22.40" hint="Use the CDF of the exponential distribution and its relationship with the mean." solution="Step 1: Set up the probability statement using the Exponential CDF.
Let $X$ be the waiting time. $X \sim \operatorname{Exp}(\lambda)$.
We are given $P(X > 5) = 0.80$.
We know $P(X > x) = e^{-\lambda x}$.

So, $e^{-5\lambda} = 0.80$.

Step 2: Solve for $\lambda$.
Take the natural logarithm of both sides:

Step 3: Calculate the average waiting time (mean).
For an Exponential distribution, the mean is $E[X] = 1/\lambda$.

Rounding to two decimal places, $E[X] \approx 22.40$.
The average waiting time is 22.40 minutes."
:::

---

Chapter Summary

---

Chapter Review Questions

:::question type="MCQ" question="Let $X$ be a continuous random variable with probability density function (PDF):

Which of the following statements is TRUE?
(I) The constant $c = \frac{3}{4}$.
(II) $P(X > 0) = \frac{1}{2}$.
(III) $E[X] = 0$.
(IV) $Var[X] = \frac{2}{5}$.
" options=["A) (I) and (II) only" "B) (I), (II) and (III) only" "C) (I), (II), (III) and (IV)" "D) (I), (III) and (IV) only"] answer="B" hint="Remember the properties of a PDF: it must integrate to 1. Also, leverage symmetry to simplify calculations for expectation and probability." solution="Let's analyze each statement:

(I) The constant $c = \frac{3}{4}$:
For $f_X(x)$ to be a valid PDF, $\int_{-\infty}^{\infty} f_X(x) dx = 1$.

So, statement (I) is TRUE.

(II) $P(X > 0) = \frac{1}{2}$:

Alternatively, since $f_X(x)$ is symmetric about $x=0$ ($f_X(x) = f_X(-x)$), $P(X > 0) = P(X < 0) = \frac{1}{2}$.
So, statement (II) is TRUE.

(III) $E[X] = 0$:

Since $(x-x^3)$ is an odd function and the integration interval is symmetric about 0, the integral is 0.
So, statement (III) is TRUE.

(IV) $Var[X] = \frac{2}{5}$:
$Var[X] = E[X^2] - (E[X])^2$. Since $E[X]=0$, $Var[X] = E[X^2]$.

Since $(x^2-x^4)$ is an even function, we can write:




So, $Var[X] = \frac{1}{5}$. Therefore, statement (IV) $Var[X] = \frac{2}{5}$ is FALSE.

Based on the analysis, statements (I), (II), and (III) are TRUE. The correct option is B."
:::

:::question type="NAT" question="A fair six-sided die is rolled repeatedly. Let $X$ be the number of rolls until a '6' appears for the first time. Let $Y$ be the number of rolls until a '6' appears for the second time. Find $E[Y|X=1]$. (Enter your answer as a plain number)." answer="7" hint="Consider the nature of the geometric distribution and the memoryless property. If the first '6' occurs on the 1st roll, how many additional rolls are needed for the second '6'?" solution="Let $X$ be the number of rolls until the first '6' appears. $X$ follows a Geometric distribution with $p = 1/6$.
Let $Y$ be the number of rolls until the second '6' appears.

We are asked to find $E[Y|X=1]$.
Given that the first '6' appeared on the 1st roll, this means roll 1 was a '6'.
Now, we need to find the expected number of additional rolls from roll 2 onwards until the second '6' appears.
Let $Z$ be the number of additional rolls needed after the 1st roll for the second '6' to appear.
Since die rolls are independent and the probability of rolling a '6' remains $p=1/6$ for each subsequent roll, $Z$ also follows a Geometric distribution with parameter $p=1/6$.
The expected value of a Geometric distribution (number of trials until the first success) is $1/p$.
So, $E[Z] = 1/p = 1/(1/6) = 6$.

The total number of rolls until the second '6' appears, $Y$, can be expressed as $Y = X + Z$.
Therefore, $E[Y|X=1] = E[1 + Z | X=1]$.
Since $1$ is a fixed value in the conditional expectation, and $Z$ is independent of $X=1$ (due to the memoryless property of the process), we have:
$E[Y|X=1] = 1 + E[Z]$
$E[Y|X=1] = 1 + 6 = 7$.

The expected value is 7."
:::

:::question type="MCQ" question="Let $X$ be a random variable with Moment Generating Function (MGF) $M_X(t) = \frac{e^{2t}}{1-3t}$ for $t < \frac{1}{3}$.
Which of the following is the variance of $X$, $Var[X]$?
" options=["A) 3" "B) 9" "C) 11" "D) 13"] answer="B" hint="Recall that $M_X'(0) = E[X]$ and $M_X''(0) = E[X^2]$. Then use $Var[X] = E[X^2] - (E[X])^2$." solution="The Moment Generating Function (MGF) is given by $M_X(t) = e^{2t}(1-3t)^{-1}$.

First, we find $E[X] = M_X'(0)$.
Using the product rule $(uv)' = u'v + uv'$:
Let $u = e^{2t}$ and $v = (1-3t)^{-1}$.
Then $u' = 2e^{2t}$ and $v' = -1(1-3t)^{-2}(-3) = 3(1-3t)^{-2}$.

So, $M_X'(t) = (2e^{2t})(1-3t)^{-1} + (e^{2t})(3(1-3t)^{-2})$.
Now, evaluate at $t=0$:
$M_X'(0) = (2e^0)(1-0)^{-1} + (e^0)(3(1-0)^{-2})$
$M_X'(0) = 2(1)(1) + 1(3)(1) = 2 + 3 = 5$.
So, $E[X] = 5$.

Next, we find $E[X^2] = M_X''(0)$.
We need to differentiate $M_X'(t) = 2e^{2t}(1-3t)^{-1} + 3e^{2t}(1-3t)^{-2}$.
Let $M_X'(t) = A(t) + B(t)$, where $A(t) = 2e^{2t}(1-3t)^{-1}$ and $B(t) = 3e^{2t}(1-3t)^{-2}$.

For $A(t)$:
$A'(t) = (2e^{2t} \cdot 2)(1-3t)^{-1} + (2e^{2t})(-1(1-3t)^{-2}(-3))$
$A'(t) = 4e^{2t}(1-3t)^{-1} + 6e^{2t}(1-3t)^{-2}$.
At $t=0$: $A'(0) = 4(1)(1) + 6(1)(1) = 4+6=10$.

For $B(t)$:
$B'(t) = (3e^{2t} \cdot 2)(1-3t)^{-2} + (3e^{2t})(-2(1-3t)^{-3}(-3))$
$B'(t) = 6e^{2t}(1-3t)^{-2} + 18e^{2t}(1-3t)^{-3}$.
At $t=0$: $B'(0) = 6(1)(1) + 18(1)(1) = 6+18=24$.

So, $M_X''(0) = A'(0) + B'(0) = 10 + 24 = 34$.
Thus, $E[X^2] = 34$.

Finally, $Var[X] = E[X^2] - (E[X])^2$.
$Var[X] = 34 - (5)^2 = 34 - 25 = 9$.

The correct option is B.

Alternatively, recognize the MGF.
The MGF of an Exponential distribution with rate $\lambda$ is $\frac{\lambda}{\lambda-t}$.
The MGF of $X_0 \sim \operatorname{Exp}(\lambda)$ is $M_{X_0}(t) = \frac{\lambda}{\lambda-t}$.
The MGF of $X_1 \sim \operatorname{Exp}(1/3)$ is $\frac{1/3}{1/3-t} = \frac{1}{1-3t}$.
The MGF of $Y = X_1 + 2$ is $E[e^{t(X_1+2)}] = E[e^{tX_1} e^{2t}] = e^{2t} E[e^{tX_1}] = e^{2t} \frac{1}{1-3t}$.
So $X$ is distributed as an $\operatorname{Exp}(1/3)$ random variable shifted by 2.
Let $X_1 \sim \operatorname{Exp}(1/3)$. Then $E[X_1] = 1/\lambda = 3$ and $Var[X_1] = 1/\lambda^2 = 9$.
$X = X_1 + 2$.
$E[X] = E[X_1+2] = E[X_1] + 2 = 3+2 = 5$.
$Var[X] = Var[X_1+2] = Var[X_1] = 9$.
The variance of $X$ is 9."
:::

:::question type="NAT" question="Let $X$ be a continuous random variable uniformly distributed on the interval $(0, 2)$. Define a new random variable $Y = X^2$. Find $E[Y]$. (Enter your answer as a plain number in decimal form, rounded to two decimal places)." answer="1.33" hint="First, determine the PDF of $X$. Then, use the Law of the Unconscious Statistician (LOTUS) to compute $E[Y]$ without explicitly finding the PDF of $Y$." solution="The random variable $X$ is uniformly distributed on $(0, 2)$.
Its PDF is given by:

We want to find $E[Y]$ where $Y = X^2$.
Using the Law of the Unconscious Statistician (LOTUS), $E[g(X)] = \int_{-\infty}^{\infty} g(x) f_X(x) dx$.
Here, $g(x) = x^2$.






Now, we need to calculate the numerical value and round it to two decimal places.
$E[Y] = \frac{4}{3} \approx 1.3333...$
Rounded to two decimal places, $E[Y] \approx 1.33$.
The expected value of $Y$ is 1.33."
:::

---

What's Next?