What is the difference between $Var(X + X)$ and $Var(X_1 + X_2)$ where $X_1, X_2$ are independent and identically distributed?

$Var(X + X)$ is $Var(2X) = 4Var(X)$, representing a single variable being scaled. $Var(X_1 + X_2) = Var(X_1) + Var(X_2) = 2Var(X)$, representing the sum of two independent sources of variation.

When calculating $Var(aX - bY)$ for independent variables, why is the sign between the variances positive?

Because the coefficient $-b$ is squared in the variance formula: $(-b)^2 = b^2$. Conceptually, combining two independent random processes always increases the total uncertainty, regardless of whether you add or subtract them.

How does the calculation of $E[aX + bY]$ differ when $X$ and $Y$ are dependent versus independent?

It does not differ at all. The Linearity of Expectation property $E[aX + bY] = aE[X] + bE[Y]$ holds true for all random variables, whether they are independent or dependent.

What is a common mistake when calculating the variance of $3X$?

A common error is forgetting to square the coefficient, resulting in $3Var(X)$ instead of the correct $3^2Var(X) = 9Var(X)$. Variance measures squared units, so scaling the variable scales the variance by the square of that factor.

Why is it incorrect to say $\sigma_{X+Y} = \sigma_X + \sigma_Y$ for independent variables?

Standard deviations do not add linearly; variances do. The correct relationship is $\sigma_{X+Y} = \sqrt{\sigma_X^2 + \sigma_Y^2}$, which is always less than the simple sum of standard deviations unless one is zero.

What happens to the variance if you add a constant $c$ to a random variable $X$?

The variance remains unchanged ($Var(X+c) = Var(X)$). Adding a constant shifts the entire distribution but does not change the spread or 'width' of the data points relative to each other.

Define a 'Linear Combination' of random variables.

A linear combination is a new random variable formed by multiplying one or more random variables by constants and adding them together, often in the form $Y = \sum a_iX_i + b$.

What is the formula for the variance of $aX + bY$ if $X$ and $Y$ are NOT independent?

The formula is $Var(aX + bY) = a^2Var(X) + b^2Var(Y) + 2abCov(X, Y)$. The covariance term accounts for the linear relationship between the two variables.

If $X$ and $Y$ are independent normal random variables, what is the distribution of $X + Y$?

The distribution of $X + Y$ is also a normal distribution. Its mean is $\mu_X + \mu_Y$ and its variance is $\sigma_X^2 + \sigma_Y^2$.

State the formula for $E[aX + b]$.

$E[aX + b] = aE[X] + b$. The expected value is linear, meaning constants can be pulled out and additive constants remain additive.

Library Podcasts

Courses

Referral & Rewards

Revision Notes

College Board

Statistics

Unit 4: Probability, Random Variables & Probability Distributions

Linear Combinations of Random Variables

Summary

Linear combinations involve creating a new random variable by scaling and summing existing random variables. This concept is fundamental for understanding how the mean and variance of a system are affected by the individual components, providing the mathematical basis for portfolio theory, error propagation, and statistical sampling.

1. Definition & Core Concepts

2. Linearity of Expectation

The Expected Value (mean) of a linear combination is equal to the linear combination of the expected values of the individual variables.

Mathematically, this is expressed as $E[aX + bY + c] = aE[X] + bE[Y] + c$ . This property is known as the Linearity of Expectation.

Crucially, this principle holds true regardless of whether the random variables are independent or dependent; the total average is always the weighted sum of the individual averages.

3. Variance of Independent Combinations

A right-angled triangle illustrating that for independent variables, the total standard deviation is the hypotenuse of a triangle where the legs are the individual standard deviations.

4. Variance of Dependent Combinations

5. Key Distinctions

6. Exam Strategy & Tips

Linear Combinations of Random Variables

Summary

1. Definition & Core Concepts

A linear combination of random variables $X_1, X_2, \dots, X_n$ is a new random variable $Y$ defined by the equation $Y = a_1X_1 + a_2X_2 + \dots + a_nX_n + b$ , where $a_i$ and $b$ are constants.

The constants $a_i$ represent scaling factors or weights applied to each individual variable, while $b$ represents a constant shift or intercept added to the total.

This mathematical structure allows statisticians to model complex scenarios, such as the total weight of a shipment containing different types of items or the total return on a financial portfolio.

2. Linearity of Expectation

The Expected Value (mean) of a linear combination is equal to the linear combination of the expected values of the individual variables.

Mathematically, this is expressed as $E[aX + bY + c] = aE[X] + bE[Y] + c$ . This property is known as the Linearity of Expectation.

Crucially, this principle holds true regardless of whether the random variables are independent or dependent; the total average is always the weighted sum of the individual averages.

3. Variance of Independent Combinations

When random variables are independent, the variance of their sum is the sum of their scaled variances.

The formula for the variance of a linear combination of independent variables is $Var(aX + bY) = a^2Var(X) + b^2Var(Y)$ . Note that the coefficients are squared because variance is a measure of squared units.

Even if the variables are subtracted (e.g., $X - Y$ ), the variances are still added because $Var(aX - bY) = a^2Var(X) + (-b)^2Var(Y) = a^2Var(X) + b^2Var(Y)$ . Variability always increases when combining independent sources of error.

The standard deviation of the combination is the square root of the resulting variance: $\sigma_{aX+bY} = \sqrt{a^2\sigma_X^2 + b^2\sigma_Y^2}$ .

A right-angled triangle illustrating that for independent variables, the total standard deviation is the hypotenuse of a triangle where the legs are the individual standard deviations.

4. Variance of Dependent Combinations

If the random variables are dependent, the variance formula must include a covariance term to account for their relationship.

The general formula is $Var(aX + bY) = a^2Var(X) + b^2Var(Y) + 2abCov(X, Y)$ , where $Cov(X, Y)$ measures how much the variables change together.

If $X$ and $Y$ are positively correlated, the total variance is higher than the sum of individual variances; if negatively correlated, the total variance is reduced.

5. Key Distinctions

It is critical to distinguish between $Var(X + X)$ and $Var(X_1 + X_2)$ .

Scenario	Expression	Result	Explanation
Scaling	$Var(2X)$	$4Var(X)$	The variable is stretched; every deviation is doubled, so squared deviation is quadrupled.
Summing	$Var(X_1 + X_2)$	$2Var(X)$	Two independent variables with the same distribution. Their errors may partially cancel out, leading to less growth in variance.

Always verify the independence assumption before dropping the covariance term from a variance calculation.

6. Exam Strategy & Tips

Check for Independence: In exam questions, look for keywords like 'independent', 'randomly selected', or 'uncorrelated' before using the simple variance sum formula.
The Square Rule: Always remember to square the coefficients when calculating variance ( $a^2$ ), but do NOT square them when calculating the mean ( $a$ ).
Subtraction is Addition: For variance, $Var(X - Y)$ is $Var(X) + Var(Y)$ , not $Var(X) - Var(Y)$ . Variability cannot be 'subtracted' away by adding more random components.
Units Check: Ensure your final answer for standard deviation has the same units as the original mean. If you are working with variance, the units are squared.