Statistical Inference | AP - Wyatt's Notes

Statistical inference uses sample data to draw conclusions about a population. There are two main types: confidence intervals (estimating a parameter) and hypothesis tests (testing a claim about a parameter).

A confidence interval provides a range of plausible values for an unknown population parameter, along with a level of confidence.

$\text{Statistic} \pm \text{Margin of Error} = (\text{point estimate}) \pm (\text{critical value}) \times (\text{standard error})$

The confidence level (90%, 95%, or 99% on the AP exam) is the long-run proportion of intervals that would capture the true parameter if the sampling process were repeated many times. It does not mean there is a 95% probability that the specific interval contains the parameter.

Conditions:

• Random sample (or random assignment)
• $np \geq 10$ and $n(1-p) \geq 10$ (large enough sample)
• Population is at least $10n$ (independence / 10% condition)

$\hat{p} \pm z^{*}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$

Where $\hat{p}$ is the sample proportion and $z^*$ is the critical value (1.645 for 90%, 1.960 for 95%, 2.576 for 99%).

Conditions:

• Random sample
• Population distribution is approximately normal (check with normal probability plot, or $n \geq 30$ for CLT)
• $\sigma$ known: $z$-interval; $\sigma$ unknown: $t$-interval

$\bar{x} \pm t^{*} \cdot \frac{s}{\sqrt{n}}$

Where $t^*$ is the critical value from the $t$-distribution with $n-1$ degrees of freedom.

• “We are 95% confident that the true population proportion is between 0.42 and 0.58” means that the method produces an interval that captures the true parameter 95% of the time
• The margin of error decreases with larger sample size and lower confidence level
• Wider intervals give more confidence but less precision

A hypothesis test evaluates whether the observed sample data provides evidence for or against a claim about a population parameter.

1. State hypotheses: Null ($H_0$) and alternative ($H_a$)
2. Check conditions: Determine the appropriate test
3. Calculate the test statistic: Measure how far the observed statistic is from the null
4. Find the p-value: Probability of obtaining a result at least as extreme as the observed, assuming $H_0$ is true
5. Make a decision: Compare p-value to the significance level $\alpha$
6. State conclusion in context

• $H_0$ (null hypothesis): The “status quo” — the parameter equals a specific value (e.g., $p = 0.5$, $\mu = 100$)
• $H_a$ (alternative hypothesis): What we are trying to find evidence for (e.g., $p \neq 0.5$, $p > 0.5$, $\mu < 100$)

The significance level is the threshold for deciding whether the evidence against $H_0$ is strong enough. Common values: $\alpha = 0.10, 0.05, 0.01$.

The p-value is the probability of obtaining a test statistic as extreme as or more extreme than the observed value, assuming $H_0$ is true.

• Small p-value ($< \alpha$): Strong evidence against $H_0$; reject $H_0$
• Large p-value ($\geq \alpha$): Insufficient evidence against $H_0$; fail to reject $H_0$

• Type I Error: Rejecting $H_0$ when it is actually true (false positive). Probability = $\alpha$
• Type II Error: Failing to reject $H_0$ when it is actually false (false negative). Probability = $\beta$
• Power ($1 - \beta$): The probability of correctly rejecting a false $H_0$ (detecting a real effect)

Power increases with: larger sample size, larger effect size, higher $\alpha$, and lower $\sigma$.

$z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$

$z = \frac{(\hat{p}_1 - \hat{p}_2) - 0}{\sqrt{\hat{p}(1-\hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$

Where $\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}$ is the pooled proportion.

$t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}$

Degrees of freedom: $df = n - 1$

$t = \frac{(\bar{x}_1 - \bar{x}_2) - 0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

Use conservative degrees of freedom: $df = \min(n_1 - 1, n_2 - 1)$ or technology.

For matched pairs or before/after data: compute the differences $d = x_1 - x_2$ and run a one-sample t-test on the differences.

$t = \frac{\bar{d} - 0}{s_d / \sqrt{n}}$

Tests whether observed frequencies match expected frequencies based on a specified distribution.

$\chi^2 = \sum \frac{(O - E)^2}{E}$

$df = \text{number of categories} - 1$

Tests whether two categorical variables are independent using a two-way table.

$\chi^2 = \sum \frac{(O - E)^2}{E}$

$df = (\text{rows} - 1)(\text{columns} - 1)$

Expected count for each cell: $E = \frac{\text{row total} \times \text{column total}}{\text{grand total}}$

Conditions: All expected counts $\geq 5$.

Tests whether the distribution of one categorical variable is the same across several populations. Same formula and conditions as the test for independence, but sampling is from separate populations rather than one population classified two ways.

$H_0: \beta_1 = 0 \quad \text{(no linear relationship)}$ $H_a: \beta_1 \neq 0 \quad \text{(linear relationship exists)}$

$t = \frac{b_1 - 0}{SE_{b_1}}$

$df = n - 2$

flowchart TD
    A[4_Statistical Inference] --> B[Key Concepts]
    A --> C[Core Principles]
    A --> D[Practical Applications]
    B --> E[Fundamental definitions]
    C --> F[Design patterns]
    D --> G[Real-world usage]

Statistical inference is about drawing conclusions about a population from a sample while quantifying how confident you should be. The two main tools are confidence intervals (estimation) and hypothesis testing (decision-making).

Confidence interval intuition: A 95% confidence interval means that if we repeated the sampling process many times, about 95% of the intervals would contain the true parameter. It does NOT mean there’s a 95% probability the parameter is in this specific interval — the parameter is fixed, and the interval is random.

Hypothesis testing intuition: We start by assuming the null hypothesis is true (innocent until proven guilty). We then ask: “If the null were true, how likely is it that we’d see data this extreme?” If the answer is very unlikely (small p-value), we reject the null. The p-value is NOT the probability the null is true — it’s the probability of the data given the null.

Type I vs Type II errors: Type I (false positive) means rejecting a true null hypothesis. Type II (false negative) means failing to reject a false null hypothesis. You can’t reduce both simultaneously — lowering the significance level reduces Type I errors but increases Type II errors.

The power of a test: Power = $1 - P(\text{Type II error})$. the answer varies based on on the effect size, sample size, significance level, and variability. Larger samples and larger effects are easier to detect.

• Confusing the p-value with the probability that $H_0$ is true
• Saying “accept $H_0$” instead of “fail to reject $H_0$”
• Forgetting to check conditions before performing inference
• Interpreting a confidence interval as “there is a 95% probability the parameter is in this interval”
• Using a z-test for a mean when $\sigma$ is unknown (should use t-test)
• Confusing Type I and Type II errors

• Probability: The Central Limit Theorem and normal distributions provide the mathematical foundation for all inference procedures.
• Exploring Data: Understanding your data’s shape and spread is essential before choosing the right inference procedure.
• Regression: Confidence intervals and hypothesis tests for slope use the same inference framework.