What is the primary difference between statistical significance and practical significance?

Statistical significance determines if an effect is likely due to chance (using p-values), while practical significance determines if the magnitude of that effect is large enough to be useful or meaningful in real-world applications.

How does correlation differ from causation in the context of drawing conclusions?

Correlation indicates that two variables change together in a predictable way, whereas causation implies that a change in one variable directly produces a change in the other. Causation requires controlled experimentation to prove, while correlation can be observed in any data set.

What is the difference between a Type I and a Type II error?

A Type I error occurs when you reject a true null hypothesis (a 'false positive'), while a Type II error occurs when you fail to reject a false null hypothesis (a 'false negative'). Both represent failures in the conclusion-drawing process.

Why is 'overgeneralization' considered a major error in evaluating results?

Overgeneralization occurs when a conclusion is applied to a group or context that was not represented in the study's sample. This leads to inaccurate predictions because the findings may not hold true under different conditions or for different populations.

What error is committed when a researcher ignores 'outliers' without justification?

Ignoring outliers can lead to an artificial inflation or deflation of the results, creating a conclusion that does not reflect the true variance of the data. Outliers should only be removed if they are proven to be errors in measurement or recording.

What is the danger of 'p-hacking' or selective reporting in drawing conclusions?

P-hacking involves manipulating data or analyses until a significant p-value is found, which leads to false conclusions. It undermines the scientific method by presenting random noise as a meaningful discovery.

Define the 'Null Hypothesis' ($H_0$) in the context of evaluating results.

The null hypothesis is the default assumption that there is no relationship between variables or no effect from a treatment. Evaluating results involves determining whether there is enough evidence to reject this default assumption.

A p-value is the probability of observing results as extreme as the ones obtained, assuming the null hypothesis is true. A low p-value (typically $< 0.05$) suggests that the observed data is unlikely to have occurred by chance alone.

What does 'Internal Validity' refer to in a study's conclusion?

Internal validity is the degree to which a study can rule out alternative explanations for its findings. High internal validity means the researcher is confident that the independent variable, and not a confounding factor, caused the observed effect.

Why must a conclusion include a discussion of 'limitations'?

Discussing limitations provides context for the conclusion's reliability and helps other researchers understand the boundaries of the findings. It prevents the misapplication of results and guides future research to address the gaps.

Evaluating Results & Drawing Conclusions | OCR AS-Level Physics

Q: What does 'Internal Validity' refer to in a study's conclusion?

Internal validity is the degree to which a study can rule out alternative explanations for its findings. High internal validity means the researcher is confident that the independent variable, and not a confounding factor, caused the observed effect.

Q: Why must a conclusion include a discussion of 'limitations'?

Discussing limitations provides context for the conclusion's reliability and helps other researchers understand the boundaries of the findings. It prevents the misapplication of results and guides future research to address the gaps.

1. Definition & Core Concepts

Data Interpretation: This is the cognitive process of assigning meaning to collected information and determining how it relates to the research question. It requires moving beyond mere description of numbers to explaining what those numbers imply about the real world.
Hypothesis Assessment: At this stage, the researcher determines if the evidence is sufficient to support the alternative hypothesis or if it fails to reject the null hypothesis. A conclusion is not a 'proof' but rather a statement of the strength of evidence provided by the specific study.
Generalizability: This concept refers to the extent to which the findings from a specific sample can be applied to the broader population. High generalizability requires a representative sample and controlled variables to ensure the results aren't unique to a narrow set of circumstances.

A normal distribution curve showing the rejection region (alpha) where results are considered statistically significant.

2. Underlying Principles

Statistical Significance: This principle uses probability to determine if an observed effect is likely due to chance. A result is deemed significant if the p-value is lower than a pre-defined threshold (usually $\alpha = 0.05$ ), suggesting that the pattern is robust enough to be taken seriously.
Confidence Intervals: Rather than providing a single point estimate, confidence intervals provide a range of values within which the true population parameter is likely to fall. A narrow interval indicates high precision in the results, while a wide interval suggests significant uncertainty.
Effect Size: While significance tells you if an effect exists, effect size measures the magnitude of that effect. A result can be statistically significant but practically meaningless if the actual difference it describes is too small to matter in a real-world context.

3. Methods & Techniques

Step-by-Step Evaluation

Step 1: Verify Data Integrity: Before drawing conclusions, check for outliers, missing data, or measurement errors that could skew results. Ensuring the data is 'clean' is a prerequisite for any valid logical inference.
Step 2: Compare Results to Predictions: Align the findings with the original hypothesis. If the data contradicts the hypothesis, the researcher must evaluate whether the hypothesis was flawed or if the experimental design failed to capture the intended phenomenon.
Step 3: Identify Confounding Variables: Analyze whether external factors (variables not accounted for in the study) could have influenced the outcome. This step is vital for establishing internal validity and ensuring the conclusion is logically sound.
Step 4: Formulate the Conclusion: State the final takeaway clearly, specifying the conditions under which it holds true. A strong conclusion acknowledges the limitations of the study and suggests areas where further evidence is needed.

4. Key Distinctions

It is essential to distinguish between mathematical findings and logical implications to avoid overreaching in conclusions.

Feature	Statistical Significance	Practical Significance
Primary Question	Is the effect real or due to chance?	Does the effect matter in practice?
Determining Factor	P-value and sample size	Effect size and cost-benefit
Example	A $0.1\%$ increase in yield ( $p < 0.01$ )	A $20\%$ increase in yield

Correlation vs. Causation: A common error is assuming that because two variables move together (correlation), one causes the other (causation). Causation can only be inferred through controlled experiments that isolate the independent variable.

5. Exam Strategy & Tips

Check the Scope: When evaluating a conclusion on an exam, always check if the conclusion stays within the bounds of the data provided. If a study was done on plants, a conclusion about animals is likely an overgeneralization error.
Look for 'Absolute' Language: Be wary of conclusions that use words like 'always,' 'never,' or 'proves.' Scientific conclusions are typically probabilistic and use cautious language like 'suggests,' 'indicates,' or 'is consistent with.'
Sample Size Sensitivity: Remember that very large samples can make tiny, unimportant differences appear statistically significant. Always look for the effect size or the context of the numbers to determine if a conclusion is practically relevant.
Verify the Null: Ensure you understand what the null hypothesis ( $H_0$ ) represents—usually the status quo or 'no effect.' Rejecting $H_0$ is the goal when looking for a significant discovery.

6. Common Pitfalls & Misconceptions

Confirmation Bias: This occurs when a researcher overemphasizes data that supports their hypothesis while ignoring data that contradicts it. To avoid this, one must actively look for 'disconfirming evidence' during the evaluation phase.
Ignoring Limitations: No study is perfect. Failing to mention small sample sizes, non-random sampling, or environmental constraints makes a conclusion appear stronger than it actually is, which is a form of scientific dishonesty.
Post-hoc Fallacy: This is the mistake of assuming that because Event B happened after Event A, Event A must have caused Event B. Logical conclusions require a mechanism of action, not just a chronological sequence.

Evaluating Results & Drawing Conclusions

Summary

1. Definition & Core Concepts

2. Underlying Principles

3. Methods & Techniques

4. Key Distinctions

5. Exam Strategy & Tips

6. Common Pitfalls & Misconceptions

Evaluating Results & Drawing Conclusions

Summary

1. Definition & Core Concepts

2. Underlying Principles

3. Methods & Techniques

Step-by-Step Evaluation

4. Key Distinctions

5. Exam Strategy & Tips

6. Common Pitfalls & Misconceptions

Step-by-Step Evaluation