Scientific validity relies on rigorous testing, evidence-based evaluation, and the peer review of ideas. Science thrives on continual testing, where hypotheses are examined through experimentation and replaced or refined when evidence contradicts them. The peer-review process serves as a cornerstone of scientific validation, requiring ideas to withstand evaluation by experts and replication by others in the field.
Ensuring students have an understanding of what makes science valid allows them to evaluate ideas and concepts during their schooling and beyond. There are two main aspects to experimental validity. Internal validity examines whether the original question will be answered through the experimental procedure. External validity examines if the results of the experiment can be generalised or extrapolated beyond the procedure.
| Validity checklist | Ways to improve validity |
|---|---|
Question Does the experiment answer the question? (for example, measuring how high someone can jump does not tell you how fit they are) | Consider your dependent and independent variable. Are they directly related to your question? If the answer is no, then redesign your experiment. |
Hypothesis Is the hypothesis constructed correctly? | Does the hypothesis allow for predictions to be made? Can these predictions be tested? |
Repeatable Will you get the same results if you repeat the experiment? Does the method control all the possible variables? | Describe all the equipment and concentrations of solutions that you are using. Identify all the variables that need to be controlled. Consider the physical environment (light, temperature, movement of air etc.) in which you are doing your experiment. Were the systematic and random errors minimised or eliminated? Were measurements repeated? |
Assumptions Were there any assumptions made? Was bias shown in the way samples were selected? Is there another factor that may cause a change in the dependent variable? Does the experiment examine causation or correlation? | Consider the variables that could affect the outcome of the experiment. Have they been controlled? Describe why you are making the measurements/observations. List any assumptions that were made (for example, did writing on the outside of a petri dish affect bacterial growth inside the dish?). Justify why each of the assumptions that were made are reasonable. Were the samples selected randomly? Do they represent the wider community? Is there any other possible explanation for the change in the dependent variable? Were all the variables controlled? Is there more than one form of evidence suggesting that the independent variable directly affects the dependent variable? Outline how all the evidence supports your hypothesis. |
Sample size Is there a large sample size? Will a random result have an impact on your calculations? | If a sample size is too small, a single unit that is different can have a big impact. The number of samples (or repeats of an experiment) must be large enough to indicate reliability and contribute to a valid outcome. |
Accuracy Were the measurement instruments appropriate and calibrated? Are the results close to the expected value? Do the tables or graphs accurately represent the results? Are the theories or laws used valid (are any equations used appropriately for the method used)? | Consider using measurement instruments that are sensitive enough to measure small variations. For example, a beaker should not be used to measure the volume of liquids. Ensure that the equipment has been zeroed or tested for accuracy before the experiment. Check the axis of the graph to make sure that small changes are not misrepresented. Data tables should include all the results and show the units of the numerical data. Check the data supports current laws of theories. These are supported by large amounts of data in the past. A single experiment that produces different results is considered not valid until it has a similar amount of evidence as the law or theory. |
Precision Were the results similar to each other? Were there any outliers? Were the outliers explained? | Identify if there are any outliers or unexpected results. Is the sample size large enough to identify the outliers? Do not discard these. Instead explain each outlier. |
Limitations What are the limitations of your result? What are the maximum or minimum values of your independent variable that you did not test? Will a value outside this range give you a different outcome? | You will not have tested every possible variation of the experiment. Predicting other results by extrapolating a graph is not good science. Identify the tested upper and lower limits (i.e. time, concentrations, species). These are the limitations of your experiment. |
This process can be challenging for students to understand at junior levels.
Common alternative conceptions include:
| Alternative conception | Accepted conception |
|---|---|
| Experiments are valid if they are conducted in a lab. | Experiments can take place in various environments, such as in the field, through surveys, or via simulations, as long as they follow scientific principles. |
| A valid experiment must confirm the hypothesis. | Valid experiments aim to test hypotheses, not necessarily confirm them. Disproving a hypothesis is as valuable as confirming it, as both outcomes contribute to scientific understanding. |
| A large sample size guarantees validity. | While larger sample sizes improve reliability, other factors such as proper sampling methods, controls, and elimination of bias, are equally crucial. |
| If two variables are related, one causes the other. | Correlation does not imply causation. Experiments must isolate variables and control conditions to establish causal relationships. |
| One well-conducted experiment is enough to establish a scientific fact. | Scientific conclusions are based on repeated experiments and independent validation to ensure reliability and reproducibility. |
| Only sophisticated experiments with advanced equipment are valid. | Simpler experiments can also yield valid results, provided they are well-designed and adhere to scientific principles. |
| Experiments are valid as long as they produce results, regardless of bias. | Bias, whether in sampling, data collection, or analysis, undermines the validity of an experiment. Rigorous controls are essential to minimise bias. |
| If an experiment is published in a peer-reviewed journal, it is flawless. | Peer review ensures scrutiny but does not eliminate all errors or biases. Continued validation by the scientific community is necessary. |
| Reliable data must be valid. | An instrument that is not calibrated will provide reliable precise measurements that are not accurate or valid. |
Discuss with your colleagues
When do you introduce the concept of validity to students?
How do you describe validity to your students? Is there a consistent approach across the department or school?
What are the commonalities of evaluating valid evidence across the school?
References
Evaluating Scientific Data. (2023). NSW.gov.au.
<https://education.nsw.gov.au/content/dam/main-education/teaching-and-learning/curriculum/key-learning-areas/science/media/documents/evaluating-scientific-data-S4-to-s6.docx >
Here's where you'll find:
- Deep dives into high impact teaching practices.
- Discussion questions for you to explore with your colleagues.
- Links to related lessons in our teaching sequences.