Scientific conclusions rarely depend on a single observation. They emerge from a chain of evidence, assumptions, analytical choices, and inference. A result can therefore appear convincing while still depending heavily on one or two conditions that carry much of the interpretive weight.
A useful way to evaluate a conclusion is to reverse the usual question. Instead of asking only what evidence supports an interpretation, ask: what would have to be different for the interpretation no longer to hold?
This question is not an invitation to manufacture doubt. It is a way of identifying the structure of an argument.
Some assumptions are relatively inconsequential. Changing them would alter a numerical estimate or the precision of an analysis without materially changing the scientific interpretation. Others are load-bearing: the conclusion depends on them being sufficiently valid. The distinction matters because an argument supported by many observations can still be fragile if those observations ultimately depend on the same critical assumption.
Consider experimental independence. A dataset may contain a large number of measurements, but if those measurements originate from only a few biologically independent sources, treating every observation as an independent replicate can substantially change the apparent strength of the evidence. The important sensitivity test is therefore not simply whether the dataset is large, but whether the conclusion persists when the correct experimental unit is used.
The same logic applies to mechanistic interpretation. A treatment may alter the expression of a receptor, protein, or pathway marker while also changing the phenotype of interest. That combination strengthens an association, but a causal interpretation still depends on an additional assumption: that the altered component is functionally responsible for the observed effect. If blocking or removing that component leaves the phenotype essentially unchanged, the original mechanistic conclusion requires revision even though the expression data themselves remain valid.
Analytical choices can also be load-bearing. Alternative model specifications, plausible definitions of an outcome, treatment of repeated measurements, adjustment for relevant confounders, or correction for multiple comparisons may leave a conclusion largely unchanged—or expose that it depended strongly on one analytical pathway. Sensitivity to such choices does not automatically invalidate a result. It tells us how conditional the inference is.
This is why robustness should not be reduced to obtaining the same P value repeatedly. A conclusion can remain scientifically coherent even when an effect estimate changes in magnitude or crosses an arbitrary significance threshold. Conversely, apparently consistent statistical significance does not guarantee that the underlying biological interpretation is robust. The relevant question is whether reasonable changes to assumptions, models, controls, or experimental conditions alter the substantive claim being made.
For mechanistic claims, the most informative challenges are often experimental rather than statistical. Does the proposed mechanism survive targeted perturbation? Is the effect lost when the proposed mediator is disrupted? Can it be restored through rescue? Can competing explanations account for the same observations? Evidence that discriminates among alternative explanations carries more inferential value than simply accumulating additional measurements compatible with the preferred one.
Importantly, asking what would change a conclusion does not mean that every scientific claim must be vulnerable to a single decisive experiment. Biological systems are complex, measurements are imperfect, and evidence often accumulates across different models and levels of organization. Conclusions may therefore be revised gradually rather than overturned by one observation.
But a scientific interpretation should still expose the conditions under which confidence in it would decrease.
If no plausible observation, perturbation, reanalysis, or contradictory evidence could ever modify a conclusion, the problem is no longer simply the amount of evidence available. The claim has become insulated from empirical challenge.
A strong conclusion is therefore not one that appears impossible to question. It is one whose evidential dependencies are visible: we know what supports it, which assumptions matter most, what alternative explanations remain, and what future evidence would strengthen, weaken, or revise it.
Before asking how strongly a dataset supports a conclusion, it is worth asking a more revealing question: what would change the conclusion?