Why primary, secondary and subgroup trial results carry different weight

The result needs its original question

A trial can report dozens of measurements without asking dozens of equally important questions. The primary outcome is the prespecified measurement intended to answer the main question. Its full definition includes what was measured, how it was summarized and when. Secondary outcomes address additional questions; exploratory analyses can reveal patterns worth testing later. Those labels map the trial’s logic. They are not scores for whether a result is true.

That map matters because many comparisons create many opportunities for at least one chance-positive result. It also matters because the groups protected by randomization can change when an analysis selects participants after assignment. A careful reading therefore keeps outcome hierarchy, analysis population, effect size, uncertainty, practical importance and replication separate.

Start with what was fixed before results were known

Prespecification means the outcome, time point and analysis were fixed before the relevant results were known. CONSORT asks reports to define primary and secondary outcomes completely and explain changes from the protocol or registry. Prespecification improves interpretability, but it does not erase missing data, weak measurement, implementation problems or limited applicability.

The primary outcome usually carries the trial’s main inferential burden. A secondary outcome can still be important, but it does not automatically inherit the primary outcome’s status. A “key secondary” result supports a formal conclusion only when it sits inside the prespecified testing strategy—for example, an ordered hierarchy that protects later tests from being interpreted after an earlier gate fails. The label alone is not enough.

Exploratory does not mean false. It means the analysis usually generates or refines a hypothesis instead of carrying the same formal weight as a protected primary or key-secondary test. The constructive response is to ask what future study would test that pattern directly.

Count the opportunities for a striking result

Every additional endpoint, time point, subgroup or outcome definition creates another opportunity for a result to look unusual by chance. A prespecified multiplicity plan defines which comparisons belong to one inferential family and how they will be handled. There is no one correction that fits every trial, and no universal threshold turns uncertainty into certainty.

The number analyzed can also change by outcome. CONSORT asks reports to show the available-data denominator, group result, effect estimate and precision for each primary and secondary outcome. A headline randomized sample cannot silently replace the smaller set that contributed a particular measurement.

A subgroup needs a comparison between groups

A statistically significant result in one subgroup and a non-significant result in another does not establish that the treatment effect differs between them. The relevant question is a direct interaction or effect-modification comparison, reported with its uncertainty. A prespecified subgroup is generally easier to interpret than an undisclosed post hoc split, but prespecification does not remove low power or multiple-testing concerns.

This is why “worked in women but not men,” “worked in older but not younger participants,” or similar statements need more than two separate p-values. Look for the interaction estimate, the planned subgroup list, the number of participants and independent replication.

Randomization protects the comparison that was randomized

An intention-to-treat analysis asks about assignment to a strategy and preserves the groups created by randomization when all randomized participants are analyzed as assigned. A per-protocol or as-treated analysis asks a different question. A naive restriction to adherent participants can compare people who differ after randomization because adherence and later inclusion may relate to prognosis.

That does not make every per-protocol analysis invalid. It means the report must identify the question, estimator and assumptions. Likewise, “not statistically significant” in an ordinary superiority test does not prove that two strategies are identical, that the effect is zero or that a clinically important difference is impossible. Equivalence and non-inferiority require their own design, margin and analysis.

Example one: secondary signals do not replace null primary outcomes

In the pectin pilot, 31 community-dwelling adults older than 65 with frailty or pre-frailty were randomized for four weeks; 29 completed. The paper-defined primary outcomes were TNF-α, IFN-γ and zonulin, and the report found no significant between-group change in them. Its sample-size calculation used TNF-α evidence from a younger healthy-volunteer pilot; the study was not powered for each functional, psychological, quality-of-life or microbiome comparison.

The registry and paper also do not expose one stable outcome hierarchy: registered and published roles differ, the sequencing method changed, short-chain fatty acids were not quantified, and the first registry version was submitted after enrollment began. Selected sit-to-stand, HADS and SF-36 signals therefore cannot overwrite the paper-defined primary outcomes or establish broad clinical benefit.

One participant withdrew because of diarrhea and one without a stated reason; no serious adverse event was reported. Four weeks and 31 randomized participants cannot establish uncommon or durable safety. The report names NIHR Nottingham Biomedical Research Centre support and consulting, investigator-funding and advisory-board relationships. Those disclosures are context, not proof that a result is true or false.

Example two: identify which randomization protects which contrast

ACTIVE randomized 2,832 independent U.S. adults age 65–94 to memory, reasoning, speed-of-processing training or no-contact control. Booster sessions were offered to people who completed at least eight of ten initial sessions, and booster assignment within that eligible subset was randomized. A later claims analysis included 2,021 participants enrolled in traditional Medicare at baseline and also separated people by booster exposure or completion. The safe question is not “randomized or nonrandomized?” in the abstract, but which comparison each randomization protects.

The claims paper reported a hazard ratio of 0.75 (95% CI 0.59–0.95) for speed trainees who completed at least one booster and 1.01 (95% CI 0.81–1.27) for speed trainees without boosters; randomized-arm comparisons for memory and reasoning were null. Those numbers do not establish that generic brain games prevent dementia. They show why the analysis set, conditioning steps and exact contrast must travel with the result.

A published critique argues that the reported hazard ratios require multiplicity adjustment. The authors respond that the distinct training arms answered prespecified, mechanistically different questions and should not be pooled into one family. Neither paper makes the disagreement disappear. The claims analysis reports NIA and original ACTIVE NIH support; Karlene Ball disclosed consulting for, stock ownership in and a scientific-advisory-board role with Posit Science. The studied protocols cannot be replaced with a generic commercial-product claim.

A seven-question reading check

1. What outcome, metric and time point were designated primary before results were known?
2. Which secondary outcomes were inside an ordered or adjusted testing plan?
3. Which analyses were exploratory or post hoc?
4. Was a subgroup difference tested directly with an interaction?
5. Who was randomized, who was analyzed and what happened after assignment?
6. What do the effect estimate and uncertainty allow—and not allow—a reader to conclude?
7. Has an independent study tested the same claim in a relevant population?

These questions do not rank studies or make a treatment decision. They provide one reusable method for keeping an attractive result from answering a broader question than the trial actually tested.