Education: The popgun fallacy

The dull metal finish, trigger guard, revolver action, metallic barrel and slight menace make it look like a revolver at a glance… but on close inspection it’s only a pop-gun.

An RCT with random allocation to treatment, blinding, good follow-up, a long list of authors, a highly accessed publication (with a complex methods section) in a well regarded journal may look like a serious piece of research… but without sufficient statistical power it lacks the ability to make any useful conclusions and is in fact a pop-gun study.

One of the few things you be sure of from a pop-gun study is false negative (type II) error.

Pop-gun studies are doomed from the start. By wasting scarce resources and patient altruism in attempting to find treatment effects they in fact have no chance of finding, pop-gun studies can be considered unethical.

Suggested reading:
underpowered comparisons are futile