Education: The treatment effect

Investigator: How’s your wife?
Statistician:
 Compared to what? 
(Senn, S) 

Statisticians are obsessed with making comparisons:

Example: Dinner party conversation

A guest at a dinner party proudly proclaims to have made $100K from an initial investment of $100K in 10 years by investing in real estate. The host however notes that his bank offered 5% per annum over this period (a low risk investment, $100K at 5% compounded  for 10 years ≈ $163K); by comparing the low risk option (the control) to the real estate option the returns look a lot more modest. A more accurate measurement of the guest’s investment prowess (the treatment effect) is +$37K over ten years.

Example: Clinical trial

In the clinical trial the exact same principle applies, the trialist must stay strictly focused on the between group differences, that is the treatment effect. Reporting the absolute change in both arms at the expense of the difference is misguided, primary emphasis should always be on the treatment effect.

Plotting the absolute scores can obscure the treatment effect (left), we recommend investigators plot the treatment effect (with 95% confidence intervals) rather than the absolute scores (or both perhaps plot both, right).

Suggested reading: