Effect Modification
Enshrined in the way much medical research is done is the tacit assumption that an exposure has an effect on an outcome. To quote Wikipedia: “Effect modification occurs when the magnitude of the effect of the first exposure on the outcome—the association—differs depending on the level of a third variable. In this situation, computing the overall effect of the association is misleading.”
The reach of Effect Modification (EM) is wildly underappreciated. Implications challenge the core of how medicine—and indeed science—are practiced.
The results of randomized, double-blinded, placebo-controlled trials (RCTs) are the foundation on which modern medicine rests. RCTs are exalted as the gold standard of study designs. These define treatment approaches, which are propelled into use through “clinical practice guidelines,” with teeth in their implementation imposed through “performance pay” to doctors. But, there is a problem. Often, a single estimate of effect is generated for a given outcome, and presumed to apply generally (and perhaps, if favorable, to impel treatment for those outside the study).
But, results of studies—including but not exclusively RCTs—may not apply to those outside the study. They may not apply to some who are enrolled in the study. They may not, in fact, apply to anyone in the study.
The chief recognized problem inherent to RCTs is “generalizability” (sometimes called “external validity”). Results need not apply to types of people outside of the study (e.g. studies of men may not apply to women)—because of potential for EM. Less appreciated is that EM also means that results of a trial need not apply to all those within the study. To ascribe what is true for a whole to its parts is to succumb to the “fallacy of division.”
It is seldom appreciated that EM can engender differences not only in the magnitude—but in the sign of effect. When subsets experience opposite effects, a neutral finding for the overall study can apply to none of its participants.
Bidirectional EM effects—let’s call this “Janus Effects”—are not rare. Penicillin can save lives, but can cost life in the highly allergic. A surgery may save lives, but take lives of poor surgical candidates. Fluoroquinolone antibiotics can raise and lower blood sugar. Benzodiazepines anxiolytics can “paradoxically” increase anxiety. Bisphosphonates to prevent fractures, cause “pathological fractures.” Statins prevented new diabetes (WOSCOPS trial), and promoted it (JUPITER trial); reduced cancer deaths (JUPITER trial), but significantly increased new cancer (PROSPER trial, the sole trial in age >70).
How is this possible? One factor is that agents that can yield antioxidant effects (like statins), are almost always prooxidant in some patients and settings—including at high doses, where co-antioxidants are depleted. Conversely agents that can have prooxidant effects can, not uncommonly, have antioxidant effects in sufficiently low doses, for some people – via “oxidative preconditioning,” by which a bit of oxidative stress ramps up endogenous antioxidant defenses. For agents meant to alter an aspect of physiology, counterregulatory mechanisms—imposed by evolution—may partly offset the intended effects– and in some people overshoot. So drugs and salt restriction that are meant to lower blood pressure, paradoxically raise blood pressure in some.
Too, many exposures activate multiple mechanisms, which may act in opposition on an outcome. Imbibing alcohol can prevent stroke, via antioxidant polyphenols and thinning the blood; and can promote stroke, via mitochondrial dysfunction, arrhythmia, and hypertension (or by thinning the blood too much). Swilling coffee is linked to reduced heart attacks in genetically fast caffeine metabolizers (likely through antioxidant effects), but increased heart attacks in slow caffeine metabolizers (likely via caffeine-induced ~adrenergic ones).
Implications are rife. When studies of the same intervention produce different, or even opposite, results, this apparent “nonreproducibility” need not mean there were study flaws, as is often presumed: “Contradictory” results may all be true.
For Janus effects, whether exposure effects on an outcome are favorable, adverse, or neutral may depend on the composition of the study group. Evidence supports such bidirectional effects for statins with outcomes including diabetes, cancer, and aggression. Selection of a study group that yields sizeable “benefit” (or for environmental exposures, no harm) may drive a product to be recommended, or exposure mandated, for vast swaths of the populace—with potential for harm to many.
Individual experiences that controvert RCT results should not be scorned, even if a source of EM is not (yet) known. Those who observed their blood sugar rise on statins, were dismissed—given neutral average statin effects on glucose in RCTs, then disparaged more contemptuously after the WOSCOPS trial reported that statins reduced diabetes risk. Later, multiple other trials, and meta-analyses, showed statins can increase diabetes incidence. That was equally true, of course, before these later trials were published. (Now that it is accepted that statins can increase glucose, recognition that statins can reduce glucose has faded.)
So, conventional thinking about studies’ implications must be jettisoned. It would be convenient if the observed association in a good quality study could be thought the final word. But, EM may be more the rule than the exception—at least in complex domains like biology and medicine—whence “computing the overall effect” can be “misleading.” An effect cannot be presumed to reliably hew to what any study “shows” in magnitude, or even direction.
The pesky play of effect modification must be borne in mind.
