There is rumor of a sacred mountain in Tibet, the peak of which can be only ascended when Jupiter, Mercury and Venus are in triangular alignment. At the summit, there lives a man who will provide the truth for any one question a plucky adventurer may pose. One day, I hope to be that adventurer. My question…is CME an effective means for impacting clinician competence, performance and (daresay) patient health?
Unfortunately, the next anticipated triangular alignment isn’t until 2021. In the interim, I have to: 1) learn how to climb mountains and 2) go about establishing cause and effect the old-fashioned way.
To that end…If I want to argue that a relationship exists between CME and some effect (eg, competence gain), I must establish three things:
- Temporal precedence: the effect comes after the presumed cause. For example, CME participants score better on a case-based, post-activity assessment than pre-activity. Pretty straightforward, right? Who needs a mountain guru?
- Covariation: the effect is systematically (ie, not randomly) related to the presumed cause. For example, a high level of competence would be more likely among CME participants than non-participants and/or more CME participation would equal more competence than less CME participation. Wait…this sounds like a control group study. Didn’t we (ie, me in this conversation with myself) say control groups in CME are bunk? Okay, I exaggerated a skosh. Simple post-test only nonequivalent control group design (ie, surveys to participants and nonparticipants after a CME activity) is pretty much at the bottom of the research credibility scale, but there are more robust methods to employ control groups. I’ll cover these in a subsequent post.
- Plausible alternatives: once both temporal precedence and covariation are established, all other possible explanations for the effect (ie, confounders) must be explored. This addresses the internal validity of your assessment (ie, how well it avoids confounding). I’ll talk about some threats to internal validity in a subsequent post. Until then, note there is no perfect study: interval validity exists on a spectrum. The more internally valid (ie, the less confounded), the more confident you can be in your interpretation of cause and effect.
In absence of divine wisdom, every CME outcome assessment should speak to these three factors. I’d say we do a pretty good job establishing temporal precedence, but it’s a rare occasion to discuss covariation or confounders. Next time you find yourself creating or reviewing an outcome report, take that opportunity to push us all forward a bit on these critical factors to establishing the value of CME.