I love SurveyMonkey…survey creation, distribution and data collection is a snap with this service (and it’s super cheap). What could possibly be bad about making surveys so accessible to everyone? Oh, yeah…it’s probably making surveys so accessible to everyone. Surveys used to represent a significant time and financial investment (e.g., postage, envelop stuffing, data entry). Now all you need is a list of emails. Without previous barriers, the decision to survey can come a little too quickly.
Admittedly, I’ve done
more than one survey too many surveys simply because it was easy…rather than necessary. Now I’m afraid that all this ease is actually making surveying harder than ever. There are only so many physicians, and if we’re all bombing their inboxes with survey invitations, what’s the difference between us and cheap Viagra spam?
In his recent JCEHP Editorial, Dr. Olson eloquently describes this concern:
“…a survey population is a commons, a resource that is shared by a community, and like other commons such as ocean fisheries or antibiotics, it can be degraded by overuse” (p. 94)
Dr. Olson goes on to detail five ways in which we most typically misuse this common resource – which are much easier to address than climate change. I highly recommend reading this editorial. Afterward, continue to “reduce, reuse, recycle” and add: resist.
How do you cook CME? Maybe simmer KOL in a venue sauce and add enduring material to taste? And how do you select your ingredients? Are you a student of food theory or do you just feel your way through?
Well, I’m supposed to be scientifically-minded, so my pantry is full of evidence-based options. Wait…did I say full? I meant I know these four things:
- Live activities are more savory than print
- You’ll make a better soup with multi-media
- Multiple tastes are preferred to just one
- Case-based discussions are the most important seasoning
According to Marinopolous SS, et al. that’s all we’ve got to work with. When you don’t know who’s coming to dinner, how hungry they are, or any of their possible dietary restrictions, you’ve got to make CME magic using only these four things. That’s pretty bleak.
Why don’t we know more? Too few studies with no standardization and very little reliability or validity data to support findings. Us outcome experts may all be wearing toques, but apparently only make french fries.
Commitment to change (CTC) questions are the caboose of every post-activity CME evaluation – stripped of all relevancy and sustained solely by nostalgia. Thirty years since its introduction, we can now all retire this method, confident that it has served us well, but that it’s now time for something more…app-ish. And off it goes, grumbling it’s final words toward obscurity: “…but, you never really knew me”.
Before you dismiss CTC, check out this article. People have been studying CTC for a long time. And there’s value to this approach – assuming you use it correctly. Should you use a follow-up survey? When? How? How should you word the questions? Include a rating scale? And how should you sort through and interpret the results? This stuff all matters. And you won’t find an easier to digest summary than this 2010 article in Evaluation & the Health Professions.
So, yes, if you’re simply maintaining a “what are you going to change in your practice” question at the end of every CME evaluation – definitely send that packing. Then read the aforementioned article. You’ll find that CTC has limitations, but when done in accordance with the latest evidence, there’s a lot of good data to be had.