Calculating effect size for your CME activities

Imagine you’ve created a survey to assess changes in competence among participants in a CME activity (pre- vs. post-participation).  Let’s say there’s a total of eight questions in your survey (each one asking participant to self-report their utilization of a specific clinical task related to the CME activity content on a 5-point scale).  One week after the CME activity is completed, you’re staring at an Excel spreadsheet containing all of the pre- and post-activity responses.  Now what?  How do you use this data to determine whether your activity was effective?  And how do you compare this data to the results of other outcome assessments?

If this was an infomercial, I’d now break into scene (colorfully sweatered) and introduce the latest advancement in outcome measurement: Cohen’s dIt slices, it dices…it reduces all of your outcomes data into a single metric that summarizes the overall effectiveness of your CME.  But wait, there’s more…you can use Cohen’s d to compare the effectiveness of your CME across activities.  That’s right.  Want to know how effective this year’s conference was relative to last year?  Cohen’s d.  Want to know how effective your live CME activity was relative to its repurposed enduring material?  Cohen’s d.  Want to know the effectiveness of your overall CME program by topic?  Or by format?  That’s right, Cohen’s d.

Although this may sound like an innovation, Cohen’s d has been around for decades.  It’s even been used in CME; however, this has largely been restricted to academic publications.  But the claims made above are true.  Cohen’s d can answer the following questions: 1) Was my CME effective? 2) How effective was it? And 3) how effective was my CME relative to other CME activities?  Better still, it’s remarkably simple to calculate – if you can calculate a mean and a standard deviation (not by hand, of course, use Excel), then you can calculate a Cohen’s d.

I’m going to dedicate the next two blog posts to Cohen’s d.  The first will provide step-by-step instructions of how to calculate it and interpret the results.  The next will demonstrate how you can use Cohen’s d to assess the effectiveness of your CME beyond the individual activity (i.e., by topic, or format, or year).  And I promise, you’ll be amazed at how easy this is to do.


Filed under CME, Cohen's d, Effect size, Effectiveness

5 responses to “Calculating effect size for your CME activities

  1. Pingback: So what’s a good effect size for CME? | assessCME

  2. Pingback: Outcomes Training | assessCME

  3. My brother suggested I might like this web site.

    He was totally right. This post actually made my day.

    You cann’t imagine just how much time I had spent for this
    information! Thanks!

  4. Pingback: Effect size kryptonite | assessCME

  5. healthscout

    So this is a terrific blog and why are so few people visiting and commenting? Jason you’ve got some terrific content here, easy to read, accessible, and a deep archive with lots of shelf life. Most important: you are serving the public by serving members of the CME community, truly worthy of your MPH degree. Where’s the page view counter at the top so we can measure visitor traffic? And where are the Google Adsense ads that should be in the lefthand column so you can monetize this ???

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s