It happens. Your carefully crafted evaluation questions are administered to the survey population using a different scale pre- and post-activity. Miscommunication, cut & paste fail, whatever the cause…what do you do with the data?
- Nothing. You report it as is, don’t attempt any statistical testing, and hope it doesn’t happen again.
- Transform. Call on your inner MacGyver and make these two scales compatible.
Tempting as option #1 may be, this blog wouldn’t be much use if we take that route. So here are the simplest fixes:
- Proportional transformation: if you want to make a 5-point scale talk to a 7-point scale, you multiple each 5-point score by 7/5 (alternatively, you could reduce a 7-point scale to 5-point by multiplying each 7-point score by 5/7).
- Transform each score (e.g., all 5-point and 7-point scores) to a standard z-score using the following formula: z = (raw score – mean of raw scores)/standard deviation of raw scores.
In this case, simple may also be right (or right enough). To see how these approaches compare to more complex transformations, check out this article.
Graduate school professors and academic mentors all soberly agreed that whatever manuscript I was currently stressing over would ultimately be read by three peer reviewers and then no one else. I could accept that my family and friends wouldn’t be interested in The effects of physician specialty and patient comorbidities on the use and discontinuation of coxibs, but surely it would fill a hole in some greater literature stream and assist in the improvement of…something?
Not bloody likely.
In yet another example of I-published-that-15-years-ago-and-nobody-read-it-and-they’ve-been-doing-it-wrong-since, Roberson et al very clearly explain how we’ve been inappropriately using the t-test to analyze Likert scale data (link). Well, I’m sorry it’s taken so long, but I promise to use the “sign test” from here forward.
I’ve been told it’s impossible to sneeze with your eyes open. I’m still trying to disprove that one, but I have accepted the impossiblity of creating a CME survey without a Likert scale question.
Before you create your next Likert scale question, check out this post. How big should your scale be? How should it be labeled? Turns out there’s a science behind these questions.