Reliability & Validity

There are very few validated instruments in CME assessment.  I’ve only encountered three, and each of these were designed to measure satisfaction.  As to why there are so few validated instruments…it’s pretty simple: 1) validating an instrument is difficult and 2) grant funding in CME research is scarce (forcing the necessary research talent to focus elsewhere).

Of course, an enterprising investigator might see this gap as an opportunity to build a research career in an area with very low competition.  For that person, I’d recommend starting with the following articles:

  1. Downing SM. Face validity of assessments: faith-based interpretation or evidence-based science. Medical Education 2006;40:7-8. (PubMed)
  2. Downing SM. Validity: on the meaningful interpretation of assessment data. Medical Education 2003;37:830-7. (abstract)
  3. Downing SM, Reliability: on the reproducibility of assessment data. Medical Education 2004;38:1006-12. (abstract)
Advertisements

3 Comments

Filed under CME, Methodology, Reliability, Validity

3 responses to “Reliability & Validity

  1. Pingback: Survey design « assessCME

  2. I agree that validating a survey is complicated but it can be done quite effectively and efficiently once the process is well-defined. The four parts of validating your survey include: face validity, content validity, construct
    validity and criterion-related validity. Face validity is pretty straightforward as it mostly relates to clarity of the questions and level of organization. Content validity determines if the questions relate to the topic and is typically achieved by critical review from experts in the field. A survey with criterion-related validity is directly comparable with measures of the same behavior; for example, certain measures should correlate with content knowledge.
    The most challenging, in my experience, is the construct validity but the references that you mention can help with that stage of validation. I think validation is often over-looked in the process of developing instruments for assessment in CME. I am happy to discuss the process in more detail as our newest survey instrument is currently undergoing the process! Great topic Jason (once again)…

  3. assesscme

    Thanks, Bonny. And it be great to hear more about the survey you’re currently validating. Any chance it will be available to the CME public?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s