We aspire to subject our resources to the same rigorous evaluation that we expect of others in health research.
Just as we do not expect a new treatment to be promoted if it hasn’t been properly evaluated, so also we encourage evaluation of resources intended to help people become more discriminating users of health research.
Resources can be evaluated using a variety of approaches, both during development, and after the resource has been made publicly available. Approaches range from formal controlled experiments testing whether the resource has improved user comprehension, to noting informal comments on the clarity of texts.
By “user comprehension” we mean that the evaluation used some means to assess whether the users had learned and understood the material.
Possible approaches to evaluation might include:
Before launch of a resource:
- User-centred design approaches
- Comments on successive drafts of text from readers representative of the target users
- Tracking use of screens and click-outs
- Questionnaires to assess comprehension
- Interviews to assess comprehension
- Cognitive assessments
- Randomized comparisons of alternative resources, with comprehension tests
After launch of a resource:
- Sales or downloads of printed resources
- Formal adoption of the resource for teaching
- Listed as recommended reading/use
- (All) published reviews, including mentions in articles in the lay press
- Trends in numbers of unique visitors to and page downloads from a website hosting a resource
- Readers’/users’ comments
- Translations into languages in addition to the original language