The Open University have devised an initial ‘PROMPT’ test for looking at results and online information.
- Presentation - is the information clearly communicated?
- Relevance - does the information answer your question?
- Objectivity - is the author’s position made explicit?
- Methods - is it clear how data was collected?
- Provenance - is it clear where information came from?
- Timeliness - is it clear when the information was produced?
If you're confident that your sources of evidence pass this initial test and are credible, then you need to critically appraise the evidence you find.
All methods of evidence collection have their own merits and limitations. To assess research evidence you need to judge whether the conclusions reached in the research reports (and evidence) are justified by the data collected.
You should focus on three specific areas.
- Reliability - the extent to which the results are consistent. Would the same results be produced if the study was replicated?
- Validity - the extent to which the data collected measures what it claims to measure. Does the study answer the question it set out to answer?
- Balance - the extent to which the claims in the evidence you’re assessing are backed up by the original source. Scan the abstract to see if the arguments made in your source give an accurate and balanced portrayal.
The Critical Appraisal Skills Programme (CASP) has produced checklists as prompts to help you assess evidence critically.
There are also various online resources that can help you assess the quality of the evidence you have found. One example is the Veto Violence Continuum of Evidence of Effectiveness.