Appraising research evidence

A woman and man sat at a table discussing some papers in front of them

Once you’ve found your evidence you need to critically assess it to ensure it’s a good enough quality to use.

Evidence can be assessed in different ways. Not all evidence is equal, so you must appraise the evidence you find.

If you’re confident that your evidence is of a high enough quality then you’re more likely to make good decisions as a result of using it.

The reliability of conclusions we draw from research evidence can be influenced by how we collect data. Where possible you should go back to the original source. 

 

The Open University have devised an initial ‘PROMPT’ test for looking at results and online information.

  • Presentation - is the information clearly communicated?
  • Relevance - does the information answer your question?
  • Objectivity - is the author’s position made explicit?
  • Methods - is it clear how data was collected?
  • Provenance - is it clear where information came from?
  • Timeliness - is it clear when the information was produced?

If you are confident that your sources of evidence pass this initial test and are credible, then you need to critically appraise the evidence you find. 

All methods of evidence collection have their own merits and limitations. To assess research evidence you need to judge whether the conclusions reached in the research reports (and evidence) are justified by the data collected.

You should focus on three specific areas.

  • Reliability - the extent to which the results are consistent. Would the same results be produced if the study was replicated?
  • Validity - the extent to which the data collected measures what it claims to measure. Does the study answer the question it set out to answer?
  • Balance - the extent to which the claims in the evidence you’re assessing are backed up by the original source. Scan the abstract to see if the arguments made in your source give an accurate and balanced portrayal.

The Critical Appraisal Skills Programme (CASP) has produced checklists as prompts to help you assess evidence critically.

There are also various online resources that can help you assess the quality of the evidence you have found. One example is the Veto Violence Continuum of Evidence of Effectiveness.

Loading Icon

Please wait... logging you in.