critically appraising the evidence

On completion of the search all items retrieved have to be reviewed and the appropriate material identified.

Of all the items retrieved, the Cochrane Systematic Review leads us to the current best evidence for this question.

Reference
Siegfried N, Uthman OA, Rutherford GW. Optimal time for initiation of antiretroviral therapy in asymptomatic, HIV-infected, treatment-naive adults. Cochrane Database of Systematic Reviews 2010, Issue 3. Art. No.: CD008272. DOI: 10.1002/14651858.CD008272.pub2.

The validity of the evidence is determined by critically appraising it.  This is done by asking the following questions:

1. Is the evidence valid; i.e. how close is it to the truth?
2. Is this evidence important, i.e. can it be potentially useful to clinicians?

To illustrate the process of critical appraisal we will use “treatment or therapy” as an example. It has to be remembered that the process can safely be applied to any of the Topics listed in Table 2.

The validity of the evidence  is determined by  critically appraising it.  This is done by asking the following  questions:

1. Is the evidence valid; i.e. How close is it to the truth?
2. Is this evidence important, i.e. Can it be potentially useful to clinicians?

There is no rule as to the sequence in which the above has to be answered. Many clinicians will try the 2nd question  first. If their answer to it is negative they will immediately discard that study and go on to the next one.

The validity of the evidence is determined by critically appraising it.  This is done by asking the following questions:

1. Is the evidence valid; i.e. how close is it to the truth?
2. Is this evidence important, i.e. can it be potentially useful to clinicians?

To illustrate the process of critical appraisal we will use “treatment or therapy” as an example. It has to be remembered that the process can safely be applied to any of the Topics listed in Table 2.

The validity of treatment can be determined by asking some simple questions. The questions can be for individual therapeutic trials; however as these can be best interpreted in the context of other trials on the same topic, we will focus on such a systematic review. A systematic review combines the results of several trials into a single, concise document.

The questions to check the validity of a systematic review will be:

1. Is it an overview of randomised trials of the treatment you are interested in?
2. Does it include a methods section that describes:
1. finding and including all the relevant trials
2. assessing their individual validity
3. Were the results consistent from study to study?

A systematic review assesses the studies included in the review individually and this is done by asking the following questions about the individual studies:

1. Was the assignment of patients to treatments randomised and was the randomisation list concealed?
2. Were all patients who entered the study accounted for at the conclusion and were they analysed in the groups to which they were randomised?
3. Were patients and clinicians kept “blind” to which treatment was being received?
4. Aside from the experimental treatment, were the groups treated the same?
5. Were the groups similar at the start of the study?

Should the report being analysed fail to meet one or more of the criteria set out above, it need to be considered whether it has a fatal flaw that might render its conclusions invalid. If this happen it means,"Search on for more evidence!"

If on the other hand the report passes the validity check, you can move on to the applicability check

Deciding whether or not a report is important  andapplicable; the clinician must take into account both his own clinical expertise as well as the best external evidence.

Can the evidence be applied to clinical practice?
Once the evidence has been found, critically appraised and deemed valid as well as important. The next, and final step, is incorporating it into the care of patients.

Applying the evidence
The results of the study are discussed with Mr Dince.  It is agreed between the doctor and Mr Dince that there would be a benefit to starting therapy in the coming weeks. It can be commenced as soon as Mr Dince is ready and understands all that he needs to know about the medication including adherence, possible side effects, disclosure to family and partners and the need to continue life-long therapy.
 
Further Reading

1. Sackett DL, Richardson WS, Rosenberg W, Haynes RB. Evidence-based  Medicine - How to Practice and Teach EBM. Churchill Livingstone 1997
2. Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. BMJ 1996; 312:71-2
3. Covell 01-Apr-2010tice: are they bieng met? Ann Intern Med 1985; 103: 596-9
4. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomised control trials and recommendations of clinical experts. JAMA 1992; 268:240-8
5. Davidoff F; Haynes B, Sackett D, Smith R. Evidence based medicine: a new journal to help doctors identify the information they need. BMJ 1995; 310:1085-6
6. Ramsey PG, Carline JD, Inui TS et al. Changes over time in the knowledge base of practicing internists. JAMA 1991; 266:1103-7
7. Evans CE, Haynes RB, Birkett NJ et al. Does a mailed continuing education program improve clinician performance? Results of a randomised trial in antihypertensive care. JAMA 1986; 255:501-4
8. Davis DA, Thompson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995; 274:700-5
9. Sibley JC, Sackett DL, Neufeld V et al. A randomised trial of continuing medical education. N Engl J Med 1982; 306:511-5
10. Shin JH, Haynes RB, Johnston ME. Effect of problem-based, self-directed undergraduate education on life-long learning. Can Med Assoc J 1993; 148:969-76
11. Oxman A, Guyatt GH. The science of reviewing research. Ann NY Acad Sci 1993; 703:125-34
12. Dickersin K, Sherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ 1994; 309:1286-91
13. Fullerton-Smith I. How members of the Cochrane Collaboration prepare and maintain systematic reviews of the effects of health care. Evidence-Based Medicine 1995; 1:7-8
14. Davis DA, Thomson MA, Oxman AD, Haynes RB. Evidence for the effectivenes of CME. A review of 50 randomised controlled trials. JAMA 1992; 268:1111-7
15. Johnston ME, Langton KB, Haynes RB. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med 1994; 120:135-42
16. Systematic Reviews. Synthesis of Best Evidence for Health Care Decisions.  Edited: C Mulrow & D Cook. American College of Physicians, 1998