The Level of Evidence Used to Answer General Practitioners’ Questions

Session 14: Karen Davies

Karen Davies presented her fascinating research on what level of evidence is most often used to answer general practitioners’ questions, done in the UK.

She started with a couple of thoughts/definitions: first was Venn diagram from Haynes (1996) that showed evidence-based practice as the over-lap of research evidence, patient preferences, and clinical expertise (which, she clarified, does not go away in EBP).  She also raised the question: if you can’t get level 1 or level 2 evidence, is it worth using evidence at all (case studies, e.g.)?  Her response–yes.  She also raised a point I found intriguing: perhaps we should question meta-analysis’s position at the top level, since they’re often inconsistent and even the statisticians can’t agree.  I’m not sure I entirely agree with her, but I definitely think it’s a good point to take into consideration with meta-analysis studies.

She had 2 main research objectives: to determine the highest level of evidence used in answering questions, and also to determine the number of times guidelines are used.  To answer them, she looked at 2 primary care answering services in the UK: ATTRACT and the NLH Primar Care Answering Service.  Their answers are publicly posted online, so it was simply a matter of tallying everything up.

Some results:

  • On average (for both services), only 11% of the answers used top-level evidence.
  • The two services had very different numbers of questions not answered…the sidenote being that there will always be questions for which there is no evidence.  Yet.
  • 42% of the answers referred to guidelines.
  • Very few of the questions were answered using level 2 evidence (clinical trials)

As of now, the NLH service has closed.  Which perhaps underscores Karen’s conclusion that since guidelines are so important, doctors need to know where to find them!

An audience member wanted to know whether there is a difference between UK and US guidelines; Karen’s answer was that she believe the UK is a bit stricter about having obvious “expiration” dates on their guidelines, but she noted that the answering services used US guidelines in many of their replies.

I found this to be a fascinating presentation, and I was very pleased to see that Karen’s slides are posted (.pptx).  Check them out for more results and pretty graphs!

Leave a Reply