Global Health

We don’t know whether most medical treatments work, and we know even less about whether they cause harm – new study

Patients with ailments for which there are no effective medical treatments may be willing to try treatments that are not yet even supported by low-quality evidence.

Patients with ailments for which there are no effective medical treatments may be willing to try treatments that are not yet even supported by low-quality evidence. Image: Unsplash/Volodymyr Hryshchenko

Jeremy Howick
Senior Researcher, University of Oxford
Share:
Our Impact
What's the World Economic Forum doing to accelerate action on Global Health?
The Big Picture
Explore and monitor how Global Health is affecting economies, industries and global issues
A hand holding a looking glass by a lake
Crowdsource Innovation
Get involved with our crowdsourced digital platform to deliver impact at scale
Stay up to date:

Global Health

Loading...
  • A recent study has found that only 1 in 20 medical treatments have high-quality evidence supporting their benefits.
  • The harms of treatments are measured much more rarely than the benefits, having only been reported in 33% of treatment reviews.
  • More research funding is need to ensure that high-quality evidence is generated for treatments that are widely used, but not yet supported by high-quality evidence about their benefits and harms.

Only one in 20 medical treatments have high-quality evidence to support their benefits, according to a recent study. The study also found that harms of treatments are measured much more rarely (a third as much) as benefits.

Patients and doctors – and anyone who pays for them – need to know that medical treatments are safe and effective, but it’s an open secret in the medical field that not all treatments, including ones that are commonly used, are safe and effective. For example, antiarrhythmic drugs were widely prescribed in the belief that they would reduce heart attack deaths until a clinical trial found that they actually increased the risk of death.

In another example, putting infants to sleep on their stomach was recommended based on expert opinion that babies would be less likely to choke on their vomit until large studies found that stomach sleeping increased the risk of sudden infant death syndrome.

So how is this lack of evidence for medical treatments a big problem?

In the early 2000s, researchers estimated that between a quarter and a half of medical treatments are supported by high-quality evidence. But these estimates are now out of date and used old methods (such as researcher opinion) to determine whether the evidence was high quality or not. More recently, in 2020, a more rigorous estimate was published and found that only 10% of medical treatments were based on high-quality evidence. However, this estimate was based on a small sample of 151 studies.

Meanwhile, some continue to insist that most medical treatments must work. How else can we explain that we live ten years longer than our great-grandparents? Yet the extension in lifespan is explicable at least partly by public health measures such as clean water, better nutrition and restrictions on smoking.

95% of medical treatments do not have high-quality evidence to support their benefits.
95% of medical treatments do not have high-quality evidence to support their benefits. Image: Unsplash/Johnny McClung

A more accurate picture of medical treatments

To resolve the controversy about the proportion of treatments that are based on good evidence, an international team of researchers from the UK (University of Oxford), US, Switzerland and Greece conducted a large study of 1,567 healthcare treatments. The sample included all medical treatments tested in Cochrane Reviews between 2008 and 2021. Cochrane Reviews are rigorous studies that amalgamate all available relevant evidence about treatments. They are often referenced in national and international healthcare guidelines.

The year 2008 was chosen as the cut-off because that was when Cochrane Reviews incorporated a system called grading quality of evidence and strength of recommendations (Grade) to rate how trustworthy the evidence is. Unlike the earlier estimates that often relied on opinions, Grade is more widely accepted and is used by over 100 organisations around the world. Using Grade results in a quality rating of high, moderate, low or very low.

The study revealed that 95% of medical treatments do not have high-quality evidence to support their benefits. Worse, the harms are reported in only about 33% of Cochrane Reviews.

It is particularly worrying that the harms of healthcare interventions are rarely quantified. For a doctor or patient to decide whether to use a treatment, they need to know whether the benefits outweigh the harms. If the harms are inadequately measured, an “informed choice” is not possible.

A potential limitation of the study is that Grade might be too strict. Doctors and patients may be happy to use treatments whose benefits are not supported by high-quality evidence as long as they are supported by moderate-quality evidence. Even if this is right, the study found that less than half of treatments are supported by high or moderate-quality evidence.

Patients with ailments for which there are no effective medical treatments may be willing to try treatments that are not yet even supported by low-quality evidence. The study should not be used to constrain these patients’ choices.

Also, the sample may not have been representative. In theory, treatments tested in recent Cochrane Reviews may be less effective or based on lower-quality evidence than older treatments. However, given the rigour of Cochrane Reviews, this seems unlikely.

In practice, doctors can use “off-label” treatments which are less likely to have been studied in Cochrane Reviews and generally have lower-quality evidence to support them. Despite these potential limitations, the study still showed that most medical treatments are not supported by high-quality evidence.

Doctors, patients and those who pay for them may wish to focus on treatments whose benefits and safety are established by high-quality evidence. Research funding should be allocated to generating high-quality evidence for treatments that are widely used but not yet supported by high-quality evidence about their benefits and harms.

Finally, potential harms should be measured with the same rigour as potential benefits. The evidence-based medicine community is correct to continue calling for higher-quality research, and also justified in their scepticism that high-quality evidence for medical treatments is common or even improving.

Discover

What is the World Economic Forum doing to accelerate value-based health care?

Have you read?
Loading...
Don't miss any update on this topic

Create a free account and access your personalized content collection with our latest publications and analyses.

Sign up for free

License and Republishing

World Economic Forum articles may be republished in accordance with the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License, and in accordance with our Terms of Use.

The views expressed in this article are those of the author alone and not the World Economic Forum.

Related topics:
Global HealthHealth and Healthcare
Share:
World Economic Forum logo
Global Agenda

The Agenda Weekly

A weekly update of the most important issues driving the global agenda

Subscribe today

You can unsubscribe at any time using the link in our emails. For more details, review our privacy policy.

Kidney disease ‘should be global health priority’, plus other top health stories

Shyam Bishen

April 10, 2024

About Us

Events

Media

Partners & Members

  • Join Us

Language Editions

Privacy Policy & Terms of Service

© 2024 World Economic Forum