Once again the lure of a “simple” test has made headlines.
This time the catalyst was a news release issued by the UK’s National Cancer Research Institute. Its title: “Simple blood test for early detection of breast cancer.”
In fact, no such test exists. As the body of the PR news release acknowledged, it’s an idea based on preliminary research.
Yet a spate of optimistic headlines appeared.
New blood test could detect breast cancer 5 years before symptoms, announced the Australian, adding in a subhead: “eradicating the need for mammograms.”
Bloomberg ran with: Blood Test to Detect Breast Cancer Could Be Five Years Away. The Guardian wrote: Breast cancer may be detected by blood test five years before clinical signs show.
The New York Post ran a ridiculously brief 69-word story: New blood test can detect breast cancer 5 years before lumps appear.
Fox TV stations across the U.S. all carried the same story on their websites topped with similarly misleading headlines. A Chicago Fox affiliate tweeted it as a “CANCER BREAKTHROUGH.”
Not surprisingly, the hype was lambasted by experts on Twitter, such as a cancer biologist at the University of New South Wales in Australia who wrote a string of comments.
It may be unfair to compare this to the massive Theranos scandal in which a company faked having technology that could run a battery of tests with a single drop of blood.
Still, journalists could have been more analytical about this news release, which claimed breast cancer “could be detected up to five years before there are any clinical signs of it, using a blood test that identifies the body’s immune response to substances produced by tumour cells.”
For one thing, news stories brushed over the question of whether the test would be sufficiently accurate. As we’ve written many times, this is a common problem with coverage of “simple” tests.
The PR news release said that among 180 patients studied, half with cancer and half without, panels assessing various numbers of tumor-associated antigens (TAAs) correctly detected breast cancer in 29-37% of the samples from people with cancer and correctly identified 79-84% of the control samples as being cancer-free.
That means the test actually missed most cancers and wrongly tagged one-fifth of healthy people as having cancer. That’s a lousy result that would have major repercussions, as experts pointed out.
The promotional news release said researchers “are now testing samples from 800 patients against a panel of nine TAAs, and they expect the accuracy of the test to improve with these larger numbers.” It quotes a researcher saying: “Once we have improved the accuracy of the test, then it opens the possibility of using a simple blood test to improve early detection of the disease.”
But the news release doesn’t explain how the test might be improved, and news stories didn’t point out that deficiency. News coverage also didn’t caution about the flipside: the possibility that accuracy won’t improve.
Wakefield, an ovarian cancer research at the University of Melbourne, said in an email: “You would expect the error rates to scale with the larger sample, and not change the proportion of true and false positives, unless the sample they used was not representative of the population to start with (which would be a problem in itself). Only changing the test will improve those figures.”
Other important questions
The PR release makes another big leap that reverberated in the headlines: ““The researchers estimate that, with a fully-funded development programme, the test might become available in the clinic in about four to five years.”
But Wakefield threw cold water on that idea:
“If they had a test now that had reasonable true and false positive rates, that would be a realistic time frame for developing the test format into a commercially viable form, gaining the required testing accreditation and starting a full scale trial.
Given the current status, add 3-5 years to get the test performance to a point where that would be a viable proposition, with no certainty that it can be achieved. I would also question their ability to get funding based on current results.”
Steven Salzberg, a professor of biomedical engineering, computer science and biostatistics at Johns Hopkins University, wrote: “I think it’s generally good to show exciting scientific progress, but the news media often over-simplify (as they did in this case, in some venues) and make it sound like a clinical test is right around the corner. That’s bad because it’s misleading, and it may lead to people deciding that science never really delivers (which isn’t true).”
Another question that went largely unexplored is whether such earlier detection would benefit women. The New York Post wrote: “Earlier detection means earlier treatment, which could possibly save thousands of lives.”
There may be benefits of identifying women who have “substances produced by tumor cells” circulating in their blood. But there may also be harms to such screening, such as unnecessary treatment.
Some better coverage
Only a few news organizations provided independent sources.
For example, HealthDay News provide some balance. In the fourth paragraph it stated: “Still, the test is so far only partially effective, so much more research is needed, the investigators said.”
Further down, HealthDay provided feedback from an outside expert.
One U.S. breast cancer expert was encouraged by the findings, but agreed there’s still a long way to go before blood tests replace mammograms in breast cancer detection.
“It will be great if someday we are able to diagnose breast cancer by a simple test,” said Dr. Lauren Cassell, chief of breast surgery at Lenox Hill Hospital in New York City. “However, the current test, which utilizes nine TAAs, only identified cancer in 37 percent of known cancer samples. What about the other 63 percent where it didn’t identify the cancer?” she said.
“This does not appear to achieve the level of accuracy we would like to achieve or which we do reach with our current mammogram and sonogram in terms of detection,” Cassell said.
HealthDay also cautioned:
Because these findings were presented at a medical meeting, they should be considered preliminary until published in a peer-reviewed journal.
The Guardian stated in its lead: “However, other cancer experts have warned these claims should be treated with caution.”
Further down it wrote:
Other researchers have urged caution in interpreting these results. “These are clearly very preliminary data,” said Cambridge University cancer epidemiologist Prof Paul Pharoah. “A lot more research would be needed before any claim can be made that this is likely to represent a meaningful advance in the early detection of cancer.”
Warwick University molecular oncologist Prof Lawrence Young agreed: “While this is encouraging research, it is too soon to claim this test could be used to screen for early breast cancer. More work is needed to increase the efficiency and sensitivity of cancer detection.”
High stakes for researchers
When it comes to announcements about blood tests, caution is warranted. “Researchers are under huge pressure to show impact of their work to get funding,” Wakefield wrote. “This means that lots of interesting basic research is repackaged as a potential test, without input by pathologists, epidemiologists and health economists that a real test would involve.”
Fallout from such pressure is on display in Germany, where the director of the women’s clinic at Heidelberg University Hospital was accused of “extensive and severe misconduct” after he provided false information to the news media about a blood test to detect breast cancer. A PR news release jointly issued by the hospital and a company marketing the test heralded it as a “new, revolutionary option” and obscured its sky-high false positive rate, according to Science.
A story in in Bild, Germany’s biggest tabloid newspaper, called the test a “world sensation.”