IBM’s artificial intelligence software, Watson, isn’t ready to replace your doctor just yet.
Medical experts working with the tech giant on its Watson for Oncology system discovered that it made ‘often inaccurate’ and ‘unsafe’ treatment recommendations, according to internal documents reviewed by Stat News.
However, no patients were reportedly harmed despite Watson’s missteps.
Scroll down for video
Medical experts working with IBMon its Watson for Oncology system discovered that it made ‘often inaccurate’ and ‘unsafe’ treatment recommendations, internal documents showed
The documents were included in two presentations given in June and July 2017 by IBM Watson’s former deputy health chief Andrew Norden.
In one case, a 65-year-old patient was diagnosed with lung cancer and said he had developed severe bleeding.
Watson recommended he be given chemotherapy and receive a drug called ‘Bevacizumab,’ but Stat News noted that the drug can result in ‘severe or fatal hemorrhage’ and shouldn’t be administered to people with severe bleeding.
A spokesperson for Memorial Sloan Kettering told Stat News that the suggestion was hypothetical; instead, it was a part of system testing.
However, several people related to the project said Watson’s inaccuracies raise ‘serious questions about the process for building content and the underlying technology.’
‘This product is a piece of s***,’ a doctor at Jupiter Hospital in Florida told executives at IBM, according to Stat News.
In one case, Watson recommended that a patient with lung cancer and internal bleeding be administered a drug that can result in ‘severe or fatal hemorrhage’
HOW DOES IBM’S WATSON HELP DIAGNOSE CANCER?
The Watson artificial intelligence system, developed by US firm IBM, examines medical images and patient records.
These are then compared to thousands of past cases and medicals journals to come to a conclusion.
Doctors at 55 hospitals around the world have been using the system to help them diagnose patients for breast, lung, colorectal, cervical, ovarian, gastric and prostate cancers.
IBM claims the technology will be extended to detect a further five cancer types by the end of the year and will be rolled out to other hospitals worldwide.
‘We bought it for marketing and with hopes that you would achieve the vision. We can’t use it for most cases.’
Not only did people have concerns about Watson’s suggestions, but people also questioned the way that Watson was trained for this particular project.
The AI system was supposed to analyze data from real patients, but instead it was fed hypothetical data.
It meant that doctors were receiving recommendations from other doctors, not those reached by an AI synthesizing vast amounts of data and reaching its own conclusions.
Doctors at Jupiter Hospital told Gizmodo that instead of relying on Watson, they use the supercomputer as an extra opinion if they cannot agree on treatment.
So while it’s not a glowing endorsement, it appears medical professionals are still using Watson in some capacity.
Additionally, a spokesperson for IBM told Gizmodo that Watson is still learning and is currently being used by 230 hospitals around the world, supporting care for over 84,000 patients.
The company also disagreed in part with the internal documents, saying they don’t give a timely representation of Watson.
‘We have learned and improved Watson Health on continuous feedback from clients, new scientific evidence, and new cancers and treatment alternatives,’ the spokesperson told Gizmodo.
Watson was supposed to analyze data from real patients, but instead it was fed hypothetical data. It meant that doctors were receiving recommendations from other doctors, not those reached by an AI synthesizing vast amounts of data and reaching its own conclusions
‘This includes 11 software releases for even better functionality during the past year, including national guidelines for cancers ranging from colon to liver cancer.’
IBM released a report last June that said Watson could accurately identify tumors in up to 93% of cases following tests on real patients.
It is hoped the system will be able to reduce the amount of time it takes to screen patients and help improve the accuracy of diagnosis.
WHAT DOES IBM’S WATSON DO?
Watson already has won a major TV game show, is looking for a cure for cancer and has ambitious gastronomy ambitions including devising a recipe for chocolate-beef burritos.
The IBM supercomputer is becoming a jack of all trades for the US tech giant — including in its new role as a business consultant and analyst for various industries by using massive Internet databases.
IBM has developed a Watson Engagement Advisor application to counsel members of the military and their families how to smartly manage shifting to life after the service.
IBM announced last April that its Watson technology would be used for tracking down rogue traders at financial firms
In the oil and gas sector, IBM has worked with the British tech group Arria to integrate Watson’s capabilities to help improve management of leaks in refineries.
Watson has teamed with Elemental Path, maker of ‘smart toys,’ such as a dinosaur that can tell stories and answer questions from children.
The computer has in its memory thousands of recipes from ‘Bon Appetit’ magazine, and it also knows the chemical properties of foods.
If Watson suggests marrying strawberries with mushrooms, it’s because the two foods share a chemical bond.