AI Will Help in the Pandemic – but It Might Not Be in Time for This One

From predicting outbreaks to devising treatments, doctors are turning to AI to combat the COVID-19 pandemic.

Why It Matters. While machine learning algorithms were already becoming a part of health care, COVID-19 is likely to accelerate their adoption. But lack of data and testing time could hinder their effectiveness – for this pandemic, at least.

What's Happening. With millions of cases and outbreaks in every corner of the world, speed is of the essence when it comes to diagnosing and treating COVID-19. So it's no surprise doctors were quick to employ AI tools to get ahead of what could be the worst pandemic in a century.

• HealthMap, a web service run by Boston Children's Hospital that uses AI to scan social media and other reports for signals of disease outbreaks, spotted some of the first signs of what would become the COVID-19 outbreak. This was days before the WHO formally alerted the rest of the world.
• Early in the epidemic, the Chinese tech company Alibaba released an AI algorithm that uses CT scans of possible coronavirus patients and can diagnose cases automatically in a matter of seconds.
• In New York, Mount Sinai Health System and NYU Langone Health have developed AI algorithms that can predict whether a COVID-19 patient is likely to suffer adverse events soon and determine when patients will be ready to be discharged. Such systems can help overburdened hospitals better manage the flow of supplies and personnel during a medical crisis. 

The Big Picture. Even before COVID-19, AI was already becoming a more significant part of modern health care. Nearly $2 billion was invested in companies involved in health care AI in 2019, and in the first quarter of 2020, investments hit $635 million – more than four times the amount seen in the same period of 2019, according to digital health technology funder Rock Health.

• The advance of AI is partially a result of the rapid increase in data, the lifeblood of any machine learning system. The amount of medical data in the world is estimated to double every two months.
• Engineer and entrepreneur Peter Diamandis told Wired an estimated 200 million physicians, scientists, technologists, and engineers are now working on COVID-19, generating and sharing data "with transparency and at speeds we've never seen before."
• "We understand who is at risk and how they're at risk, and then we can get the right treatment to them," says Zeeshan Syed, the CEO of Health[at]Scale, an AI health care startup.

In Trials. AI has demonstrated in recent trials a decent record of success, especially when it comes to rapidly diagnosing COVID-19 by interpreting medical scans.

• A study published in Nature Medicine in May found an AI system was more accurate than a radiologist in diagnosing COVID-19 patients using CT scans – X-ray images of lungs – combined with clinical symptoms. 
• A systematic review of preprint and published studies of AI diagnostic systems for COVID-19 published in the British Medical Journal in April noted the models reported "good to excellent predictive performance." However, they cautioned the data was still limited for real-world applications and at high risk for bias.

The Catch. That's the perennial challenge for AI systems in any field. Experts worry models that perform well in an experiment may not be able to replicate that success in a hospital under stress.

• "There is a lot of promise in using algorithms, but the data in the biomedical space can be really difficult to deal with," says Gabe Musso, the chief science officer at BioSymetrics. This biomedical AI company uses machine learning for simulation-based drug discovery. Genetic data, imaging data, and data from electronic health records are often unstructured and rarely share a common format, complicating efforts to feed the information into an algorithm.
• Many of the AI diagnostic systems being rushed into the fight against COVID-19 were developed before the pandemic and thus were trained on other respiratory diseases like tuberculosis. That reduces their accuracy – especially if their training datasets don't match the gender or age of typical COVID-19 patients.
• As a result, pioneering computer scientist Kai-fu Lee wrote recently, "I would give [AI] a B-minus at best" for its performance during the pandemic.

The Bottom Line. As both the size and quality of medical data on COVID-19 improves, so should the AI systems that draw from it. But that will take time.