The Defense Advanced Research Projects Agency (DARPA) has selected Raytheon BBN to lead a team to develop an augmented reality device that will provide the combat medic with a virtual assistant. The system will use a set of AR goggles, which will provide visual information on 50 different medical procedures.
Medics are highly trained in the most common battlefield injuries, but they aren’t doctors or surgeons and often have no experience in little-used procedures which may be needed at a moment’s notice. This is why DARPA is working on its Medical Assistance, Guidance, Instruction and Correction (MAGIC) system.
MAGIC uses a pair of augmented reality goggles equipped with audio and video sensors and special artificial intelligence software that can act as an assistant to monitor the situation and advise the medic on how to proceed.
Raytheon will use machine learning technology to ‘teach’ the system both medical skills and situation assessment skills. The initial prototype will study 2,500 stereo videos and almost 50 million images. The machine learning process will review the historical data and synthesize useful concepts and solutions from that data.
When the AI software is ready, MAGIC should be able to provide spoken suggestions to medics or project visual overlays on the scene to guide their hands through needed medical procedures. The system will also provide events timing from engagement to final hand-off to field hospital personnel. MAGIC will also provide dosage guidance for in-field medications.
A first prototype is expected in about 18 months.
"The combat medical environment is challenging and chaotic," said Raytheon BBN scientist Brian VanVoorst. "Our goal for the Raytheon BBN MAGIC AI tool is to help support personnel to provide guidance as needed without disrupting their concentration."