Larry was admitted to hospital for gallbladder removal, a common surgical procedure. Unfortunately, during his operation, Larry’s bile duct was injured. A rare occurance – it happens in less than 0.5% of all gallbladder removals – but it can have catastrophic consequences, As such, Larry was transferred to Massachusetts General Hospital for more specific care.

What Larry thought would be a minimally invasive, one-day operation involving four small incisions, resulted in a second major operation, a month in hospital, followed by months of painful rehabilitation.

“Fortunately, Larry recovered but we have to ask ourselves, can this be better prevented?” says Dr. Daniel Hashimoto, Associate Director of Research at the Massachusetts General Hospital Surgical AI and Innovation Laboratory (SAIIL).

Dr. Hashimoto explains that every year, 48 million surgical procedures are performed in the US with injuries occuring in approximately 2% of cases, resulting in nearly a million patients being adversely affected. These mishaps, some of which are avoidable, not only impact patients’ quality of life, sometimes causing permanent disability or even death but also push the cost of hospitalization up by 41%. Dr. Hashimoto believes tracking how many operations a particular surgeon has undertaken; how many went well, how many led to complications, readmissions etc, could yield highly valuable lessons. “Studying outcomes makes us better surgeons,” he says. “Because we can learn from the data.”

But datasets from patient registries or insurance companies are written in code eg: K81.0 denotes a gallbladder infection; 47562 is the minimal invasive procedure to remove the gallbladder and S36.13XA is the injury of the bile duct. All of which do nothing to inform surgeons as to why the injury occurred, what happened in the operating room; what was the sequence of events that led to the problem, and how it could be avoided in the future. So Dr. Hashimoto challenged his fellow engineers and research colleagues at MIT, to find a better solution.

“When surgeons like myself perform minimal invasive procedures, we use a fibre optic camera to look inside the abdomen,” says Dr. Hashimoto. “But we never thought of using this footage beyond the actual operation. Every day, my MIT colleagues use data extracted from cameras to understand their values in developing intelligent vehicles but have never used their applications in surgery. So, we thought, why don’t we combine what we have?”

Dr. Hashimoto and his team took a sequence of videos in surgeries and converted them into quantitative data to train AI models to understand what had taken place in an operation. They discovered the technique gave over 93% accuracy in identifying and interpreting what occured during operations. During a surgical procedure, the model would compare what was taking place with operations that had occured in the past. Surgeons would then be informed if they went off trajectory.

The model is also capable of detecting when a surgeon’s action is about to deviate from expectations. It can then warn the surgeon and allow them to make a decision about correcting their approach to avoid causing injury. “This is leveraging AI for real-time clinical decision support,” Dr. Hashimoto adds. “It’s like having a GPS for surgery.” This development has been the cornerstone of SAIIL, which focuses on the utilization of computer vision for insights into intraoperative care.

Currently, SAIIL and Dr. Hashimoto are building a ‘collective surgical consciousness.’ Dr. Hashimoto explains AI has the ability to learn from hundreds or even thousands of medical cases at once but we still lack a major database of videos to train this at scale. Of course, there are strict rules governing how hospitals should use their data. But some of these were written decades ago before the technology was even developed to enable use of this data.

Dr. Hashimoto hopes that through an initiative like SAIIL, fellow surgeons can work together to aggregate their experiences and share their techniques in a quantitative, data-driven manner for the purpose of training better AI models to guide surgeons. Such a massive amount of data also allows surgeons to quickly compare techniques, model possible surgical outcomes and make suggestions on next steps.

“Our goal is to make surgery safe, better, and more affordable and accessible to the people who need it,” Dr. Hashimoto. “Ultimately, we aim to reduce complications, save money and lives.”