How Big Data and AI are helping sift through data

By Karl Hightower, Chief Data Officer & SVP – Data Products and Services, Novant Health

How do you use data to find long-term chronic health problems?

Healthcare practitioners function in an event-based system with some discreet data points, but most of the good information about patients is contained in unstructured notes.

How can we use these notes to piece together chronic conditions and their causes across multiple visits, multiple doctors, across a sea of time?

Fortunately, many other industries have been down this path before and found ways to transform how they used data and technology to differentiate themselves. Healthcare, meet Big Data, AI, and the Digital Twin!

The glut of data, data everywhere

Each visit to a healthcare provider generates a ton of data about who you are, why you are visiting, what the diagnoses are, additional information needed in labs or images, and next steps. After the visit, even more information is generated through back-end billing information to the insurance provider.

Over the course of many different visits, to different providers, different systems, and over many years, the amount of data can create a complex, messy picture.

The retail transformation brought on by Amazon offers a roadmap for how healthcare might handle this glut of data. Amazon users know that the company creates an efficient and tailored experience based on what the user has looked at and purchased before. It’s almost like they knew what you wanted, and it engaged you in a way that you have now come to expect. This did not happen without using many individual points of data over time to get patterns and actions.

In healthcare, unlike in retail, much of the important data must be teased out of free-text notes. In order to get a accurate picture of a patient’s health, all of the relevant historical notes and images would have to be read and assimilated.

Where is the good stuff?

In healthcare, unlike in retail, much of the important data must be teased out of free-text notes. In order to get a accurate picture of a patient’s health, all of the relevant historical notes and images would have to be read and assimilated. This type of investigation is laborious for clinicians but also necessary to truly understand chronic diseases.

Let’s take a typical MS patient, for example. Flare-ups may manifest themselves over years, coming in loss of hearing for a week or two, losing eyesight for a brief period of time, or numbness in the legs that seems to come and go. The patient might go to an ear doctor, an eye doctor, and a spinal specialist. The individual specialists might not put together a bigger picture because of the nature of the interfaced systems, the dispersed data, and the amount of time needed to review all of the notes. Those notes could include innocuous-seeming but insightful clues: headache, fatigue, the environment at the time of the flare-up.

This example shows that much of a patient’s data is not captured in regular columns but is contained in free-text fields that, unless read or pulled out, are the missing key to creating a full picture of health.

Unlocking it into something I can use

Using Natural Language Processing and Machine Learning, or ‘AI’, against this sea of data is the key to distilling out useful bits of data to be a part of the complete picture. By adding these key data points to the visits through NLP, the ML can start to recognize patterns that are more meaningful to understanding the overall patient health profile.

The key is providing that information within an easy-to-consume and actionable manner that removes the friction of too much data and not enough information. At Novant Health, we implemented these processes in the electronic medical recordMeeting the clinicians where they spend their working time in the EMR means that all the data is put through the Big Data and AI machine to assist clinicians with investigation and treatment.

Making big progress and the challenges ahead

While implementing the NLP and ML is technically challenging, the bigger hurdle in our heathcare system and in others is AI’s acceptance. We’ve met that challenge by holding to explainable AI and no black boxes principles, and the technical teams have worked side-by-side with those who will use the new tools. That clinical user input and partnership help zero in on the differentiating bits of information and provide insights into what is needed next.

The next iteration of healthcare AI will be the development of a “digital twin,” a digital model of the patient created using the full range of collected data. Clinicians can use the digital twin, with all new compute power, to simulate different treatment factors.

In this next frontier of digital medicine, the digital twin will also provide new insights about how diet affects chronic diseases and will help narrow down food triggers without trial and error.

In this future of personal health, data will be a more powerful tool for clinicians and patients in managing disease because important information will no longer be left behind in clinician notes.