Alan Stein is a Vice President at Saxony Partners. Connect with him on LinkedIn.
For those familiar with Gartner’s proprietary hype curve, you’ll likely find Artificial Intelligence technologies somewhere between “Peak of Inflated Expectations” and the “Trough of Disillusionment.”
AI is sexy. It’s having a moment, as the kids say. Because of this, it’s hard to know what about AI is hype and what is real. The good news for the healthcare industry is that, when you filter out the noise and drill down on specific use cases, it’s clear that the value and impact on productivity in healthcare is very real.
We’re getting a sense for this, ironically, by way of human error.
You’ve heard the old project axiom, “when you can identify what’s not going well on a project, you can fix it”? It used to be that most AI projects simply failed to launch. Now, launches are becoming common – although the eventual failure rate remains high. Even the post-launch mistakes that are driving these failures are following similar patterns, indicating that thought processes across this emerging industry are beginning to coalesce.
This incremental improvement indicates that we are moving in the right direction regarding AI implementation.
Higher Risks, Higher Rewards, Same Rules
The fundamentals of a successful healthcare AI project are the same as they have been for generations – hardware, software, source data, team dynamics, timelines. The power of technologies like machine learning, predictive and prescriptive analytics has served to raise the stakes of doing those fundamentals right.
One big factor, though, is consistently overlooked in AI implementations – insufficient focus on the human decisions and outcomes that are being automated or supplemented. Those in charge of healthcare AI projects would do well to repeat this mantra daily: “Focus on the decisions and outcomes at every step."
Applying the Focus
Let’s walk through what it means to focus on decisions and outcomes at every step.
- Business Understanding
- Data Acquisition and Understanding
- Pre-Model Data Processing
- Model Algorithm
- Model Feature Engineering
- Model Training
- Model Validation
- Model Deployment
- Model Web API
- Customer Acceptance
- Model Retraining
- Future Model Enhancements
It’s no coincidence that nine of those twelve steps have the word “model” in their title. Virtually all AI technologies are centered around the model. That’s the way it should be, to an extent – models stand as the foundation for AI projects.
As Yasir would tell you, we consistently see AI project teams falling into the trap of what we call “model tunnel vision.” This happens when there’s a fixation on getting the data to the modelers, then get the modelers going on model development and training, and then testing the models, and then rolling out the models, and of course profit.”
Let’s look at those twelve steps, again, though, and apply our mantra: Focus on the decisions and outcomes at every step. Let’s look at the questions that teams should be asking?
Business Understanding: What decisions are healthcare professionals making that we may improve with our AI project? What are today’s outcomes – and what would an improved outcome (one driven by AI) look like?
Data Acquisition and Understanding: What data do people use today to make those decisions? How are they using it? How does its accessibility, quality, volume, and structure impact outcomes today? How might that change when the data is feeding a model?
Pre-Model Data Processing, Model Algorithm/Feature Engineering/Training/Validation/Deployment: The process of getting data ready for the model (as well as drafting the model, mapping its features, its training, and its validation) is iterative. Each iteration improves based on answers to this question: How is this improving our decisions and outcomes?
Model Web API: The model results need to be integrated into the user’s environment. How? Often, it’s an API that interfaces with a user tool. This step, which is frequently an afterthought in AI projects, is crucial to success. Great models can be rendered worthless by not thinking through how and when the user gets their results – results which, of course, rest upon the dynamics of the decisions and outcomes.
Customer Acceptance: There are two reasons why customer acceptance places the highest premium on a decisions-and-outcomes focus.
First, the hype around AI (in both professional and personal settings) has made it feel like some sort of magic. This is not the case, of course – it’s science, not magic. Therefore, it’s critical to manage customer expectations.
Second, a general lack of public trust in automated decision making is beginning to emerge. Since the decision-making processes for the most advanced technologies (deep learning, machine learning, etc.) can be opaque – the public can have a hard time determining how much trust to invest in AI-assisted suggestions and outcomes.
The focus on decisions and outcomes is a two-way street. Project teams need to understand users, and they need to educate users on what AI can and cannot do. Of course, this responsibility is critically important in clinical contexts.
Model Retraining and Future Model Enhancements: Now you’re in production. The model can’t be static or brittle; rather, it must be a dynamic, living thing. Resilient models change with the source data and organizations usually get that part right. But they also must change with business processes and outcomes.
These differences that separate resounding success versus a so-so AI project are subtle. After all, nobody sets out to ignore the decisions and outcomes. Teams simply succumb to the distractions. The AI hype — and there ain’t no hype like AI hype — amplifies this effect.
Thankfully, AI’s promise appears to be surviving the hype – and the benefits for the entire healthcare industry will be worth it – that is, as long as implementors remember to focus on decisions and outcomes at every step of the process.