ChatGPT Almost Derailed Your Healthcare AI App? Here’s Your Next Step

Joe Tuan

Founder, Topflight Apps

November 30, 2023

In the fast-paced world of AI and machine learning, OpenAI’s ChatGPT has emerged as a game-changer, revolutionizing the healthcare application landscape. Yet, recent shifts within OpenAI have raised questions about the sustainability of relying solely on this proprietary technology.

This blog delves into the nuances of these changes, the potential risks for founders, and the possible mitigation strategies. Are you ready to secure your healthcare AI app’s future amid these uncertainties? Let’s dive in.

Top Takeaways:

• The Importance of Diversification: Diversifying your AI toolset is crucial to mitigate risks and ensure uninterrupted value delivery. Explore alternatives like open-source models or other proprietary LLMs to complement or fall back from relying solely on ChatGPT APIs.
• The Potential of Microsoft’s Azure OpenAI Service: Microsoft’s Azure OpenAI service is a promising contender, offering robust infrastructure, scalability, and reliable support. Transitioning can safeguard your healthcare app without compromising its core value, but consider the challenges for optimal utilization.

Table of Contents:

1. Setting the Stage: OpenAI’s Recent Shifts
2. Understanding the Impact of OpenAI’s Changes
3. Exploring Alternatives to Mitigate Risks
4. Azure OpenAI: A Promising Contender
5. Top Open-Source Language Models
6. Ready for the Next Step?

Setting the Stage: OpenAI’s Recent Shifts

OpenAI has recently undergone significant shifts in its organizational makeup that could cast a cloud over its future trajectory. The little ping-pong action with the company’s CEO, who is not only a visionary leader but also central to defining OpenAI’s strategy, has introduced an unwanted dynamic. It took less than a week for Sam Altman to become fired, “move on” to Microsoft, and then get back to his brainchild.

During Altman’s dismissal, almost 800 employees (including principal engineers) also decided to part ways with OpenAI and follow the founder. These individuals were the creative force behind ChatGPT LLMs, and their exit would call into question the company’s capacity to maintain its innovative edge.

It’s quite amusing how all of this unfolded right after OpenAI unveiled a range of remarkable updates to its ChatGPT LLM.

Should founders of healthcare digital solutions pay attention to these ChatGPT shenanigans?

After all, it seems like the situation has settled down.

Of course! These events only highlight the fragility of relying entirely on proprietary technology. It’s hard to resist the allure of a technology that helps clinician founders tackle so many challenges:

• creating comprehensive medical notes and summaries
• simplifying complex medical jargon for easier comprehension and translating medical info
• compiling empathetic messages in patient support groups to provide emotional support
• drawing from a proprietary knowledge base to explain treatment plans in detail
• facilitating consistent patient engagement via personalized follow-up messages
• replacing alerts, graphs, etc., with an interactive chat interface for providers

So why should you still consider fallback options?

Understanding the Impact of OpenAI’s Changes (November 2023)

The recent changes at OpenAI should serve as a wake-up call for founders who have tethered their healthcare apps’ core functionality to the company’s technology, particularly ChatGPT; let’s delve into understanding the potential implications of these changes.

Sam Altman and Key Engineers Leaving OpenAI

Developers well-versed in engineering advanced machine learning solutions are an extremely scarce asset these days. If Sam leaves OpenAI and 800 or so employees (including leading engineers) follow him – this creates a humongous gap in OpenAI’s human capital, leading to:

• Lost momentum – who’s going to work on GTP-5?
• Much slower development pace

As a result, the company might even have to revisit its pricing strategy and face slower adoption of paid services and less exciting DAU stats.

What This Could Mean for ChatGPT’s Future

In addition, OpenAI depends on Microsoft’s processing power to train its LLM models, which includes quite an expensive server infrastructure. Without it, ChatGPT’s development pace would come to a screeching halt.

Plus, Microsoft could decide to play dirty and get Sam and all the new staff busy with ChatGPT-6 before OpenAI can even start to think about it. Gloomy worst-case scenario, right? Thank goodness, everything is back to normal.

The Potential Risks for Founders Dependent on ChatGPT

As you understand, healthcare founders who use ChatGPT could get into trouble:

• slower release of new ChatGPT APIs hindering their product development momentum
• less or hardly accessible support, causing major hiccups in resolving technical challenges

Being stuck with a tool with no clear future is not so much fun. Founders will need to hedge, just in case, even though the present situation with OpenAI has stabilized.

Read more on ChatGPT in healthcare in our blog.

Exploring Alternatives to Mitigate Risks

Given the uncertainty surrounding ChatGPT, it’s prudent to explore alternatives for your healthcare AI app. Broadly speaking, these alternatives fall into three categories: open-source tools, other proprietary LLMs, or a combination of both.

Diversifying with Open-Source Software

Open-source alternatives provide the freedom to customize and tweak the language model according to your specific needs. Tools such as GPT-J-6B, GPT-Neo, and EleutherAI’s GPT models are worth considering. While they may not match the performance of ChatGPT, they can be a cost-effective solution, and you’ll avoid dependency on a single company.

Pros:

• you’re in complete control of product development
• lower risk of unauthorized access to sensitive data
• cost savings

Cons:

• switching to a new LLM takes a lot of resources (prompt modifications at minimum)
• no guarantee of the same quality and capabilities (e.g., missing function calling, which is available in ChatGPT APIs)

It certainly helps if you’ve built the existing ChatGPT implementation using some flexible framework like LangChain. They can handle adaptations to different models (including open-sourced ones).

Proprietary LLM Alternatives

If you’re not keen on managing your own infrastructure, other vendors offer proprietary LLMs. Google’s BARD, Microsoft’s Turing, Facebook’s BART, and Anthropic’s Claude are all strong contenders. While they also come with a dependency risk, diversifying your toolset among multiple providers can spread this risk.

This scenario is a clear waste of resources without immediate advantages unless you have a strategy to make use of the AI and BigData infrastructure provided by a particular vendor like Google’s Vertex AI.

Naturally, any transition like that should align seamlessly with your product’s long-term vision and offer distinct advantages. For instance, it could facilitate swifter development or reduce processing costs associated with LLM maintenance.

Combination of Both

The third option is to use a combination of open-source and proprietary tools. The proprietary LLM would be your primary tool in this setup, with the open-source model acting as a failsafe. This arrangement offers a balance between having a high-performing LLM and a backup plan should there be any unforeseen disruptions with the proprietary LLM.

To safeguard your healthcare AI app against potential disruptions in the future, consider the implementation of a router like OpenRouter or Helicone’s Gateway Fallback. Such a system can seamlessly switch between different language models in case of an error. For instance, if ChatGPT becomes unavailable, the router will automatically activate an open-source model. This way, your app continues providing valuable services without any major interruptions.

Pros:

• ensures continuity of service
• provides an effective backup strategy
• mitigates dependency on a single tool

Cons:

• requires technical know-how to set up and maintain
• demands ongoing support and updates to ensure compatibility with multiple models
• higher costs

An optimal but costly solution, as you can imagine.

The Best “Alternative” to ChatGPT at the Moment

Despite the uncertainties and potential risks, Microsoft’s Azure OpenAI service currently stands out as the best proprietary solution, offering robust performance and reliable infrastructure for an AI health app that runs on ChatGPT at its core.

Topflight’s leading ML engineers and product managers have reached a consensus: Azure OpenAI emerges as a viable fallback option for existing ChatGPT-based solutions and a sensible choice for building a GenAI core in new digital products.

Let’s expand on that and explain our reasoning in the next section.

Azure OpenAI: A Promising Contender

Azure OpenAI’s robust infrastructure, quick scalability, and reliable support make it an ideal solution for healthcare AI apps, even in the face of uncertainties. It’s not merely a backup plan but a strategic move toward diversification. By adopting Azure OpenAI, founders can safeguard their healthcare apps against potential disruptions without compromising on the core value proposition their product delivers.

Why Azure OpenAI Might Be a Better Fit for Your App

Azure OpenAI is essentially ChatGPT housed within Microsoft’s cloud infrastructure. As a tech giant with a proven history in AI and machine learning, Microsoft brings a layer of reliability and trust (99.9% uptime per its SLA) that is crucial for startups to gain the confidence of customers and investors.

Azure OpenAI is not only a powerful alternative, but it also offers a suite of advantages that further enhance its appeal.

• Quick Migration: Migrating from OpenAI to Azure is a breeze, requiring minimal effort.
• Same ChatGPT: With this seamless transition, all the impressive features of ChatGPT are seamlessly transferred to Microsoft’s cloud, including its lightning-fast text embedding model.
• Cloud Integrations: Azure easily integrates with a wide range of cloud services, making it a breeze to scale up your operations when needed.
• More security: By leveraging Azure, you also add an extra layer of security to your applications, benefitting from additional functionalities like Network Isolation and Azure AD-based Authentication/Authorization.
• $ Boon: Moreover, Azure offers generous credits ranging from $1,000 to $150,000 for startup founders, depending on the level of development of their product.
• HIPAA Compliance: Microsoft’s Azure provides HIPAA compliance for those dealing with sensitive PHI data, which may be crucial for your healthcare app (not out of the box, but has the required tooling and guidelines)

Challenges of Switching to Microsoft’s Azure OpenAI Service

While the transition from ChatGPT to Microsoft’s Azure OpenAI service is relatively seamless, it’s crucial to bear in mind a few considerations. These aren’t necessarily challenges but rather key points to be mindful of to ensure a smooth transition and optimal utilization of Azure OpenAI’s capabilities.

• GPT-4 Availability: As of now, Azure OpenAI only offers GPT-4 on a waitlist basis. Thus, by default, you’ll be working with GPT 3.5. If you’re keen on using GPT-4, you must apply and wait for approval.
• Support for GPT4-V, GPT4 FT, and Multimodal: Azure OpenAI’s support for these models may not be as prompt as the native OpenAI ChatGPT implementation. This lag could potentially affect your product’s performance, especially if these are critical aspects of your application.
• Different Responses: Azure OpenAI and the native ChatGPT apply different filters, resulting in varied responses. It’s vital to understand these variances to predict and handle the resulting differences in your app’s performance and user experience.
• Azure Cloud Services Competency: Transitioning to Azure OpenAI requires familiarity with Azure cloud services. If your team lacks this expertise, you may face a learning curve, which could slow down your development process. Consider investing in training or hiring professionals with the relevant skillset to streamline the transition.

Top Open-Source Language Models

In the realm of AI and machine learning, there’s a vast universe of open-source language models that can form a cornerstone of your healthcare application, offering you an array of options to diversify risk and ensure resilience.

1. LLaMA 2: Meta’s LLaMA 2 is an open-source large language model that supports common programming languages and boasts outstanding performance across various benchmarks. It’s freely available for research and commercial use, promoting the democratization of artificial intelligence.
2. MedAlpaca: This model, while still in its infancy, shows promise in the healthcare industry. It’s designed to understand and generate medical language, making it potentially valuable for improving patient care and streamlining administrative tasks. I would thoroughly examine this model as the foundation for a comprehensive medical bot consultant specializing in delivering supplementary information on specific topics.
3. Claud 2: Claud 2 is a robust model known for its impressive understanding of human language and ability to generate accurate, contextually relevant responses. It’s used in various domains, from customer service chatbots to content creation tools. The wait time for access to their model has frequently been a topic of complaint, particularly among founders on Twitter.
4. Nous Hermes models: These models are designed for high-level comprehension and generation of complex narratives. They’re instrumental in areas like storytelling, journalism, and content marketing. So, it is probably a good fit for automating marketing at a hospital or clinic.
5. OpenChat: OpenChat is a versatile language model that excels in conversational AI applications. It’s known for its ability to maintain engaging, coherent dialogues over extended interactions.
6. Zephyr: Zephyr is a powerful model primarily used for translation and transcription services. Its strength lies in accurately converting spoken language into written text and vice versa.
7. Mistral: Mistral shines in the realm of sentiment analysis, capable of understanding subtle nuances in tone and emotion. This makes it an excellent tool for market research and social media monitoring.
8. Phi: Phi is a large language model specializing in technical and academic language. Its strengths include generating precise, well-reasoned content in fields such as science, engineering, and law. So, it may work out for medical research.
9. Vicuna: Vicuna is a versatile model known for its adaptability. It’s capable of learning and adapting to various tasks, making it a useful tool in numerous sectors. This model is exceptionally cost-efficient, delivering 90% of ChatGPT’s capabilities while the entire training process costs just $300.

In addition to these open-source models, Google’s Med-PaLM is worth mentioning. Although at present available only to select Google Cloud customers, it’s being tested in some hospitals and shows promising results in healthcare applications.

Each of these models brings unique strengths to the table, making them valuable tools in their respective domains. Whether you’re looking to improve customer interactions, generate engaging content, or analyze sentiment, there’s likely a large language model that fits your needs.

Read more on Medical Large Language Models in our blog.

Ready for the Next Step?

To sum up, OpenAI’s ChatGPT has opened up vast possibilities in the domain of AI health apps, offering both opportunities and challenges for founders whose products depend solely on this proprietary technology.

In light of recent developments, diversifying to other open-source models and AI tools can help mitigate risks and ensure the sustainability of your product.

At the same time, switching to Microsoft’s Azure OpenAI service seems like a no-brainer and offers a seamless transition for those already using Azure cloud services.

Whichever path you choose, the key takeaway is to be aware of the potential impact on your product and take necessary measures to mitigate any associated risks. With continuous advancements in AI technology, it’s essential to stay updated and informed, leveraging various resources and tools available in the market.

Talk with our experts today to discuss optimal ChatGPT implementation and fallback options to ensure your product continues to deliver its core value proposition, irrespective of individual vendor fortunes.