In the realm of healthcare technology, setting up the perfect architecture for an AI health app right from the start is nearly impossible. The landscape is ever-changing, with evolving requirements and new developments continuously emerging.
Often, developers must work with pre-existing setups or rush to test their ideas to validate them as quickly as possible. This agile approach can delay refactoring efforts since the immediate need is to demonstrate results rather than perfect the underlying architecture from day one.
This was precisely the case with GaleAI, an AI-driven medical coding platform designed to simplify billing for medical organizations. GaleAI leverages a suite of sophisticated AI and machine learning algorithms to streamline the billing process, making it more efficient and accurate.
However, like many innovative solutions, GaleAI had its share of growing pains, especially when it came to performance issues that significantly impacted user experience.
Here’s how we helped GaleAI boost its performance by nearly 10x.
NB This tech-focused blog will be particularly valuable for product managers, CTOs, or VPs of Technology who are navigating growth challenges in healthcare software platforms.
Top Takeaways:
- Founders’ Insight: Optimal architectures aren’t always achievable at the start. Initial over-optimization for scaling can necessitate further changes to enable faster growth later on.
- Effective Growth Strategy: Partner with proactive healthcare developers who offer valuable insights and implementation expertise, ensuring your product is well-positioned in the market.
Table of Contents:
- The Challenge: Overcoming Initial Performance Hurdles
- Diagnosing the Performance Bottlenecks
- The Solution: Transforming GaleAI with a Node.js Backend
- The Implementation Process: Seamlessly Migrating to a Node.js Backend
- The Results: Quantitative and Qualitative Gains
- Broader Implications: Positioning GaleAI for Success
- Insights and Future Directions: Moving Forward with Confidence
The Challenge: Overcoming Initial Performance Hurdles
GaleAI, an AI-driven medical coding and billing platform, was designed to bring efficiency and accuracy to a traditionally cumbersome process. However, adding new features led to a noticeable slowdown in user experience, signaling the need for a closer look at the evolving architecture.
Importance of Speed in HealthTech
- Critical for Workflow: Delays of even a few seconds can disrupt healthcare providers during patient consultations.
- User Impact: Some operations, like looking up medical codes, took up to 10 seconds—unacceptable for modern applications.
Initial Performance Issues
- MVP Nature: Early development prioritized functionality over-optimization, leading to performance bottlenecks.
- AWS Lambda Functions: Initially chosen for scalability and serverless architecture, but they introduced latency problems.
Impact on User Experience
- Frustrated Users: Significant lag in critical operations caused inefficiencies and errors.
- Cold Start Issue: Lambda functions required extra time to initialize, exacerbating delays.
- Complex Architecture: Multiple AI engines and individual database connections for each function added to the problem.
These challenges highlighted the need for a comprehensive overhaul to enhance GaleAI’s performance and user satisfaction.
Diagnosing the Performance Bottlenecks
When tackling performance issues, understanding the underlying technology choices and their implications is crucial. For GaleAI, the decision to use AWS Lambda functions initially seemed logical, given their promise of scalability and a serverless architecture. However, as we delved deeper, it became clear that these very features were contributing to the sluggish user experience. Let’s explore why.
What Are AWS Lambda Functions?
AWS Lambda is a serverless computing service that allows developers to run code without provisioning or managing servers. Essentially, the code runs in response to specific events and can automatically scale from a few requests per day to thousands per second. This elasticity makes Lambda an attractive option for applications expected to handle varying loads.
The Initial Choice: A Seemingly Perfect Fit
In the early stages of GaleAI’s development, we needed a solution that could quickly scale with demand, integrate easily with existing technologies, and support rapid iteration. Lambda functions fit the bill perfectly:
- Scalability: Automatically adjusts to fluctuations in workload.
- Cost Efficiency: Charges only for the compute time used.
- Ease of Deployment: No need to manage servers or infrastructure.
Given these benefits, Lambda appeared to be an ideal choice for handling GaleAI’s backend operations. However, this decision came with several trade-offs that gradually surfaced as the platform grew.
The Cold Start Issue
One of the primary drawbacks of using Lambda functions is the cold start issue. When a Lambda function hasn’t been invoked for a while, it needs extra time to initialize before executing the request. This cold start latency can range from hundreds of milliseconds to several seconds, depending on various factors such as the runtime environment and function complexity.
For GaleAI, the cold start times were particularly detrimental. Operations like reading, saving, and updating notes or looking up medical codes, which are central to the user experience, were often delayed by these cold starts. Users faced spinning loaders and delays of 2 to 10 seconds, which is far from acceptable for a modern application.
Complex Architecture and Database Connections
GaleAI’s architecture involves several AI engines and over 100 Lambda functions, each responsible for different aspects of the platform. While this modular approach offered flexibility, it also introduced significant complexity. Each Lambda function needed to establish its own database connection every time it was invoked. Here’s why that became a problem:
- Initialization Delay: Establishing a new database connection for each request added extra latency.
- Resource Overhead: Constantly opening and closing connections consumed additional resources.
- Network Latency: Depending on network conditions, establishing a new connection could add 30 to 300 milliseconds to each request.
Upgrading Resources: A Minimal Impact
To address the performance issues, we initially tried upgrading the resources allocated to the Lambda functions and the database. While this improved performance marginally, it did not resolve the core problems. The improvements were minimal because the issue was not merely about having more power but about how effectively that power was being utilized.
The cold start delays and frequent database connections persisted despite the resource upgrade. It became evident that a more fundamental change was necessary to achieve the performance improvements we sought.
The Need for Change
Recognizing these challenges, we realized that continuing with the current architecture would not yield significant performance gains. The Lambda functions, although powerful, were not suited to the high-frequency, low-latency operations required by GaleAI. The complexities of maintaining multiple functions and dealing with their individual quirks also added to the development overhead.
The Solution: Transforming GaleAI with a Node.js Backend
When faced with performance issues, it’s easy to get bogged down in quick fixes. However, we understood that GaleAI’s sluggish performance needed a fundamental shift in its architecture to deliver the seamless user experience our clients expect. After a thorough analysis, we identified switching from AWS Lambda functions to a Node.js backend as the optimal solution.
Why Node.js?
Why choose Node.js over continuing with Lambda functions? Let’s explore some compelling reasons:
- Continuous Running: Unlike Lambda functions, which can suffer from cold start delays when they haven’t been invoked recently, a Node.js server runs continuously. This ensures that requests are processed without initialization delays, providing a much faster response time.
- Efficient Database Connections: Each Lambda function creates a new database connection when invoked, adding overhead and potential bottlenecks. Node.js, on the other hand, maintains persistent connections through connection pooling, reducing latency and improving performance.
- Scalability and Maintainability: Node.js provides a robust environment for efficiently handling multiple simultaneous connections. Migrating to a Node.js backend simplified our architecture, making it easier to maintain and scale: we already handled some critical back-end operations like PDF file generation there.
Key Performance Improvements
Our transition to a Node.js backend brought several key improvements that significantly enhanced GaleAI’s performance:
1. Optimized Database Resources
First, we increased the database resources specifically for the production environment (by reoptimizing development and staging environments) to ensure that it could handle more connections and process requests more efficiently, reducing bottlenecks.
2. Efficient Database Connection Pooling
One major issue with Lambda functions was the creation of a new database connection for each function invocation. In high-traffic situations, this could quickly exhaust the database’s connection limit, leading to failed requests. By moving to a Node.js backend, we implemented efficient connection pooling. This allowed us to manage database connections more effectively, reusing them rather than creating new ones for each request. Consequently, the speed and reliability of database interactions improved significantly.
3. Eliminating Cold Start Delays
Lambda functions often suffer from “cold start” delays, particularly when they haven’t been used for a while. These delays can slow down response times considerably. By migrating to a continuously running Node.js server, we eliminated these cold start issues. The server remains active, ensuring that it can immediately respond to incoming requests without the additional startup time.
4. Query Optimization
While implementing the Node.js backend, we also took the opportunity to optimize our database queries. For complex requests, we combined multiple queries into single, more efficient ones and ensured that database indexes were used properly. These changes not only reduced the number of queries but also enhanced the overall query performance.
The Implementation Process: Seamlessly Migrating to a Node.js Backend
Here’s a detailed look at how we made this transformation happen.
Gradual Transition for Seamless Transformation
Rather than taking an all-or-nothing approach, we opted for a gradual transition. This method allowed us to maintain service continuity and ensure stability while systematically migrating critical components.
Calculating Before-and-After Performance
Before initiating the migration, we conducted comprehensive performance assessments to establish baseline metrics. These metrics served as benchmarks against which we could measure improvements post-migration. By calculating before-and-after performance, we ensured that each step of the transition provided tangible benefits.
Running Lambda and Node.js in Parallel
During the initial phase of migration, we ran both the Lambda functions and the Node.js backend in parallel. This dual-operation setup offered several advantages:
- Customer Convenience: It allowed customers to update their mobile apps gradually, reducing the risk of downtime or service disruptions.
- Failsafe Mechanism: Running both systems concurrently served as a failsafe, ensuring that if any issues arose with the Node.js backend, we could revert to the Lambda functions without significant impact on users.
How the Migration Was Done
Migrating GaleAI’s backend from Lambda functions to Node.js involved several critical steps. Each step was meticulously planned and executed to ensure a smooth transition.
Setting Up Authentication
The first order of business was to move the authentication code from the Lambda functions to the Node.js server. We created middleware to handle user authentication for the API endpoints. Middleware acts as a filter, ensuring that only authenticated users can access certain parts of the application.
- Authentication Middleware: This component checks user credentials and session validity before allowing access to the backend services.
- Security Enhancements: Moving to Node.js allowed us to implement additional security measures, enhancing overall user data protection.
Connecting to the Database
Next, we adapted the database connection logic for the Node.js server. This required only minor adjustments, as the core functionality remained consistent.
- Persistent Connections: Unlike Lambda functions that establish a new connection for each request, Node.js maintains persistent connections through connection pooling. This significantly reduced latency.
- Optimized Queries: We also optimized our database queries to ensure that they were efficient and leveraged indexing properly.
Handling Requests and Responses
Migrating request and response handling to the Node.js environment involved adapting the way the code reads incoming requests and sends responses.
- Request Parsing: We ensured that the parsing of HTTP requests in Node.js was consistent with how it was handled in Lambda.
- Response Formatting: Adjustments were made to how responses were formatted and sent back to the client, ensuring compatibility and consistency.
Migrating Utilities and Configurations
Finally, we moved utility functions, models, and configuration variables used by the Lambda functions into the Node.js project. This involved:
- Code Refactoring: Minor refactoring to ensure that the utilities and configurations were compatible with the Node.js environment.
- Config Management: Centralizing configuration management within the Node.js server for easier maintenance and updates.
Challenges Faced and Overcome
No migration is without its challenges. Here are a few hurdles we encountered and how we overcame them:
- Concurrency Management: Ensuring that the Node.js server could handle concurrent requests efficiently required fine-tuning the event loop and optimizing resource allocation.
- Compatibility Issues: Some legacy code from the Lambda functions needed to be rewritten to be compatible with the Node.js environment. This was addressed through thorough testing and iterative development.
- Performance Monitoring: Continuous monitoring during the migration helped us identify and resolve performance bottlenecks in real time.
The Results: Quantitative and Qualitative Gains
After migrating GaleAI’s backend from AWS Lambda functions to Node.js, the improvements were substantial and measurable. Here’s a detailed look at how our efforts paid off:
Quantitative Improvements
- 90% Speed Improvement: Our metrics showed a significant decrease in response times. Previously, operations like looking up medical codes in some use cases could take up to 10 seconds. Post-migration, these tasks were completed in under a second.
- Reduced Latency: Eliminating cold start delays and optimizing database connections led to smoother and more consistent performance across the board.
- Increased Throughput: The number of concurrent users that GaleAI could handle increased significantly, enabling the platform to scale effectively with growing demand.
Sidenote: While Lambda functions could theoretically handle more users, our lack of proper database pooling meant each function created a new database connection. Switching to a Node.JS backend has significantly increased our capacity to manage users efficiently.
Qualitative Enhancements
- Enhanced User Experience: Users reported a much smoother interaction with GaleAI. The app felt 10x faster due to the architectural changes and UX updates like optimized loading indicators and spinning wheels, which provided immediate feedback to users.
- Smoother Operations: The persistent database connections and reduced latency contributed to a more reliable and stable platform, minimizing disruptions during critical operations.
By addressing both the technical and user experience aspects, we have transformed GaleAI into a robust and responsive tool that meets the high standards of the healthcare industry.
Broader Implications: Positioning GaleAI for Success
The enhancements to GaleAI’s architecture do more than just improve performance; they position the platform more competitively in the market and offer tangible benefits to users and clients.
Benefits for Users and Clients
- Improved Efficiency: Faster response times translate directly into higher productivity for medical professionals who rely on GaleAI for quick and accurate medical coding and billing.
- Reliability: With a more stable backend, users can trust GaleAI to perform consistently, even under heavy load, reducing the risk of downtime or slowdowns during peak times.
Competitive Edge
- Market Differentiation: By overcoming the initial performance hurdles, GaleAI now stands out in the crowded healthcare tech market as a high-performance, reliable solution.
- Client Confidence: Demonstrating our ability to proactively identify and resolve issues reinforces client confidence in Topflight’s capability to deliver cutting-edge technology solutions.
These broader implications underscore the strategic importance of our architectural improvements, bolstering GaleAI’s reputation and market position.
Insights and Future Directions: Moving Forward with Confidence
Reflecting on the journey, several key lessons and future steps emerge, offering valuable insights for GaleAI and other healthcare tech companies.
Key Lessons Learned
- Architectural Choices Matter: Selecting the right architecture from the start is crucial, but it’s often challenging—if not impossible—when working on an MVP. Flexibility and readiness to pivot are essential.
- Proactive Optimization: Continuous monitoring and optimization are vital to maintaining high performance and user satisfaction. Addressing issues before they become critical ensures sustained success.
Future Optimizations
- Further Enhancements: We plan to continue optimizing GaleAI by exploring advanced caching strategies and further refining database queries to eke out additional performance gains.
- User Feedback Integration: Actively soliciting and integrating user feedback will remain a priority to ensure that GaleAI evolves in line with user needs and expectations.
Advice for Healthcare Tech Companies
- Prioritize Scalability: Design your architecture with scalability in mind, even when developing an MVP. This approach minimizes the need for drastic changes as your user base grows.
- Invest in Performance Monitoring: Implement robust monitoring tools to track performance metrics continuously. Early detection of issues allows for timely interventions.
- Engage Experts: Partnering with knowledgeable IT firms like Topflight can provide the expertise needed to navigate complex architectural challenges and optimize platform performance.
If you want to learn more about optimizing health tech solutions or seeking personalized advice, reach out to us at Topflight. We’re here to help you transform your ideas into powerful, high-performing applications.