At TopFlight Apps, we pride ourselves in always challenging our assumptions to help our clients find product market fit. We have a culture of iterating with the user at the center of our design– we iterate, test, iterate again until we hit the Key Performance Indicators (KPIs) that we use to determine whether a product has a good chance of becoming successful in a real-world setting.
Reason for doing: We wrote this guide to get your feet wet and demystify testing to the point where you can start applying it. Our team believes we’re more successful at our jobs when tons of people methodically create products and succeed as a result. Without further ado, here is our Introductory Guide to Increasing Your ROI With App User Testing:
Start With High-Level Goals
Setting goals is the most crucial part of our process. We align with our clients on what the goal(s) of their product are and then determine what the best methods are to test towards that.
Here’s why goal setting is such an important first step:
Client A may be building an app that helps users find the perfect used car. For this exercise let’s call it LemonAid:
Client B may be building an internal web app that helps existing employees visualize data. For this exercise let’s call it HealthDash:
Now, the definition of a successful product would differ between LemonAid and Healthdash. LemonAid might say that success to them equals users being able to discover their ideal cars in the shortest amount of time. HealthDash might say that success to them equals users being able to comprehend data more quickly.
Set App User Testing Methodologies
Now we can start setting the optimal testing methodologies to drive towards our goals. This is where we put on our lab coats and get rather scientific.
First, let’s break down the two major categories of user testing we do: remote testing and in-person testing.
The main advantage to remote testing is that you can get your design prototype in the hands of a lot of people very efficiently (using platforms like usabilityhub), and these platforms will provide not only user responses but statistics that you can use to quantitatively demonstrate KPIs being hit. These statistics make it easy to measure how a large group of users respond, which can be instrumental in validating that the app will or won’t do well once it’s launched.
The main advantage to in-person testing is you can see the tester use the product from end to end, which can reveal qualitative insights that data from any platform simply cannot (at least not very efficiently). The depth of insights from 5 in-person tests can sometimes trump data from 100 remote tests.
For optimal testing results, we aim to do both whenever possible.
The 4 Main Types of User Tests We Do
Whether we’re in-person or remote, here are the 4 main types of tests we run:
1) 5 Second Test
A 5 second test can allow you to test if user’s first impressions match your assumptions. It is believed that once users get past 5 seconds of observation, they start thinking more like “designers” and less like users.
LemonAid might expect that after seeing a screen for 5 sec, most users would remember what they do, or remember the precise location of the Call To Action.
HealthDash might expect after seeing a screen for 5 sec, most users would remember what type of data is being displayed in the graph.
Tell me moreWe'll provide insight into what tech stack and APIs work best for your app idea.
Real-Life Example: Here is an actual 5-second test run on usabilityhub to see if users remember what we offer. We might set the KPI here as 80% of users remembering it’s a software product.
2) First Click Test
A first click test, like the 5 second test, is also meant to test users’ first impressions. Unlike the 5 second test, a first click test is focused on where users, well, click first and how long it takes them to click (which would indicate how quickly they understand what they’re supposed to do). These findings allow you to assess potential conversion rates for CTAs, or the ease of doing something important very quickly (like adding to cart).
LemonAid might expect most users to click first on the “view 360” button on the car details screen, as opposed to the back button.
HealthDash might expect most users to click first on the lines within the line graphs, as opposed to the dropdown showing other types of data that could be displayed.
Real-Life Example: Here is a heatmap captured by usabilityhub that shows whether our expectations match reality. We might set the KPI here as 80% of users clicking on the desired icon.
3) Navigation Based Test
For longer, more complex flows, navigation-based, or task-based tests are excellent at identifying where you are losing users in the funnel (do users understand how to do what you need them to do?) and how long users take to complete a single user story.
In the case of LemonAid, the form of testing an entire onboarding process where users would enter demographic info and preferences is needed, in order to arrive at the page where they start seeing suggested cars.
For HealthDash, this might take the form of testing the process of toggling data on the graph to view revenue over the last 30 days, finding the total amount for a specific date, and then printing out a summary of all invoices for that specific date.
Real-Life Example: Here is a screenshot of an actual test that we ran in usabilityhub for an onboarding process for a client’s healthcare app. We ended up setting the KPI here as completion rate and time to completion.Using this test, we started out with a baseline of 70% completion and 1.5 min time to completion. We continued iterating on this test until we reached a 80% completion rate and 1 min time to completion.As you can see, user testing allowed our team to iterate toward our “measure of success” and tackle one of the hardest parts of building a healthcare app: engaging users towards completion of data-heavy questionnaires.
4) Preference Testing
Preference testing is considered a test for UI (user interface) as well as UX (user experience). This test shows you whether users have a preference for a particular design. This test should be with a grain of salt for UX because, just because a user enjoys a design does not mean that they’ll engage with them. (That’s why combining this with the previous tests is so critical for UX)
However, we’ve found that if a specific design is a core part of the app where you expect them to spend a disproportionately long amount of time over other features in the app, a preference for the design goes beyond aesthetic appeal and into usability.
For LemonAid, this might involve testing two car finding features and seeing which one users prefer, and why.
For HealthDash this might take the form of two color schemes of the graph and seeing which one users prefer, and why.
Real-Life Example: Here is a screenshot of an actual preference test that we ran in usabilityhub for a spinning wheel of the day: We weren’t sure if users would prefer seeing the whole wheel or half the wheel to visualize the text more clearly. So we tested it and found that almost 80% of the users preferred the whole wheel. Through our qualitative questionnaire, we also learned that users, against our expectations, preferred the whole wheel largely because of readability. The team internally was split on which wheel was better, and by conducting the test, we quickly settled it without a statistical doubt.
And there you have it. This was an introductory guide to how we do app user testing here at Topflight to challenge and, oftentimes, upend our assumptions. This was by no means an exhaustive list of the types of tests you can do, but just by using these 4 tests you should be able to start drawing some conclusions.
Testing is extremely important because as designers, entrepreneurs, and developers, we are naturally biased by our beliefs and our attachments to our own creations. Yes, we have to start with our personal experience and our intuition to create our first iteration of design, but half of our job as product designers is about being humbled by data, iterating, and iterating again until we hit those KPIs and find a winning product for our clients.
Looking to get started today? At Topflight Apps, we are ready to help you streamline an existing app, or even elevate an idea that’s been floating in your head for a while, and we do it all using a data-driven approach. We’re an excellent team of developer-founders who boast a record for superior customer service, and a 100% product launch success rate, and we’re always looking for inspiration for new projects.
Request a proposal, and we can get the ball rolling before tomorrow’s emails start pouring in.