Joe Tuan
Joe Tuan
Founder, Topflight Apps
January 23, 2018

Merriam-Webster defines an entrepreneur as, “a person who starts a business and is willing to risk loss in order to make money.”

It further goes on to clarify that the word “carries an additional connotation of far-sightedness and innovation.

There it is folks…it’s in the books. We’re risky; we’re innovative; we think on our feet and fly by the seat of our pants. It’s all great and wonderful. But while these qualities inherently make up who we are and have led us to come up with that awesome idea or quit our 9 to 5 to design that awesome app, we cannot overlook the words business and that we are striving to (at least eventually, someday, somehow, hopefully) make money.

If money isn’t your metric for embarking on this entrepreneurial adventure, insert: gain 25K followers, get your music to as many listeners as possible, help provide clean drinking water to children in Uganda, or whatever else is. You’ve got a lofty goal and you’re taking on financial risks to reach it.

It is therefore important to measure the results of the actions you take and react accordingly. This is the difference between a true businessman (or woman) and that (f)unemployed friend living in their parent’s basement, playing video-games all day and occasionally making a business-y phone call here or there, in the name of “entrepreneurship.” Okay, okay…there are many differences…not all that include tracking your results. But you get what I’m saying. You should track your results.

This is where A/B testing is a necessity. Otherwise known as split testing, A/B testing is, quite simply, comparing two variants to determine which performs better. It is an evidence based practice that can be utilized continuously on almost anything.

Just so we’re all on the same page, imagine a clothing website in which the “Add to Shopping Cart” button is above the product image in one version of the site and below it in another version (of the same exact site). Performing A/B testing, we would share each version of the site with an even and unique set of shoppers to see which button received more clicks, and ultimately, which version achieved more sales. To clarify further, we’re identifying changes that increase the maximum outcome of interest (i.e. click-through-rate with an advertisement, downloads with an app, or purchases on a website). It’s measurable. It’s scientific. And I’ve found it especially useful in web design, user experience design, and marketing.

Improvements can be seen in testing elements such as layouts, images, color, text, target audience, and more. In addition, now that we spend so much of our lives online or on our smartphones, most apps and marketing automation software provide consistent measurable data you can track.

Now that you understand what an A/B test is, how do you best organize one and what do you do with the data once you have? Below are a few tips to optimize your test:

1. You must test one variable at a time

This is critical in order to not skew your results. Expanding upon the example above, if you placed the “Add to Shopping Cart” button above the image in the first version of your website, and below the image in the second version of your website, but also changed the product’s image in the second version, you will be unable to determine what actually resulted in either more or less sales (the edgy, side-angle photo of those cool, new ankle-booties or the location of the shopping cart button). To reiterate, the two versions (of whatever you’re testing), must be identical except for the one variant that might affect behavior.

2. Set a hypothesis (Ad x with the pink background will achieve more app installs than Ad y with the green background), conduct an experiment (your A/B test), and analyze your results (hypothesis denied as Ad y actually produced more app installs)

Once again (see my last blog post on putting the cart before the horse), I’m brought back to the scientific method we all learned in the fifth grade. This is statistical hypothesis testing; a controlled experiment with two variants (A and B). Keep in mind, that we want to craft a hypothesis that is measurable (i.e. targeting “foodies” in my Facebook ad will result in more conversions than targeting “food lovers”). You can get ultra-specific as I just did there or leave it more broad (i.e. my click-through-rate will increase if I target men in the United States vs. when I target women in the United States). As with many things, the more specific you get, the more clear you will be on who is your target audience and with your results in general.

3. Determine your outcome of interest and then decide which variable to test

As expressed in tip #1, you must test one variable only. I imagine this might feel difficult to narrow down. That being said, once you decide whether your outcome of interest is traffic, views, app installs, lead generation, conversions, sales, etc. (the list goes on) you will then have an easier time determining which (of many) variables to test. If you decide that your outcome of interest is YouTube views you may test the cover image of your YouTube video as opposed to the description text underneath the video that directs to your website.

4. Consider a site such as Visual Website Optimizer or “Split Testing” in Facebook Ads Manager

Obviously, A/B testing is such a broad tool and so the following suggestions may not work for your needs, but sites such as Visual Website Optimizer and the “Split Testing” option within Facebook Ads Manager makes A/B testing easy. With Facebook, you set a campaign (3–14 days is ideal) and an appropriate budget (ensuring sufficient data) and Ads Manager provides you with the “winner” based on the web analytics and your test’s results.

5. Don’t introduce changes while the test is running

It might be tempting to alter your test, especially if you aren’t seeing the results that you were hoping for, but please wait until after the test is complete to analyze its results and make changes. Otherwise, your test is less meaningful and your data, essentially, is made useless.

Ultimately, when done correctly, A/B testing is a tool that can help you make more informed business (read: financial) decisions. It is especially useful in determining ad strategy and design for your next marketing campaign or deciding upon the design for your website or mobile application. User experience and behavior is key to the success of a company. Tracking analytics has never been easier and analyzing those results can help you react accordingly, allowing you to reach your goals with more clarity and ease. Hopefully, A/B testing is now a tool you may add to your box that will help you narrow down what is best for you and your business in the future.

Related Articles:

  1. App Store Optimization Guide
  2. Mobile App Monetization Guide
  3. Mobile App Metrics KPIs
  4. How to Develop an MVP
  5. A Guide to Redesigning a Mobile App
  6. Ultimate Guide to Mobile App Design
Joe Tuan

Founder, Topflight Apps
Founder of Topflight Apps. We built apps that raised $165M+ till date. On a mission to fast-forward human progress by decentralizing healthcare and fintech.
Learn how to build winning apps.

Privacy Policy: We hate spam and promise to keep your email address safe

Copy link