5 Rules of A/B Testing That Every Marketer Should Know

Wednesday 25th, May 2016

As the online competition is increasing and the customer attention spans are decreasing, it has become mandatory for marketers to be skilled in Conversion Rate Optimization (CRO).

CRO is one of the critical aspects of online marketing strategy since conversion rate is the only metric that correlates with ROI. Even if the ‘conversion’ is not a sales/purchase, the rules of CRO still apply.

While designing landing pages in accordance with the CRO rules, it is totally possible that you get lost in a sea of advice telling to change the color of buttons, optimize the copy and so on.

Unless you’re backing the effectiveness of your landing page with real time user feedback and web analytics, taking a leap of faith by running the landing page without testing isn’t going to bring any success to your campaign.

Now, let’s discuss the rules that should be kept in mind while doing A/B testing.

Rule #1: Forget Any Pre-Conceived Notions about Your Customers

Given that you’re selling a product/service to a particular audience, you might assume your audience’s interests, their age, demographics, income etc. You need to resist the temptation when you should! During the initial days of marketing, customer profiling was the best way to target customers.

But, in today’s digital era we have lot more options to test your audience before launching a campaign. With advanced tools, marketers have the ability to leverage every digital touchpoint as an opportunity to understand the customer’s behavior and preferences.

Rule #2: Test One Parameter at a Time

This rule is self-explanatory, nevertheless worth mentioning since it’s important.

While performing A/B testing on your page, test one variable at a time so that the results can be easily analyzed at the end. For instance, if you’re running a headline test, make sure the headline works with the remaining digital touchpoints through the sales funnel. Consistency is paramount in digital campaigns which builds credibility.

Rule #3: Things don’t work for everyone in the same way

If there was a thumb rule for CRO that worked in the same way for every website, there would no need of testing at all. Marketers would understand how all the e-commerce sites perform and everyone would follow the same rules.

Unfortunately that’s not the case, is it?

Which is why you need to perform A/B testing on your unique content with your target audience. You can, of course, take ideas from the existing set of CRO rules, but you won’t get the same results.

Sample this: Company A sells shoes and Company B sells software applications. Clearly, the buying cycle will be different for both the companies, even if they have common customers. While testing Company A might find changing the color of primary call-to-action button from red to green increases CTR by 70%. But you can’t expect the same results for Company B, can you?

This is the difference. You need to test your pages according to the rules that applies to your industry and audience rather than relying on existing test conditions.

Rule #4: User Testing Is Mandatory

User testing has never been more important than now!

Before going live with your landing page, you need to ensure your page has all the necessary elements required to convert a user, and check if the copy conveys the message obviously and immediately. If you don’t have the luxury of User experience (UX) department on hand, there are a lot of free and low-cost services that offer usability testing.

Some of them are:

Peek User Testing: Peek is a great tool to gather qualitative feedback for your website.

Pros: Feedback is unbiased, detailed, and free!

Cons: Testing an interface outside its intended audience doesn’t make sense. Also it’s difficult to gather large feedback using this method since it is very time consuming.

Amazon Turk: Amazon Turk lets you gather feedback from thousands of people in a short window through quantitative research methods such as surveys.

Pros: Inexpensive, scalable and quantitative and allows you to pre-select qualifying criteria for your testers.

Cons: It is generally performed via a survey engine that can introduce artificial filters.

Any feedback is better than no feedback! Therefore do a thorough user testing before going live.

Rule #5: Define your success metric clearly

Don’t lose sight of your ultimate success metric. CRO is always about conversions and not about open rates, click-through rates, tweets, shares, or pins. Unless, of course, tweeting and pinning is the “conversion” on your website.

Therefore define your goal and optimize your content around that goal. Everything else is a key performance indicator (KPI).


likeLike this article(1)

Subscribe to our Newsletter

We collate information around email opening and clicks, using industry standard technologies, to help us monitor and improve our e-newsletter. Please read our Privacy Policy for more information on how we use your data.



Subscribed successfully Thanks for Subscribing