A beginner’s guide to the A/B testing in event promotion

A/B testing in event promotion is a fantastic tool to make a controlled experiment in your marketing. Figure out the best strategy to improve your performance. What is it and how to take the most out of it? In “A beginner’s guide to the A/B testing in event promotion” you can check what benefits it brings and tips on what you should avoid not to destroy your results with ill-considered decisions.

What is A/B testing?

A/B testing is a controlled experiment with two variants, A and B. It is a form of statistical hypothesis testing and is used to test options with the aim of deciding which variant is the most successful based on the achieved outcome.

Where can I use A/B testing? What can I test?

In event promotion, you have a lot of materials which you can test in a couple of places. In order to achieve that, the first thing you should do, when planning an A/B testing, is finding out what you want to test and improve. Choose the place: is that your website, a Facebook promotional campaign or do you want to improve your email marketing?

Are you running an on-site or an off-site test? You don’t know which version works better for your page visitors? Let your target see two options, track which one converts better, analyze the result and decide – “yes, I’m sending the A version!”. It’s smarter to justify how you spend your money when you are able to forecast the result.

What is worth measuring and testing?

• The title of the email (“Brand new newsletter about event promotion” or “Want to know more about event promotion? Join our newsletter”)
• Length of the message (shorter or longer, more packed with a content message)
• Personalization (Dear Chris or maybe more official Dear Mr. Smith)
• Look of an email (which layout to use)
• Colors on the landing page (catching eye red or calming green)
• Call to action (“Sign up here” or “Check more info here”)
• The specific offer (“Get a free shipping” or “Get 20% discount”)
• Testimonials (include the one from X company, Y or maybe not at all)
• Pop-ups (with the information about e-book or newsletter)
• Image (picture or graphic)
• Target group (which one of your targeted group works better)

How many things should I test?

When you write a post, you most probably have a few dilemmas. Which color of the font is catching attention, but isn’t too aggressive? How many pictures should you include in the text? What should be the title? Should you write a formal message or use more personal and casual approach? And all these things work differently with each other, what gives you many combinations in an A/B testing.

There are two approaches to answer those questions:

1. Imagine that you want to change the color, heading and the call to action on your website. You change everything at the same time and… Well, your conversion rate is decreasing. Now you don’t know which change made it worse and which is worth implementing. You should make a few tests, let’s say that the first one is checking 2 options of the call to action and then you can choose the better performing one. The next step is the heading and so on. Do it step by step and you will achieve clear results.
2. The second approach is to take into consideration how many options of one change you can test at the same time. As the name says A/B testing is giving you a possibility to check 2 variants. Why not A/B/C testing? If you have to check 3 options it is better to split your visitors into three groups instead of two, and it could likely still be considered an A/B test. This is more efficient than running three separate tests (A vs. B, B vs. C, and A vs. C). You just need to give your test an extra couple of days to run, so that you still have enough results to base conclusions on. So A/B testing with 2 variants is better for you when you want to get faster results, while A/B/C testing requires a little more time to get conclusions.

Multivariate testing is a technique for multiple variables modified. The main goal is to test a combination of options to decide which performs the best of the all possible. This test will change a picture and the color of a background: 2 pictures and 3 colors give you 6 versions to compete.

The number of variations is:
[Number of Variations on Element A] x [Number of Variations on Element B] x … = [Total Number of Variations]

The biggest limitation of multivariate testing is the amount of traffic needed to complete the test, since more people need to check your test in order to get valuable data. Remember not to use too much of your contact base for testing or you won’t have anybody left to actually run the best version.

A/B Testing Process

If your A/B test is effective, the result will show you the path to raise the visibility of your event and brand. To achieve that follow the 6 steps:

1. Decide what you want to measure
   The first step is deciding what is important to measure. Analytics will provide you with an insight into where you should begin optimizing. In order to notice the need for changes, check the pages or posts with low conversation rates or a lot of drop-offs. You need to understand what works for your customers. Google Analytics or a simple survey or interview can come with help.
2. Define your goals
   Goals can be anything, choose you KPI, starting from clicks on a button, link in an email to go to your store, to the bought products from your link.
3. Work on hypotheses
   Do you know your goals? Now think about hypotheses for why you think they will be better than the current version. Prioritize the list of ideas depending on the expected feedback and possibility of implementation.
4. Create variations
   Now, when you have your ideas it’s time to bring them to life. Using a software prepare variations for your website, fanpage or email. A change might be the order of elements on your menu, a color of the button or different pictures. Many tools are letting you easily create variations for your A/B test.
5. Run experiment
   So let the game begin! Wait for your visitors to participate. Now you will have randomly assigned users from your target group. Run your test for at least a few days to see the real outcome. Before you end your test, make sure that you managed to get a significant number of feedback and reached the statistically significant target group. Every interaction of your visitors is being counted and measured, compared and determined how each of them performs.
6. Analyze the outcome
   Your experiment is over, so now it’s time for the final step. After all the work and patience during the process, the A/B testing software will present you the results. Now you see whether there is a significant difference between two versions’ performance and how big it is. If a test ran in a proper way, you have got to know something new about your customers, have a nice revenue gain. So now, how about the next test?

What tools can I use?

You can check how you can test your Facebook ads and its version in Campaign Manager. By a split testing tool, you can check and schedule your posts with the best graphics, check their length or target.

There are tools like Optimize in Google Analytics Solutions which can tell you which variant performs the best.

Another tool, named as one of the leading, is Optimizely. It gives you the possibility to run a user-friendly test without any technical knowledge.

VWO – friendly platform for small businesses and smaller budgets.

If you are using email campaign software, most of them have built-in tools for A/B testing. Good examples are Campaign Monitor, MailChimp and Active Campaign. You can also do it manually by splitting your list into 2 separate ones, however, it is then harder to collect data and analyze.

What is the outcome?

The point of A/B testing is having a measurable outcome after you implemented changes. When you decide what’s your goal it is time to analyze, for example, the number of sales made, the number of people signed for a newsletter or registered for an event, click-through rate, conversion, email open or response rate, etc. Going back to your target group, it needs to be big enough to show meaningful results and differences between 2 variations. You can decide which one makes bigger impact on the performance of your brand.

When you see the results, you simply choose the campaign with the best performance and implement it.

• Don’t trust only your gut feeling, use collected data.
• Test only one variable at a time for the best results. (Not satisfying? Then check multivariate testing instead of A/B testing)
• Test in advance to achieve the best results and implement changes on time.
• Run your tests at the same time to achieve trustworthy results.
• Test and choose the right available tools for more structured and easier A/B testing.
• Run your tests on your clients as a target group to make the results relevant.