Split Testing in WordPress - The

I have to admit that lately I’ve been doing significantly less split testing than I used to do in the past. My bad…but why am I telling you this? Quite simply because split testing is the best method of improving our websites over time. And I do mean THE best.

Let me tell you what the problem is. Basically, whenever we’re building a site, we are the only people who decide what should be featured on the site and in what manner. Despite that we might all be professionals, we still have a very limited possibility of accurately predicting what exactly will guarantee the best results on the new site. And that’s where split testing comes into play.

In short, split testing is your tool for finding out which solutions work best for your specific site, based on some real-world stats and user participation data.

And when I say “solutions,” what I mean is every element that your site is comprised of. For example: the theme, plugins, design structure itself, headlines, layout, color scheme, and so on.

The traditional way of performing a split test on a website involved signing up to Google Website Optimizer, generating some split test code, setting experiments and including them manually on the site. It took some time.

Now, since we have WordPress, things are much easier. Let’s start with the basics.

How split testing works

Regardless of the tool you’re going to use, the split test itself will be performed in a similar manner.

Split testing is also referred to as A/B testing. That’s because in its standard form, split testing is about running two alternative versions of something at the same time and then checking the stats to see which version is performing better.

Although some webmasters decide to run more than two versions at a time, it’s a much more labor-heavy structure and usually lengthens the whole process a lot. If you’re testing just two solutions then you can quickly get the required amount of traffic to make the test statistically significant.

Once you determine which version is the winner, you can scrap the lower performing version, create a new one, and restart the test all over again. Now just repeat until you get the absolute best performing version of your test subject.

That’s essentially it. Split testing theory in a nutshell. Next, the practice.

Split testing tools

You obviously need some specific tool that will run the test for you (display two versions at the same time, gather the stats, present the results).

For WordPress, there are only two sensible solutions: external tools like the Google Content Experiments module (can be found in Google Analytics – section Content > Experiments) or one of the available plugins delivering custom solutions (self-hosted on your server).

Depending on what you actually want to test and how WordPress-specific your test is, one can be better than the other.

Let’s start with the big player…

Using Google Content Experiments in WordPress

Content Experiments (CE) is the new version of the tool formally known as Google Website Optimizer. Now, it’s integrated into Analytics, which makes it much easier to use.

Here’s how a standard test in CE plays out:

  1. Create a certain goal in your Analytics that you want to improve. It can be connected to things like the number of clicks for a particular button, some simple traffic stats or basically anything else Analytics allows you to do.
  2. Create two separate versions of the page you want to test. Version #1, which we’ll call the original and version #2, which is the variation. Both versions should be available under individual URLs. Again, that’s two separate pages each with its own URL.
  3. Create a new experiment in CE and assign the original and the variation. Also, pick the goal you want to improve.
  4. You’ll get a piece of code that you need to include somewhere in your theme.
  5. CE will take care of displaying both versions simultaneously on your behalf. No other code modifications needed.

Step #4 is where a plugin that’s simply called Google Content Experiments comes handy. It allows you to include the code in the WP Admin as opposed to having to do this manually.

All you have to do now is wait for the stats to come in and then choose the winning version. As always with Google, you get clear stats that you can interpret in hundreds of different ways.

Using Optimizely in WordPress

Optimizely is another external tool for split testing. It’s quite similar to CE, but a bit easier to use in my opinion. It comes with a price tag though.

Basically, when you go to Optimizely.com and create an account, you get to use their WYSIWYG page editor to create a variation of any page on your site. You can change anything from images, text, icons, form fields to the layout…you name it.

Then, Optimizely gives you a simple JS code that you have to include in your theme. There’s a plugin to make this more streamlined (just like with CE).

Doing more in-depth tests

Just changing a piece of content here and there might not be enough for some advanced split testing. That’s why I want to present you with two more plugins that go a little deeper into the whole idea.

First, we have landing page building and then testing.

WordPress Landing Pages is quite a functional plugin that allows you to build a set of custom landing pages and then run them against each other featuring full split testing ability.

What’s even more interesting, you don’t have to use your basic site design for that. The plugin comes with some pre-made templates for custom landing pages and you can even run tests with inactive themes from your wp-content/themes folder.

Very cool for testing all sorts of sales pages, email signup pages, or affiliate reviews/sales.

(Just a note. There are some in-plugin purchases. So not every link you click after getting the plugin will lead to free functionality.)

SES Theme Split Test is the final plugin on the list and it’s my plugin of choice for testing whole new designs and themes.

The plugin uses Google Analytics for gathering and managing stats. To get started, set some new goals in your Analytics and then let the plugin take care of the rest (running two themes simultaneously).

A plugin like this can be very handy as it allows you to act fast in case your new design is not bringing in the results you’ve hoped for. If that happens, you can always stop the test with one click and go back to the previous design/theme.

How to split test properly

Again, this is just my personal opinion, but I’d say that split testing more than one thing at a time is too difficult to manage effectively. For example, if you want to test two designs along with two headlines placed in four different places then you’re probably overkilling it. For starters, just one test subject is really enough.

As it turns out, the actual best thing about split testing is that you will witness improvements regardless of the current size or the popularity of your site. For instance, even if you’re getting only 2 newsletter signups a day, improving it to 3 with split testing is still possible and it means 50% increase. The numbers can add up quickly. And if you’re starting with 200 instead of 2 then it’s a whole different story.

Okay, enough of me talking. What’s your experience with split testing in WordPress? Is it a regular part of your webmaster’s routine?

P.S. If you’re interested in WordPress Tests, check out this article to find even more about WP Experiments and their impact: A/B Testing for your WordPress website – tools and resources.

back to top