Unexpected Benefits of A/B Testing

I am currently beta testing a new website called Optimizely – a super simple A/B testing product… and it’s gotten me thinking.

A/B testing is a process that allows you to optimize your website for your users. You create one or more variations of a page and then measure and compare the interactions on the new page against the old.

The bit that got me thinking was creating the variations.

When you design the tests you have the option to make any changes you like. Rather than make small tweaks and other simple edits I decided to make larger changes that alter the structure of the page entirely.

The part that interested me most was when I was trying to work out what to change. Thinking of it as an experiment I was able to clear my mind of all preconceptions and look at the site objectively. I was able to ask myself questions like:

  • what would I do if I was visiting the site?
  • what would encourage me to stick around longer?
  • what would make me click that link?

And instead of trying to put myself in my visitors shoes, I became the visitor. The distinction is subtle’ but also important.

The problem I had before is that I was looking at the problems from the wrong perspective. I was trying to guess what the site visitor was trying to do, when instead I should have considered what I would do if I was the site visitor.

Previously I would have looked at the site and said:

so and so web design blog said making the button pink will increase the likeness of a click by X% so I should do that.

Whereas now I look at the button and think:

if it was orange I’d be more likely to click it because of this reason.

It’s a small distinction but a big realisation. It’s opened my mind to more possibilities, and this can only benefit me and my website visitors. As good as statistics and tutorials are, every website is unique and the only way to improve is to try new things and measure success. I think this is something all web designers can benefit from without doing any A/B testing.

What you need to do is this.

  1. Look at your website and imagine you are going to A/B test it.
  2. Think about the tests you would create.
  3. Think about WHY you would create the tests. How do these changes benefit your users? Why would making these changes improve your website?
  4. Make changes based upon your new understandings.

With any luck – following these steps will help you realise why people don’t do what you expect/ want them too and then you can improve your site accordingly. Of course this is all entirely speculative. Ideally you would run your changes as A/B tests and see how well they do, however I have found the test creation process a really interesting exercise and it’s given me some great things to consider.

As for Optimizely, the software is awesome and I plan to do a more detailed write-up on it in the future.

Let me know what you think on Mastodon, or BlueSky (or Twitter X if you must).

Related Posts

16 Dec 2010

A New Secret to Increasing Your Page Views

Do you want to increase the page views on your website? Everyone does right? Well this is something I have been spending a lot of time thinking about, and I recently found out something quite surprising.In hindsight I probably shouldn’t...
20 Jul 2016

Empathy in Web Design

I wasn’t able to make WordCamp Europe this year, but they’ve been really quick at getting all of the talks online, and so I have been watching some of them – and this one stood out.Morten Rand-Hendriksen is an experienced...
20 Feb 2013

The Amazing Art of Flipping Websites

Recently I have looked at expanding my internet empire. I have all sorts of ideas and never enough time – so I thought I would see if I could buy some websites relatively cheaply, improve them, and then either flip...
30 Jun 2007

WordPress tips and tricks – Custom Page Templates

Some time ago I posted the first of my tips and tricks for WordPress, and I thought it was about time I posted some more so, to start things off, here is a short tutorial on custom page templates in...
28 Aug 2009

Binary Moon WordPress design vIII

As I briefly mentioned on Monday, I have finally redesigned Binary Moon.I actually started redesigning about 2 years ago. Initially it was going to be an update rather than a totally new look, and I even built most of it,...
01 Mar 2018

Website Colour Analyzer

In January I published Colour, a simple app that analyses the colours used in a website and suggests ways you can merge the colour to make the design simpler.I don’t know if it’s an age thing, or a generational thing,...