Unexpected Benefits of A/B Testing

I am currently beta testing a new website called Optimizely – a super simple A/B testing product… and it’s gotten me thinking.

A/B testing is a process that allows you to optimize your website for your users. You create one or more variations of a page and then measure and compare the interactions on the new page against the old.

The bit that got me thinking was creating the variations.

When you design the tests you have the option to make any changes you like. Rather than make small tweaks and other simple edits I decided to make larger changes that alter the structure of the page entirely.

The part that interested me most was when I was trying to work out what to change. Thinking of it as an experiment I was able to clear my mind of all preconceptions and look at the site objectively. I was able to ask myself questions like:

• what would I do if I was visiting the site?
• what would encourage me to stick around longer?
• what would make me click that link?

And instead of trying to put myself in my visitors shoes, I became the visitor. The distinction is subtle’ but also important.

The problem I had before is that I was looking at the problems from the wrong perspective. I was trying to guess what the site visitor was trying to do, when instead I should have considered what I would do if I was the site visitor.

Previously I would have looked at the site and said:

so and so web design blog said making the button pink will increase the likeness of a click by X% so I should do that.

Whereas now I look at the button and think:

if it was orange I’d be more likely to click it because of this reason.

It’s a small distinction but a big realisation. It’s opened my mind to more possibilities, and this can only benefit me and my website visitors. As good as statistics and tutorials are, every website is unique and the only way to improve is to try new things and measure success. I think this is something all web designers can benefit from without doing any A/B testing.

What you need to do is this.

1. Look at your website and imagine you are going to A/B test it.
2. Think about the tests you would create.
3. Think about WHY you would create the tests. How do these changes benefit your users? Why would making these changes improve your website?
4. Make changes based upon your new understandings.

With any luck – following these steps will help you realise why people don’t do what you expect/ want them too and then you can improve your site accordingly. Of course this is all entirely speculative. Ideally you would run your changes as A/B tests and see how well they do, however I have found the test creation process a really interesting exercise and it’s given me some great things to consider.

As for Optimizely, the software is awesome and I plan to do a more detailed write-up on it in the future.