Welcome back to Teatime! This is a weekly feature in which we sip tea and discuss some topic related to quality. Feel free to bring your tea and join in with questions in the comments section.
Tea of the week: All the Lemon! by The Tea Dude. I know I usually recommend black teas, but I’ve got a sore throat this week, and lemon tisane hits the spot perfectly.
Today’s Topic: Improving websites, with science!
“What should I do for teatime this week?” I asked my fellow Sock Devs.
“A/B Testing! With cookies!” It was so off the wall, I had to do it. You might want to grab some cookies to go with your tea this week 😉
Introduction
So let’s say we have a website.
It’s a very good website; it sells cookies, and cookies are amazing. Everyone wants your cookies. Well, your boss’ cookies. You don’t bake the cookies; you’re the IT guy, keeping the registers working and the website up and running. It’s a small business, so there’s just you that does “the computer things”.
The website takes orders and puts them in a queue; every morning, the baker makes the custom orders, boxes them up, and sends them fedex, marking the orders done on their tablet. They then bake the day’s cookies for the shop, open the place, and a series of bored teenagers try to make the stupid Square dongle work on the tablet so people can buy cookies in person. And it works, but not well enough.
For some reason, the online business isn’t taking off the way they said it should. Mrs Smith, the baker, asks you, “Do we need to call up the Googles and ask them to make the website better?” You assure the baker – she’s a sweet older lady, she doesn’t know much about computers – that your SEO is fine, really, your site is at the top if you search for “Akron Cookies rainbows”, and that’s the best search term. Analytics say a bunch of people are showing up on the site, but there’s not a lot of orders. Maybe the website needs an update?
Conversion Rate
What she’s really worried about isn’t the SEO, it’s the Conversion Rate. The conversion rate is simply a percentage that indicates how many people that land on your website go on to “do the thing” – in this case, buy cookies. It could be signing up for a cookie newsletter, or leaving a review on Yelp, or whatever, but today we’re worried about sales, because when your sales are low, nothing else matters.
Well, you don’t know much about improving conversion rates. You don’t know much about baking cookies, either, or what kinds of behaviors drive people to buy cookies online. I mean, the problem could be anything, right? Maybe people just don’t like cookies anymore? No, no, Mrs Smith assures you that people buy cookies when they’re in the store. Her physical conversion rate is like, 80%. It’s something about the website.
The Scientific Method
Well, you don’t know much about conversion optimization. And you don’t know that much about websites. But you do know something about science. You aced your science fair in middle school, and you got all the way through AP bio in high school before you decided to do websites fulltime instead of becoming a world-famous botanist. Surely the scientific method could help you here, right?
So you dig out your composition book and turn to a new page. We’re ready to begin doing science now. How did it start….
Ask a Question
Oh right. The first thing you need to do is ask a question. As scientists, we are always making observations about the world around us; we don’t start digging into something, however, until we have a question. Why does that flower look that way? Why does paper fall slower than bowling balls if acceleration due to gravity is a constant? What happens if I bake cookies at twice the temperature for half the time?
So what’s our question? How about, “Why aren’t people buying cookies online?”
Gather Data
Next, we need to do some research. If we don’t have good, concrete data about the world around us, it’s going to be impossible to answer the questions. We’re not forging ahead into a realm of experimental physics here; we’re looking at something tangible and concrete in the world, so we can measure what it’s like currently.
So you go and you pull data from google analytics for a week or two, tracking where people go when they’re on the site. It’s not very exciting data; you have a bunch of visitors, but most of them seem to be hitting the home page, scrolling down for a bit, then leaving. Why do they leave? What’s going on here? Now we can dig into the specifics.
Construct Hypothesis
So here’s your cookie homepage:
You hit the site, pretending you’ve never been here before. Maybe you don’t realize you can buy cookies here? I mean, it says cookies, but there’s sky behind it, maybe this is like, a cookie fansite? Or a dictionary page?
It’s time to make a hypothesis, a concrete statement of fact that can be proven or disproven by the data you can collect. One format for a hypothesis is the following madlib: “If I _____, then ________ will happen.”
So you use this template and jot down your hypothesis in your composition notebook:
“If I change the home page picture, then more people will buy cookies.”
Conduct Experiment
Alright, time to get down, dirty, and dangerous! Wait, that’s botany. We’re dealing with websites here. But still, time for science! We’re going to change the home page, and see if conversions go up. But, wait a minute, Mrs Smith was just telling you about her new ad campaign that’s sure to “fix the Googles problem”. And it’s almost October and that’s National Cookie Month, and people tend to buy more cookies when they hear about that on the radio. So how will you know it works? The data’s going to be contaminated by all these outside factors changing! You can’t very well expect Mrs Smith to put her advertising efforts on hold for the purity of your data, and you can’t ask time to stop for you. So what can we do?
A/B Testing
This is where A/B testing comes in. The idea is, you show the new homepage header to half the incoming visitors. That way, the influx of new visitors doesn’t impact your data, because you’re comparing results over the same time period rather than across separate months. The data remains pure and clean, and you can track this specific variable without any outside interference.
Since this is a simple website and you’re using Google Analytics, this is pretty simple to set up. You FTP upload an alternate version of the home page, one with the cookies header. Then you go into google analytics, add an experiment, and put in both URLs. You put some tracking code onto each version of the home page and re-upload them, and voila, Google handles the rest. From there, half the visitors are routed to one version of the home page, and the other half to the other version. Google tracks how many of them go on to buy cookies, and starts giving you data pretty quickly.
Analyze Data
Now in Analytics you can see the results: a breakdown of how many conversions you’re getting on each version of the homepage on each day. And when the website goes down because the power goes out to the bakery and Mrs Smith doesn’t understand what the cloud is or why she should use it when there’s a perfectly good server sitting in the broom closet, that doesn’t affect the data either, because you’re getting 0 conversions for those six hours with both variations. So it all works out!
Act on the Results
Now that you have your results, you can act on it. The cookie homepage is definitely bringing in more sales; you stop the experiment, and re-upload the cookie header version as index.php so it shows up for everyone. The orders start pouring in, so much so that Mrs Smith is able to hire another baker to help handle just home deliveries. She’s so happy, she offers to design a line of cookies with your name on it! You thank her politely, but honestly, it was your interest in botany that saved the website, so maybe cookies that look like plants?
So she agrees! And you all live happily ever after… at least, until the conversion rate tops out and you start learning about SEO in earnest. Then you get to start all over again from the top.
Cookie Monster copyright Sesame Street. Sample website created with Mobirise website generation software.