A quick look at Google Trends for the term ‘a/b testing’ shows a growing interest on this subject over the years. With so many A/B Testing products flooding the markets, we hardly need Google Trends to sense the rush. But, what has A/B testing got to do with Site Speed?
So, lets say you run a travel blog where you post your travel experiences and photos. You also track your blog posts’ bounce rate, time spent on page and other similar metrics. Every now and then, you also fire up your blog and make sure your posts show up ‘fast enough’. Now, some of your visitors check your post on their Macbook via fast broadband connection. But, a few may try to read your content while traveling in a Metro. Or on a slow network in a third-world country. Your blog post will load at different speeds for each of these visitors.
With exactly the same post loading up at different speeds, there is a natural A/B test always in action. Group A - hits where the post loaded in, say, less than 5 seconds and Group B - hits where the post loaded in more than 5 seconds. Now, if you track a post’s bounce rate sliced into these two groups, you’d know, in precise numbers, the visits that bounced due to slowness.
But, how to act upon this information? At the end of a standard A/B test, you go ahead with the winner - the button color that got the most clicks stays. Or you leave it as-is in case if the experiment outcome wasn’t conclusive. But, in case of site speed, what can you do for hits that loaded up in more than 5 seconds?
Knowing the number of visits bounced due to slowness, you can calculate the business impact of slowness. If this number is significant, you’d definitely want to look at the speed profile of your visitors. This includes their geo-location as well as the kind of device and network they use to access your site. Especially those visitors for whom your posts loaded in more than 5 seconds. But, what can you do about the speed profile of your visitors?
Of course, you cannot speed up your visitors’ devices or networks. But you can optimize your site to reduce the impact of your visitors’ speed profile on your conversion metrics due to slowness:
Linking page speed to conversions forces you to measure your visitors’ speed profile and know your pages’ key content. This helps you make data-driven speed optimizations rather than trying to make things run as fast as possible.
Before jumping into the topic, a little bit of background. I'm a techie who has worked on software speed for a decade. Trying to get software to run faster, I have often wondered how fast my site needs to run? Ofcourse, faster the better. But in a constrained world of effort versus gain, is it alright if it loads in five seconds and not three? Or two? What am I leaving on the table by not making it load faster?...continue reading