One less second for the page, one giant leap for the business

Matching site speed with business metrics was always something we had in mind here at Farfetch, since we began this path of changing how we look at the topic of performance. Achieving it would mean unlocking the next level towards a company-wide performance culture. We believed that the steps we took to get there were fundamental in creating a strong foundation in the company. These steps were properly documented in previous posts:

We were perfectly aware that making people care about performance looks good on paper but it's really quite hard to pull off, in practice. Everyone understands that being fast is important - no one wants to spend time on a site that takes a long time to load. 

But what is a "long time" and what is "fast"? Is half a second that important? What does it mean to be one second faster or one second slower? 

All of these questions crossed our minds, at some point. The truth is, unless we make the connection between performance and the business side, everything would be about awareness and keeping things under a budget that we can't quantify in business value.

We set out to finally answer this question, we were on a mission. But to get there, we required a strategy. This strategy would turn out to be quite simple and familiar to other projects at Farfetch. 

We have our own multi-channel tracking solution used by front end applications. It tracks the events generated by page views, users actions and system actions. We call it Omnitracking. The Omnitracking Data Model is Farfetch's solution for analytical, data exploration and reporting cases, made on the top of the events generated by the trackers. The data model's objective is to help and support anyone who needs to understand:

The idea was to use Omnitracking to track performance data of each pageview on farfetch.com. By following the Omnitracking data model we would guarantee a match between performance data and conversion funnel metrics for each session.

The data would follow the Omnitracking pipeline until it reached our Product Analytics team that would be able to make the necessary analysis and correlations to build a model that would translate site speed into business metrics (eg conversion rate,% single-page visits, etc) . We would be then able to understand how much does one second mean for Farfetch.

In terms of performance data, we are talking about the metrics that are available in RUM (Real User Monitoring) where all our users are (field data). We started by using the following metrics:

From these 3 metrics, our hypothesis was that TTI would reveal the strongest impact on the business side as we truly believe that interactivity metrics play a big role in determining if our user experience is frustrating or not. 

Our numbers also reveal that the amount of code (first and third party) the browser's main thread needs to handle is significant and pushes our TTI away. With the help of SpeedCurve we can see a big gap between PLT and TTI. This big gap was one reason we felt that TTI was an interesting metric to follow on RUM and be part of this exercise. The biggest bottleneck in web performance is no longer the network, it's the CPU time, and thus, TTI could turn out to be our north star.

We can get First Paint natively using the Performance API, specifically paint events.

performance.getEntriesByType ("paint"). find (function (item) {return item.name === "first-paint";});

We injected Google's TTI polyfill into the site. It uses the Long Tasks API to report long tasks. Long tasks are tasks taking longer than 50 milliseconds on the browser's main thread. When a task takes longer than 50 milliseconds, this is also called blocking time. It means that the browser's main thread will be unable to handle user input immediately because it's busy. This polyfill will only work in a browser that supports the PerformanceObserver and the PerformanceLongTaskTiming entry. At the moment, this is only fully supported on Chrome.

import ttiPolyfill from './path/to/tti-polyfill.js';

ttiPolyfill.getFirstConsistentlyInteractive (opts) .then ((tti) => {

  // Use `tti` value in some way.

});

With all this performance data flowing through our tracking system and data models, it was time to start analyzing the information and getting to the insights.

For context, this was the data used in the analysis:

The first step was to evaluate the quality of the data: detect the presence of missing data, outliers or strange values. In fact, 52% of total views have null values of TTI. Understanding the reasons behind missing data was one of the challenges in the analysis. 

Analysis on missing data shows that time spent on the page is the most relevant variable to justify missing data on TTI. 49% of total views with missing values spent less than 10 seconds on the page. This reflects the way TTI is calculated by the browser, and the effect that a bouncing user has, before the value has been determined.

Another important factor in the analysis is the statistical method to aggregate data. As TTI has some outliers, the best way to aggregate data without any bias due to outliers is to use the median value. So, to report TTI per day, we calculated the median of TTI registered on each pageview that Farfetch had. 

 

We found out that, on average, a user waits 1.5 seconds to see something on a page and 10 seconds to be guaranteed a fluid interaction with the page.

Without surprise, we saw that TTI for the mobile web is 1.7x slower than desktop.

78% of our mobile page views are considered slow on TTI, according to our internal assessment. We use an internal standard to assess speed classified in 3 buckets: fast, average and slow. We have a very demanding and ambitious standard for a fast assessment. This is our own way to drive performance and set the level that we want to achieve. On the desktop, the percentage of pageviews considered as slow is much lower, at 49%.

Taking into account our top pages, Product and Listing pages present the biggest opportunity for TTI improvement. On Listing pages, 71% of total page views are considered as slow. However, looking at the First Paint results, Listing pages are the pages where users need to wait less time to see something appear on the page.

The second challenge of the analysis was to calculate the correlation between performance metrics and Farfetch business metrics.

We found a correlation between the number of different page views in a user visit and TTI. We detected that the% of visits with more than 3 page views increases on visits with faster TTI. In the presence of faster values of TTI, users are not leaving Farfetch on first or second-page view in the visit, continuing with their usual journey.

Another important correlation found was the correlation with the single page visits and TTI values. On landing pages, the chance of a user leaving increases 5% for each additional second of TTI until 20 seconds. After 20 seconds of TTI, the change of user leave stabilises. 

So, in general, TTI seems to affect the chance of a user leaving Farfetch in the first 20 seconds of the page. The impact is not only seen on the first page view of the visit, but also in the following page views. In the presence of faster values of TTI, the user is not leaving Farfetch, continuing with their usual journey. So, as the user continues with the journey, in the end, we will see more conversions. The correlation with main business metrics such as conversion rate, "add to bag” and "start checkout” is evident.

On the next chart, we can see that the conversion rate decreases for each additional second that the user waits to start to interact fluidly with the page. On average, for each additional second of TTI, conversion rate decreases 2.6%.

The opportunity on conversion rate is similar by device, but still, the impact is even more evident on the mobile web. 

With all of the data above, we conclude that TTI has a huge impact on our business metrics, totally validating our hypothesis and in line with what we know from other eCommerce players. TTI has a direct relationship with the conversion rate. If TTI improves, conversion rates will also increase.

It’s a great feeling being able to prove with our own tools something that we always felt was the reality, despite not being able to put a monetary tag on it. We can now navigate the trade-offs around performance regressions and improvements because we now know how much that will generate to our business.

With this work, we did what has never been done at Farfetch, but there are several other important findings that came out of this exercise:

Farfetch.com is no different to other websites where performance is at the heart of the business. Being able to make this connection elevates our work and also our responsibilities for not losing sight of things and keeping a healthy balance between the functional and non-functional side of our products.