How Fast is the Web Getting Slower?

Analysis
1

Continuing the discussion from What are the quantiles of the Pagespeed score for the latest crawl?:

This got me thinking about ACTUAL speed of loading of webpages, and how it is changing over time. I modified Ilya’s query from the previous post to look at trends in visualComplete, onLoad and SpeedIndex. The numbers are all similar, so I’ll just run my analysis with the SpeedIndex numbers. As you know, SpeedIndex is a WebPageTest measurement of how quickly the screen paints (perceived load time). The faster you paint the whole screen, the lower the score.

So, copying Ilya’s code from the last post, and substituting in SpeedIndex, we can see how perceived page load time has changed over time.

SELECT date, 
    NTH(10, quantiles(SpeedIndex)) tenth, 
    NTH(20, quantiles(SpeedIndex)) twentieth, 
    NTH(30, quantiles(SpeedIndex)) thirtieth, 
    NTH(40, quantiles(SpeedIndex)) fortieth, 
    NTH(50, quantiles(SpeedIndex)) fiftieth, 
    NTH(60, quantiles(SpeedIndex)) sixtieth, 
    NTH(70, quantiles(SpeedIndex)) seventieth, 
    NTH(80, quantiles(SpeedIndex)) eightieth, 
    NTH(90, quantiles(SpeedIndex)) ninetieth 
FROM (
   SELECT 
     STRFTIME_UTC_USEC(INTEGER(createDate*1000000), "%Y-%m") date, SpeedIndex
   FROM httparchive:runs.2013_04_01_pages_mobile,httparchive:runs.2013_05_01_pages_mobile,httparchive:runs.2013_06_01_pages_mobile,httparchive:runs.2013_07_01_pages_mobile,httparchive:runs.2013_08_01_pages_mobile,httparchive:runs.2013_09_01_pages_mobile,httparchive:runs.2013_10_01_pages_mobile, httparchive:runs.2013_11_01_pages_mobile
  ) 
GROUP BY date, order by date

You may notice that the data is only for the last 7 months. In March, the default speed for HTTP Archive was changed from 1.5 MBPS to 5.0 MBPS. This obviously skews the load times, and so the older data was omitted.

Remember that here, the LOWER percentiles are the faster sites, and the higher percentiles are the slower sites.

In the last 7 months, there is a slow upward growth to these lines. This implies that all sites are getting SLOWER over time. If we assume that the growth is linear (the lines are not really great linear fits, but with only 7 data points, it seems an ok assumption), we can get the slope of these lines:

The fastest sites are (on average) getting 23 ms slower a month, and the slowest sites are adding nearly 90 ms every month. That is not totally unexpected. However, I will save the scariest piece of data for last. While it appears that the faster sites are slowly getting slower, if you look at the increase as a percentage of load time:

The faster sites are (as a percentage of load time) gettting slower faster than the big bloated sites.

NOTE: I attempted the same for mobile, but the mobile “speed” was changed in July, and there is a lot of noise - there are 2 months with over 100% jump in the data. It will be a while until the data settles down to perform this same analysis on mobile.

4 Likes
2

@doug_sillars that is depressing… Curious, have you tried comparing the top 1/10/100K sites? Curious if this is a uniform trend or if it can be isolated to some parts of the ecosystem.

1 Like
3

@igrigorik, this is a great question. I re-ran these queries limiting the rank to 1k, 5k, 10k, 100k and 1M.

There are 293k sites with Speedindex and Rank in the archive. The rank goes up to 998k (looks like there must be some sampling being done at higher rank numbers?) [@stevesoudersorg?]

SELECT  date, 
  NTH(10, quantiles(SpeedIndex)) tenth, 
  NTH(20, quantiles(SpeedIndex)) twentieth, 
  NTH(30, quantiles(SpeedIndex)) thirtieth, 
  NTH(40, quantiles(SpeedIndex)) fortieth, 
  NTH(50, quantiles(SpeedIndex)) fiftieth, 
  NTH(60, quantiles(SpeedIndex)) sixtieth, 
  NTH(70, quantiles(SpeedIndex)) seventieth, 
  NTH(80, quantiles(SpeedIndex)) eightieth, 
  NTH(90, quantiles(SpeedIndex)) ninetieth FROM 
     (SELECT STRFTIME_UTC_USEC(INTEGER(createDate*1000000), "%Y-%m") date, rank, SpeedIndex FROM 
    httparchive:runs.2013_04_01_pages, 
    httparchive:runs.2013_05_01_pages, httparchive:runs.2013_06_01_pages, 
    httparchive:runs.2013_07_01_pages, httparchive:runs.2013_08_01_pages, 
    httparchive:runs.2013_09_01_pages, httparchive:runs.2013_10_01_pages, 
    httparchive:runs.2013_11_01_pages 
    where rank <5000
    )  GROUP BY  date, order by date 
    ;

I see the same basic trend of sites getting slower over time.

I reported in the first post that the slope of the 50th percentile (median) was 45ms (implying a 45ms increase in SpeedIndex every month). Breaking it down by rank:

So the top ranked sites are ‘resisting’ the slowdown more than sites in the middle of the pack (those with ranks~10,000-100,000) based on median scores.by about 6 ms/month.

That is for the median website. What do we see at the other percentiles? I have graphed the rate of change for several groupings of rank by percentile:

There are few interesting features to note. Look at the top 1,000 websites. The 10, 20 and 30th percentile (the fastest 30%) of the top 1,000 websites are getting slower FASTER than the ‘pack.’ That is not good. :frowning:

At percentiles 70, 80 and 90 - top ranked apps are getting slower faster as well. Just looking at the 90th percentile - you can see that the highest ranked applications.have the highest rate of change, and the lowest ranked apps have the lowest rate of change. I can only guess that the 500k - 1M websites might not get updated as much every month - causing the rate of change to be smaller.