Performance Cost of Additional Domains

Analysis
1

In my last post, I was looking at how well 3rd party domains cache content. Looking through old posts, I saw Paddy’s post on domain count percentiles. Then I began to wonder – what happens as you add more and more domains. It has to slow down your site, right?

Let’s see what the data tells us.

SELECT domain_count, COUNT(domain_count) as numberofsiteswithcount,
 NTH(10, quantiles(SpeedIndex)) p10,
 NTH(50, quantiles(SpeedIndex)) p50,
  NTH(75, quantiles(SpeedIndex)) p75,
  NTH(90, quantiles(SpeedIndex)) p90,
  NTH(95, quantiles(SpeedIndex)) p95,


FROM (

SELECT  
  IF (req_host CONTAINS REGEXP_EXTRACT(origin, r'([\w-]+)'), INTEGER(1), INTEGER(3)) AS party,
  COUNT(DISTINCT req_host) as domain_count,
  pages.numDomains,
  
  COUNT(*) as cnt,
  SpeedIndex,
 
 

FROM [httparchive:runs.latest_requests_mobile] requests JOIN EACH (  
  SELECT rank, pageid, url, DOMAIN(url) as origin, SpeedIndex, numDomains
  FROM [httparchive:runs.latest_pages_mobile]
) pages ON pages.pageid = requests.pageid
WHERE NOT (req_host CONTAINS REGEXP_EXTRACT(origin, r'([\w-]+)'))
GROUP BY  pages.rank, pages.url,SpeedIndex, pages.numDomains,party


)
Group By domain_count
ORDER BY  domain_count;

This gives me the number of sites, and SpeedIndex percentiles for sites with various domain counts (in this case Mobile 3rd party). Removing the “NOT” from the WHERE parameter gives the 1st party domain count.

As expected, this is an exponential decrease, with an extremely long tail. Below is the graph of 3rd party domain count for websites. There is even one site with 514 external domains!

But what do we see for SpeedIndex? As expected, we see that as more domains are added, the SpeedIndex of the site generally increases. If we graph the SpeedIndex vs. # of domains, we see the following:

Each line is a different percentile. I’ll admit that these are most likely a logarithmic fit, but I used linear fits to make the math simpler. What we see is that as you add domains, there is an increase of 20-60ms per domain added.

On mobile, we see a similar effect:

For mobile the SpeedIndex increase is 208-324ms per domain.

For each new domain – there is a DNS lookup – which amounts to 1 RTT. Thanks to Ilya’s book, we know that RTT from NYC to SF is 42ms , and that RTT for mobile is ~200 ms for 3G.

What I think we are seeing here is that each additional 3rd party domain adds ~1 RTT of delay to the SpeedIndex of a website.

That doesn’t really make sense at first. Even though the 3rd party DNS lookup is 1 RTT, there is still the TCP handshake and the HTTP request. However, look at this waterfall chart from CNN.com:

Each DNS lookup pushes the waterfall out 1 RTT. If request 23 were made from i.cdn.turner.com instead if i2.cdn.turner.com, the waterfall would be pulled to the left by 2 RTTs (DNS lookup and TCP establishment). It is interesting how well this holds, since some DNS lookups are done concurrently.

Anyone else have thoughts as to why each domain is nearly exactly 1 RTT?