Max Requests to One Hostname

Analysis
1

The HTTP Archive plots “maxDomainReqs”. The average is 50 requests on a single domain across the top 300K URLs. But what are the percentile values?

SELECT
  NTH(50, quantiles(maxDomainReqs)) median,
  NTH(90, quantiles(maxDomainReqs)) ninetieth,
  NTH(95, quantiles(maxDomainReqs)) ninetyfifth
FROM [httparchive:runs.latest_pages]

The results from the Sep 1 2013 crawl are:

39 is lower than 50 - that’s good. But the 90th percentile is 97!

1 Like
2

Trimmed at 200 req’s… the max is at 800 - yikes.

SELECT bucket, COUNT(*) pages FROM (      
  SELECT
    ROUND(maxDomainReqs/10)*10 bucket
  FROM [httparchive:runs.latest_pages]
)
GROUP BY bucket      
ORDER BY bucket;
1 Like