Use of Keep-Alive header on the web?

Analysis
1 
SELECT timeout, count(*) as cnt,
  RATIO_TO_REPORT(CNT) OVER ()
FROM (
  SELECT REGEXP_EXTRACT(headers, r'Keep-Alive","value":"timeout=(\d+)') as timeout
  FROM (
    SELECT 
    JSON_EXTRACT(payload, '$.response.headers') as headers
    FROM [httparchive:har.2016_07_01_chrome_requests]
  ) resp
)
GROUP BY timeout
ORDER BY cnt DESC

Data for 07/16 run (top 500K alexa):

~17% requests specify Keep-Alive header, with 5, 15, and 2 seconds (yikes) as most popular values.

2

Wanted to revisit the above in a different manner i.e. distribution of % of requests in a page having keep-alive in response header.

SELECT bucket, count(keepAliveRatio)
FROM
(
SELECT
keepAlive/reqTotal keepAliveRatio,
ROUND(keepAlive/reqTotal * 100, 0) bucket
FROM
(
SELECT
COUNT(*) reqTotal,
SUM(IF(resp_connection = “keep-alive”,1,0)) keepAlive,
FROM httparchive.runs.2017_08_01_requests
GROUP BY pageid
)
)
GROUP BY bucket
ORDER BY bucket ASC

image

It appears a large number of web pages have less than 50% of their responses with keep-alive. It would be interesting to see if this is because of requests to third party sites.

To check the overall use of keep-alive, used the following query.

SELECT
SUM(IF(resp_connection = “keep-alive”,1,0)) keepAliveOn,
SUM(IF(resp_connection != “keep-alive”,1,0)) keepAliveOff
FROM httparchive.runs.2017_08_01_requests

image
T

3

Unless explicitly closed, HTTP 1.1 defaults to keep-alive so just having the header missing doesn’t mean it isn’t enabled. Also, for some beacon cases where you are only ever expecting a single request it might make sense to not enable.

The data should include a check from WebPageTest where it only dings pages for keep-alive if more than one request would have used the connection and if the connection is explicitly closed.

2 Likes
4

+1 to Pat’s point on HTTP/1.1.

If you want to figure out adoption of long-lived connections, you should segment by http/1.0 and 1.1 and:

  • (a) look for keep-alive header on http/1.0
  • (b) use of connection: close on http/1.1

For more details, see: https://hpbn.co/brief-history-of-http/#http11-internet-standard

1 Like