How frequently do servers use chunked encoding for various content types?

Analysis
1 
SELECT mimetype, resp_transfer_encoding, size, 
  ROUND(ratio*100) as percent FROM (
    SELECT mimeType, resp_transfer_encoding, size, 
      RATIO_TO_REPORT(size) OVER(PARTITION BY mimeType) ratio 
    FROM (
      SELECT mimeType, resp_transfer_encoding, 
        count(1) as cnt, sum(respSize) as size
      FROM [runs.2015_04_15_requests] 
      GROUP by mimeType, resp_transfer_encoding
      HAVING cnt > 10000
      ORDER BY mimeType, cnt
    ) 
)
  • As of 04/2015, ~9% of responses are received via chunked transfers.
  • There is significant variation between content types - e.g. 44% of HTML resources are chunked transfers.
2

Why isn’t it higher? This is especially important for HTML - since HTML is parsed in streaming fashion.

I tried digging into this further by looking at the Server response header for HTML requests:

SELECT mimetype, resp_server, resp_transfer_encoding, cnt, size, 
  ROUND(ratio*100) as percent FROM (
    SELECT mimeType, resp_server, resp_transfer_encoding, cnt, size, 
      RATIO_TO_REPORT(size) OVER(PARTITION BY mimeType) ratio 
    FROM (
      SELECT mimeType, resp_server, resp_transfer_encoding, 
        count(1) as cnt, sum(respSize) as size
      FROM [httparchive:runs.2015_04_15_requests] 
      WHERE mimeType like "%html%" and resp_transfer_encoding like "%chunked%"
      GROUP by resp_server, mimeType, resp_transfer_encoding
      HAVING cnt > 10000
    ) 
)
ORDER BY percent desc

[FYI - I had to add the “httparchive:” prefix to your query to make it work: “FROM [httparchive:runs.2015_04_15_requests]”.]

The results are kinda interesting:

Why is Apache so high?
Do proxies like Cloudflare not support chunked encoding?

3

Re, cloudflare: you do see them in #6 with 8% of responses. Not sure about Apache vs others, if you add up all the nginx versions in above lis then you’re also looking at ~20%. The empty tokens (#2) is a security best practice… and that could go in any direction (e.g. in nginx its as simple as server_tokens off;).