Who is driving dns-prefetch adoption?

Analysis
1

Posing a challenge to all the data analysts in the room :cowboy_hat_face:

In the 2021 Resource Hints chapter of the Web Almanac (coming soon!) author @imkevdev says:

The most widely used resource hint is dns-prefetch (36.4% on mobile); which is unsurprising, considering it was introduced in 2009. With the widespread use of HTTPS, in many cases it should be replaced with preconnect (12.7% on mobile).

If true, this sounds like a good opportunity to identify who is driving dns-prefetch adoption and try to improve the ecosystem by encouraging them to use the preconnect hint instead.

2

Would be interesting to see some analysis on this. However he does say “in many cases” for good reason.

DNS look-ups are cheap.

Preconnects are more expensive due to HTTPS certificates using up bandwidth (not to mention the chatiness needed to look up DNS, TCP, and HTTPS). Small in grand scheme of this but at critical initial loading time “small” can make a big difference.

But still would be good to dig into it with that in mind…

3 Likes
3

Seconded on the expensiveness of preconnects. DNS prefetches are low cost, and will probably hit some DNS caching layer somewhere (sometimes even in a local network’s router). I think encouraging preconnect over dns-prefetch could also have the unintended side-effect of encouraging developers to use expensive preconnects that could go to waste in scenarios where a resource from the target origin never gets used.

2 Likes
4

Looking at the domains which contribute mostly towards dns-prefetch usage, Wordpress is the biggest contributor. I think by default Wordpress includes the snippet. *

<link rel=dns-prefetch href='//s.w.org'/>

or

<link rel=dns-prefetch href='//s0.wp.com'/>
<link rel=dns-prefetch href='//wordpress.com'/>

There is no preconnect hint by default on Wordpress sites. (please correct me if mistaken).

The next most popular domain is related to Google Fonts. Google Fonts now (couldn’t find a date when this changed) recommends preconnect.

client host freq total pct
mobile s.w.org 1888032 9575451 19.72%
mobile fonts.googleapis.com 1344386 9575451 14.04%
mobile www.google.com 261441 9575451 2.73%
mobile ajax.googleapis.com 211449 9575451 2.21%
mobile monorail-edge.shopifysvc.com 190904 9575451 1.99%
mobile www.googletagmanager.com 181711 9575451 1.90%
mobile s0.wp.com 162677 9575451 1.70%
mobile cdnjs.cloudflare.com 151468 9575451 1.58%

Query:

#standardSQL
# Most popular hosts users dns-prefetch to
# capped to one hit per url to avoid having the results skewed by websites which dns-prefetch many resources from the same host

CREATE TEMPORARY FUNCTION getResourceHintsHrefs(payload STRING, hint STRING)
RETURNS ARRAY<STRING>
LANGUAGE js AS '''
try {
  var $ = JSON.parse(payload);
  var almanac = JSON.parse($._almanac);
  return almanac['link-nodes'].nodes.filter(link => link.rel.toLowerCase() == hint).map(n => n.href);
} catch (e) {  
  return [];
}
''' ;

SELECT
  client,
  host,
  freq,
  total,
  pct
FROM (
    SELECT
      client,
      host,
      COUNT(0) AS freq,
      SUM(COUNT(0)) OVER (PARTITION BY client) AS total,
      COUNT(0) / SUM(COUNT(0)) OVER (PARTITION BY client) AS pct,
      ROW_NUMBER() OVER (PARTITION BY client ORDER BY COUNT(0) DESC) AS pos
    FROM (
      SELECT
        client,
        url,
        host
      FROM (
          SELECT
            _TABLE_SUFFIX AS client,
            url,
            NET.HOST(href) AS host
          FROM
            `httparchive.pages.2021_07_01_*`,
            UNNEST(getResourceHintsHrefs(payload, "dns-prefetch")) AS href
        )
      GROUP BY
        client,
        url,
        host
    )
    GROUP BY
      client,
      host
    ORDER BY
      client,
      freq DESC
)
WHERE pos <= 100

When looking at the numbers, keep in mind that Wordpress makes up around 25% of origins in HA.

EDIT (2021-11-12): * The s.w.org domain is the CDN for serving emoji SVG images (provided by Twemoji) when WordPress detects (via its inlined emoji-loader.js) that the browser doesn’t support native emoji characters. This CDN has been used since WordPress 4.6. Source