It is exciting to see the new Core Web Vitals technology report Rick Viscomi created, even though it shows an abysmal rate of 22% of WordPress powered origins pass the Core Web Vitals “Good” threshold .
However, part of what I want to understand isn’t reflected in the report’s graphs directly: how have CWV scores changed over WordPress versions? Are there measurable improvements in the wild after recent changes like adding native image (version 5.5) and iframe (version 5.7) lazy loading and WebP image support (version 5.8)?
I worked with Rick to arrive at this query to extract the performance data, grouped by WordPress version:
CREATE TEMP FUNCTION IS_GOOD(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64) RETURNS BOOL AS (
good / (good + needs_improvement + poor) >= 0.75
);
CREATE TEMP FUNCTION IS_NON_ZERO(good FLOAT64, needs_improvement FLOAT64, poor FLOAT64) RETURNS BOOL AS (
good + needs_improvement + poor > 0
);
SELECT *
FROM (
SELECT
date,
ARRAY_TO_STRING(ARRAY_AGG(DISTINCT category IGNORE NULLS ORDER BY category), ', ') AS categories,
app,
info,
REGEXP_EXTRACT( info, "(\\d.\\d).*" ) AS major_version,
client,
COUNT(DISTINCT url) AS origins,
COUNT(DISTINCT IF(good_fid, url, NULL)) AS origins_with_good_fid,
COUNT(DISTINCT IF(good_cls, url, NULL)) AS origins_with_good_cls,
COUNT(DISTINCT IF(good_lcp, url, NULL)) AS origins_with_good_lcp,
COUNT(DISTINCT IF(any_fid, url, NULL)) AS origins_with_any_fid,
COUNT(DISTINCT IF(any_cls, url, NULL)) AS origins_with_any_cls,
COUNT(DISTINCT IF(any_lcp, url, NULL)) AS origins_with_any_lcp,
COUNT(DISTINCT IF(good_cwv, url, NULL)) AS origins_with_good_cwv,
COUNT(DISTINCT IF(any_lcp AND any_cls, url, NULL)) AS origins_eligible_for_cwv,
SAFE_DIVIDE(COUNTIF(good_cwv), COUNTIF(any_lcp AND any_cls)) AS pct_eligible_origins_with_good_cwv
FROM (
SELECT
date,
CONCAT(origin, '/') AS url,
IF(device = 'desktop', 'desktop', 'mobile') AS client,
IS_NON_ZERO(fast_fid, avg_fid, slow_fid) AS any_fid,
IS_GOOD(fast_fid, avg_fid, slow_fid) AS good_fid,
IS_NON_ZERO(small_cls, medium_cls, large_cls) AS any_cls,
IS_GOOD(small_cls, medium_cls, large_cls) AS good_cls,
IS_NON_ZERO(fast_lcp, avg_lcp, slow_lcp) AS any_lcp,
IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_lcp,
(IS_GOOD(fast_fid, avg_fid, slow_fid) OR fast_fid IS NULL) AND
IS_GOOD(small_cls, medium_cls, large_cls) AND
IS_GOOD(fast_lcp, avg_lcp, slow_lcp) AS good_cwv
FROM
`chrome-ux-report.materialized.device_summary`
WHERE
date = '2021-09-01' AND
device IN ('desktop', 'phone')
) JOIN (
SELECT DISTINCT
CAST('2021-09-01' AS DATE) AS date,
category,
app,
info,
_TABLE_SUFFIX AS client,
url
FROM
`httparchive.technologies.2021_09_01_*`
WHERE
app = 'WordPress'
AND info != ''
) USING (date, url, client)
GROUP BY
date,
major_version,
app,
info,
client
) WHERE origins > 100
ORDER BY major_version DESC
It limits results to the app “WordPress” and groups the results by WordPress major version, which is extracted from the info field using REGEXP_EXTRACT. Here is the resulting data which I used to create the Dashboard.
To answer my previous questions:
- Q: Are there measurable improvements in the wild after recent changes like adding native image (version 5.5) and iframe (version 5.7) lazy loading and WebP image support (version 5.8)?
- A: No, unfortunately it does not appear these changes have had a significant impact on CWV scores in the wild.
- Lazy loading may be too aggressive as it is applied to all images (see https://web.dev/lcp-lazy-loading/ ). A fix is being worked on for version 5.9.
- WebP adoption in WordPress has been growing since the 5.8 release, however users need to manually convert their images to WebP before uploading to take advantage of the format. Landing WebP as the default format for sub-sized images which was started in this ticket will have a much bigger impact by automatically converting uploaded images to WebP.
- A: No, unfortunately it does not appear these changes have had a significant impact on CWV scores in the wild.
Data Caveats
- The data here relies on the presence of the WordPress “meta generator” tag which is output by default on WordPress sites and includes the version number. However, some sites hide this tag, so our data is incomplete.
- Because the underlying data is only available for more active sites, certain correlated statistics may be exaggerated. For example, the percentage of sites upgrading to WordPress 5.8 is much higher than the official statistics data. The dataset is only for sites with a significant amount of traffic which likely correlates to sites that would update to the latest version of WordPress.
What stands out
Several other points stand out in the report (link):
- 70% of origins are on the latest version of WordPress and 88% are on one of the last two versions, meaning changes we make to core reach the majority of sites relatively quickly.
- The number of origins is quite low for older versions of WordPress, with fewer than 5k origins for most versions before 4.7
- A huge bump at 4.9 - this group of sites is avoiding upgrading to the new block-based editor which was introduced in WordPress 5.0
- Overall CWV pass rates have generally decreased over WordPress versions. Although it might also be the case that “leading-edge” websites that update to the latest version are generally slower than those that linger on older versions.
- FID is generally very solid across all versions
- CLS is fair and trending worse over versions
- LCP is poor and trending much worse over versions
Going Forward
Our solid FID scores suggest it would make sense to focus more on changes that impact CLS and LCP
The performance team can use this dashboard to monitor progress, seeing how changes we make with each WordPress release affect Core Web Vitals scores.