# Growth of the Web in 2020

**URL:** <https://discuss.httparchive.org/t/growth-of-the-web-in-2020/2029>\
**Category:** Analysis\
**Created:** [September 29, 2020, 6:38pm UTC](https://discuss.httparchive.org/t/growth-of-the-web-in-2020/2029 "2020-09-29T18:38:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![paulcalvano](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/paulcalvano/32/1601_2.png) [@paulcalvano](https://discuss.httparchive.org/u/paulcalvano)\
**Post date:** [September 29, 2020, 6:38pm UTC](https://discuss.httparchive.org/t/growth-of-the-web-in-2020/2029/1 "2020-09-29T18:38:35Z")

</div>

_In a blog post on [paulcalvano.com](https://paulcalvano.com/2020-09-29-growth-of-the-web-in-2020/), I shared some insights into the growth of the web, as measured by an increase in the number of origins in the Chrome User Experience Report. This post shares the SQL queries and data I used to prepare that article._

**Global Growth of Domains Accessed in 2020**

The graph below illustrates the total number of websites and registered domains that the Chrome User Experience reported across all form factors during the previous 12 months.

![](https://lh6.googleusercontent.com/_0oXgHRwPF7rIQW8xT0HALbMG6zgl97TNWsVVkLBWg7CaHaHDFOeFPmSooFnLmR0592RrKEoSDiCv2TvipacUr2P6T7XXJJR-TDmwqZ0HldLbFWWoZqTXDrw9v9BDlRzkGGxY3qA)

![](https://lh4.googleusercontent.com/e-7pA-i8AV1e0sMPm0gRr-1m90ZS_04luJ6YBfdcrPBaM1zVjGdHMKKQB8IzCHMDIONzyaWDhlC9KcvUlCvsARLQsfw0SCC095eyMoZu1AU9D0u4-neZ_M9tkv9NuOv1p7CqU8-1)

The SQL I used for creating this table is below. It simply counts the number of origins, registered domains and top level domains in each table of the dataset.:

```sql
SELECT
   _TABLE_SUFFIX AS month,
   COUNT(DISTINCT origin) AS sites,
   COUNT(DISTINCT NET.REG_DOMAIN(origin)) AS registered_domains, 
   COUNT(DISTINCT NET.PUBLIC_SUFFIX(origin)) AS top_level_domains
FROM 
   `chrome-ux-report.all.*`
GROUP BY 
   month
ORDER BY 
   month ASC

```

**Mobile vs Desktop Growth**

This next graph illustrates the number of origins by form factor

![](https://lh3.googleusercontent.com/90RGMthckkrMr3GwwupP9_QytxFuI7IhUwgSY2cbYkJoBDRXVrpBCyldqJlAEB5fPY72lbYcvYn2aH0-rkDhgh6wZUXKGXb7m9ZRjjvN0mTkE3XSab7MLPvkJI8Vsy6xrTIwaKTw)

The SQL used for this is similar to the previous query, but includes the form factor as a grouped column:

```sql
SELECT 
    _TABLE_SUFFIX AS month,
    form_factor.name As form_factor,
    COUNT(DISTINCT origin) AS sites,
    COUNT(DISTINCT NET.REG_DOMAIN(origin)) AS registered_domains,
    COUNT(DISTINCT NET.PUBLIC_SUFFIX(origin)) AS top_level_domains
FROM  
    `chrome-ux-report.all.*`
GROUP BY 
    month, 
    form_factor
ORDER BY 
    month, 
    form_factor ASC

```

How Has this Varied by Region?

The tables below detail the number of sites included in the CrUX dataset during December 2019 as well as January, May, June and August 2020.

![](https://lh5.googleusercontent.com/TXTBYyexOY8YrN0uk3Dj_LOjLP1kvt4IczdYXbCA-_YmxJLy1UvF4pZCviEZtCgCkjSvSMvvuSzO3OtPPSRflpXt8xsvrfH5iMIKS-HlnE6MzqVgyz2bGfBUOFojVv3wpl62z4ot)

The SQL used for this uses a technique that @rick wrote about a while back, where you can assign a subregion to each country in the dataset. It’s too large to paste in this article, so you can find it here - [https://console.cloud.google.com/bigquery?sq=226352634162:5cae584e069d49838aec9407d7e8c788](https://console.cloud.google.com/bigquery?sq=226352634162:5cae584e069d49838aec9407d7e8c788)

Analyzing by Top Level Domain

The next two graphs show the distribution of websites by TLD.

![](https://lh6.googleusercontent.com/SeF2eqHkZEw8Dn1YOjXu6pXHHKeY57Cq4L-WSoXx0zNHFtRajaQ4DS1F1dz-kz1Yv5T4cvVJYhpuTh_X3NJEQth43CPZ_1ATJFmkHMsWGY4QGOxTPpJuJ0Yt0PlG91SnTd6So-er)

![](https://lh5.googleusercontent.com/LUgvmi5Sd_271sukdOKxYKy1L6LNkzzFpK4r7j04tgtgU-OHhDt75zAgr9dE1UYM2qzmx4Er_PvL7dinMdwL21VGeArHGfc-VA9eD5pxaha5XN8L4wxPauSi_GWJhdL_S6LkCtrF)

The SQL for this is below, and the graphs were created by creating pivot tables from the results

```sql
SELECT
    month,
    NET.PUBLIC_SUFFIX(origin) AS tld,
    COUNT(*) AS sites
FROM (
    SELECT DISTINCT "202008" AS month, origin FROM `chrome-ux-report.all.202008`
    UNION ALL
    SELECT DISTINCT "202007" AS month, origin FROM `chrome-ux-report.all.202007`
    UNION ALL
    SELECT DISTINCT "202006" AS month, origin FROM `chrome-ux-report.all.202006`
    UNION ALL
    SELECT DISTINCT "202005" AS month, origin FROM `chrome-ux-report.all.202005`
    UNION ALL
    SELECT DISTINCT "202004" AS month, origin FROM `chrome-ux-report.all.202004`
    UNION ALL
    SELECT DISTINCT "202003" AS month, origin FROM `chrome-ux-report.all.202003`
    UNION ALL
    SELECT DISTINCT "202002" AS month, origin FROM `chrome-ux-report.all.202002`
    UNION ALL
    SELECT DISTINCT "202001" AS month, origin FROM `chrome-ux-report.all.202001`
    UNION ALL
    SELECT DISTINCT "201912" AS month, origin FROM `chrome-ux-report.all.201912`
)
GROUP BY month, tld

```

You can also see all the data used for these graphs in[this Google Sheet](https://docs.google.com/spreadsheets/d/1eGfT0dBslpSl8Xl6ey7a_fNT0Cj5tTfJzX6gpwpXDBE/edit?usp=sharing).