# Analysis **URL:** https://discuss.httparchive.org/c/analysis/1.md [Latest](https://discuss.httparchive.org/latest.md) · [Categories](https://discuss.httparchive.org/categories.md) --- ## [Response body is empty in the crawl dataset](https://discuss.httparchive.org/t/response-body-is-empty-in-the-crawl-dataset/2935)
**Author:** [@jiboncom](https://discuss.httparchive.org/u/jiboncom)\ **Replies:** 5\ **Last updated:** [August 14, 2026, 5:44pm UTC](https://discuss.httparchive.org/t/response-body-is-empty-in-the-crawl-dataset/2935 "2026-08-14T17:44:07Z")
I’ve been trying to retrieve the response\_body field for certain websites, but I’m running into some issues. For all data before 2015, the content.response\_body field consistently returns NULL. After 2015, the field is p… --- ## [No JSON requests 2022-2024](https://discuss.httparchive.org/t/no-json-requests-2022-2024/3003)
**Author:** [@Wuteken](https://discuss.httparchive.org/u/Wuteken)\ **Replies:** 5\ **Last updated:** [July 26, 2026, 3:43pm UTC](https://discuss.httparchive.org/t/no-json-requests-2022-2024/3003 "2026-07-26T15:43:19Z")
I’m investigating the evolution of the web over time for my masters thesis. One of the things I wanted to map out was the percentage of every type of request per year. When constructing this graph I realized there were … --- ## [Truncated Scripts and Frame Information Questions](https://discuss.httparchive.org/t/truncated-scripts-and-frame-information-questions/3023)
**Author:** [@anisenoff](https://discuss.httparchive.org/u/anisenoff)\ **Replies:** 1\ **Last updated:** [April 14, 2026, 2:48am UTC](https://discuss.httparchive.org/t/truncated-scripts-and-frame-information-questions/3023 "2026-04-14T02:48:12Z")
I noticed that for some of the response\_body values for scripts appear to be truncated. I was wondering if there was a specific reason that might be happening or systematic reason those requests might be different from o… --- ## [Why are some of the crux pages missing?](https://discuss.httparchive.org/t/why-are-some-of-the-crux-pages-missing/2997)
**Author:** [@ydimova11](https://discuss.httparchive.org/u/ydimova11)\ **Replies:** 1\ **Last updated:** [February 21, 2026, 7:16pm UTC](https://discuss.httparchive.org/t/why-are-some-of-the-crux-pages-missing/2997 "2026-02-21T19:16:37Z")
I noticed that the there are less websites included in the HTTPArchive dataset than in the CrUX list. For example, for the top 10k CrUX, only 6531 websites are in httparchive.crawl.pages. The difference seems way to high… --- ## [The Performance Impact of Cryptocurrency Mining on the Web](https://discuss.httparchive.org/t/the-performance-impact-of-cryptocurrency-mining-on-the-web/1126)
**Author:** [@doug\_sillars](https://discuss.httparchive.org/u/doug_sillars)\ **Replies:** 15\ **Last updated:** [September 27, 2025, 11:57am UTC](https://discuss.httparchive.org/t/the-performance-impact-of-cryptocurrency-mining-on-the-web/1126 "2025-09-27T11:57:26Z")
Last week there was an article pointing out that hundreds of websites have been hacked to add bitcoin cryptocurrency mining scripts into them. This of course is interesting to the media because it uses the words ‘hacke… --- ## [New dashboard: the Core Web Vitals technology report](https://discuss.httparchive.org/t/new-dashboard-the-core-web-vitals-technology-report/2178)
**Author:** [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\ **Replies:** 16\ **Last updated:** [September 12, 2025, 12:49pm UTC](https://discuss.httparchive.org/t/new-dashboard-the-core-web-vitals-technology-report/2178 "2025-09-12T12:49:33Z")
Jump right into the dashboard: httparchive.org/reports/cwv-tech The technologies you use to build your website can have an effect on your ability to deliver good user experiences. Good UX is key to performing well with C… --- ## [Httparchive.technologies table gone on big query?](https://discuss.httparchive.org/t/httparchive-technologies-table-gone-on-big-query/2917)
**Author:** [@gbrlmtrz](https://discuss.httparchive.org/u/gbrlmtrz)\ **Replies:** 2\ **Last updated:** [April 21, 2025, 10:25pm UTC](https://discuss.httparchive.org/t/httparchive-technologies-table-gone-on-big-query/2917 "2025-04-21T22:25:16Z")
Hello everyone, every now and then I would run a report based on the technologies table of the httparchive dataset, today I’ve noticed that the tables are gone, I see the technologies group is still referenced on the doc… --- ## [Adding a new product to the Technology dropdown menu](https://discuss.httparchive.org/t/adding-a-new-product-to-the-technology-dropdown-menu/2868)
**Author:** [@adriansp](https://discuss.httparchive.org/u/adriansp)\ **Replies:** 3\ **Last updated:** [April 1, 2025, 5:08am UTC](https://discuss.httparchive.org/t/adding-a-new-product-to-the-technology-dropdown-menu/2868 "2025-04-01T05:08:01Z")
Hi there, I was just reviewing an HTTP Archive Core Web Vitals Technology Report and was wondering how I could add another product to the Technology menu and have it displayed on the graph. What would be the process for… --- ## [Exploring the Impact of Progressive Web Apps (PWAs) on Core Web Vitals: Insights from HTTP Archive Data](https://discuss.httparchive.org/t/exploring-the-impact-of-progressive-web-apps-pwas-on-core-web-vitals-insights-from-http-archive-data/2893)
**Author:** [@DonnaPaulsen](https://discuss.httparchive.org/u/DonnaPaulsen)\ **Replies:** 0\ **Last updated:** [March 4, 2025, 9:59am UTC](https://discuss.httparchive.org/t/exploring-the-impact-of-progressive-web-apps-pwas-on-core-web-vitals-insights-from-http-archive-data/2893 "2025-03-04T09:59:51Z")
Hello everyone, I’ve been diving deep into how Progressive Web Apps (PWAs) are evolving and wanted to discuss their potential impact on Core Web Vitals, particularly using the data we can extract from HTTP Archive. As … --- ## [HTTP/2 or HTTP/3 Requests per page and their size distribution](https://discuss.httparchive.org/t/http-2-or-http-3-requests-per-page-and-their-size-distribution/2819)
**Author:** [@csosto-pk](https://discuss.httparchive.org/u/csosto-pk)\ **Replies:** 0\ **Last updated:** [November 25, 2024, 4:04am UTC](https://discuss.httparchive.org/t/http-2-or-http-3-requests-per-page-and-their-size-distribution/2819 "2024-11-25T04:04:45Z")
Hello, There was a recent post by a CDN that shared data about their QUIC connection response sizes (to at least one GET in the request) which have a median of about 7KB. This does not align with HTTP Archive data which… --- ## [Detecting the cookies consent popup in HAR](https://discuss.httparchive.org/t/detecting-the-cookies-consent-popup-in-har/2793)
**Author:** [@richard](https://discuss.httparchive.org/u/richard)\ **Replies:** 1\ **Last updated:** [September 15, 2024, 8:23am UTC](https://discuss.httparchive.org/t/detecting-the-cookies-consent-popup-in-har/2793 "2024-09-15T08:23:16Z")
Hi everyone, We are interested in navigating the time trend of webs having a cookie Consent banner. Specifically, we would like to know the proportion of websites that have such banner from 2014-2024. Is there a good wa… --- ## [Have free quota but keep getting error on query quota in bigquery](https://discuss.httparchive.org/t/have-free-quota-but-keep-getting-error-on-query-quota-in-bigquery/2790)
**Author:** [@richard](https://discuss.httparchive.org/u/richard)\ **Replies:** 2\ **Last updated:** [September 8, 2024, 5:53am UTC](https://discuss.httparchive.org/t/have-free-quota-but-keep-getting-error-on-query-quota-in-bigquery/2790 "2024-09-08T05:53:32Z")
Hi everyone, I am new to the bigquery and httparchive. I follow Analysis of Cookie Size and try to extract the informaiton on third-party cookies. Everytime i run the code, it keeps pop up an error msg "Quota exceeded: … --- ## [List of domains using a given technology](https://discuss.httparchive.org/t/list-of-domains-using-a-given-technology/2755)
**Author:** [@gbrlmtrz](https://discuss.httparchive.org/u/gbrlmtrz)\ **Replies:** 2\ **Last updated:** [May 23, 2024, 7:09pm UTC](https://discuss.httparchive.org/t/list-of-domains-using-a-given-technology/2755 "2024-05-23T19:09:21Z")
Hello! I would like to know if there is a way we can check the CMS technology reported by a given domain on the Core Web Vitals technology report Specifically, I am curious to see which are the 93 domains that are flagg… --- ## [SAN Certificates: How Many Alt-Names are Too Many?](https://discuss.httparchive.org/t/san-certificates-how-many-alt-names-are-too-many/1867)
**Author:** [@paulcalvano](https://discuss.httparchive.org/u/paulcalvano)\ **Replies:** 5\ **Last updated:** [July 25, 2024, 1:44pm UTC](https://discuss.httparchive.org/t/san-certificates-how-many-alt-names-are-too-many/1867 "2024-07-25T13:44:35Z")
According to the HTTP Archive, 84% of HTTPS certificates are using the Subject Alternate Name (SAN) extension, which allows multiple hostnames to be protected by a single certificate. The largest certificate I found in t… --- ## [如何获取api 报文数据，进行学习？](https://discuss.httparchive.org/t/api/2746)
**Author:** [@chennian-dev](https://discuss.httparchive.org/u/chennian-dev)\ **Replies:** 1\ **Last updated:** [April 28, 2024, 2:15pm UTC](https://discuss.httparchive.org/t/api/2746 "2024-04-28T14:15:25Z")
从主流网站采集的数据可以开源，进行研究学习吗？ --- ## [Are Popular Websites Faster?](https://discuss.httparchive.org/t/are-popular-websites-faster/162)
**Author:** [@doug\_sillars](https://discuss.httparchive.org/u/doug_sillars)\ **Replies:** 1\ **Last updated:** [January 6, 2014, 5:47pm UTC](https://discuss.httparchive.org/t/are-popular-websites-faster/162 "2014-01-06T17:47:43Z")
--- ## [Progressive Web Apps in the HTTP Archive](https://discuss.httparchive.org/t/progressive-web-apps-in-the-http-archive/1401)
**Author:** [@tomayac](https://discuss.httparchive.org/u/tomayac)\ **Replies:** 3\ **Last updated:** [February 20, 2024, 9:27pm UTC](https://discuss.httparchive.org/t/progressive-web-apps-in-the-http-archive/1401 "2024-02-20T21:27:33Z")
Progressive Web Apps in the HTTP Archive Thomas Steiner, Google Hamburg, Germany :e-mail: tomac@google.com • :bird: @tomayac • :smile\_cat: tomayac (Cross-posted to Medium: https://medium.com/dev-channel/progressive-web… --- ## [Higher level stats for structured data](https://discuss.httparchive.org/t/higher-level-stats-for-structured-data/2670)
**Author:** [@leaverou](https://discuss.httparchive.org/u/leaverou)\ **Replies:** 14\ **Last updated:** [November 22, 2023, 6:12am UTC](https://discuss.httparchive.org/t/higher-level-stats-for-structured-data/2670 "2023-11-22T06:12:21Z")
I looked up the Structured Data chapter to cite data on what % of sites use structured data at all, and what % specify any type of metadata like description, image etc, but it’s currently far more granular than that, mis… --- ## [Understanding embeds in WordPress](https://discuss.httparchive.org/t/understanding-embeds-in-wordpress/2547)
**Author:** [@adamsilverstein](https://discuss.httparchive.org/u/adamsilverstein)\ **Replies:** 0\ **Last updated:** [March 21, 2023, 10:31pm UTC](https://discuss.httparchive.org/t/understanding-embeds-in-wordpress/2547 "2023-03-21T22:31:46Z")
I am researching lazy loading of embeds on WordPress sites. In particular, I want to look into the “Embed block” WordPress places into a post when a user pastes in a URL from a supported oEmbed provider. Currently suppor… --- ## [Planning the Web Almanac 2019](https://discuss.httparchive.org/t/planning-the-web-almanac-2019/1553)
**Author:** [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\ **Replies:** 14\ **Last updated:** [June 4, 2019, 5:10am UTC](https://discuss.httparchive.org/t/planning-the-web-almanac-2019/1553 "2019-06-04T05:10:44Z")
This year we’re setting a new goal to create an annual report of the state of the web, called the Web Almanac. The purpose of the Almanac is to take a snapshot of how the web is built and how it has changed over the year… --- ## [How many https websites are using Mixed Content images](https://discuss.httparchive.org/t/how-many-https-websites-are-using-mixed-content-images/2531)
**Author:** [@zcorpan](https://discuss.httparchive.org/u/zcorpan)\ **Replies:** 7\ **Last updated:** [February 28, 2023, 7:16pm UTC](https://discuss.httparchive.org/t/how-many-https-websites-are-using-mixed-content-images/2531 "2023-02-28T19:16:02Z")
We (Mozilla) wanted to find out how the landscape has changed since 2020 wrt to usage of mixed content images on https websites, and how many of those images are broken anyway (fail to fetch). Here’s my method to find ou… --- ## [Websites with cache-control: no-store, by CDN](https://discuss.httparchive.org/t/websites-with-cache-control-no-store-by-cdn/2619)
**Author:** [@aaronpeters](https://discuss.httparchive.org/u/aaronpeters)\ **Replies:** 7\ **Last updated:** [September 8, 2023, 7:36pm UTC](https://discuss.httparchive.org/t/websites-with-cache-control-no-store-by-cdn/2619 "2023-09-08T19:36:00Z")
I recently queried the HTTP Archive to get the number of websites (pages) that serve the HTML with response header cache-control contains no-store . My query: SELECT \_TABLE\_SUFFIX AS client, count(DISTINCT pageid) … --- ## [Page weight in 1995?](https://discuss.httparchive.org/t/page-weight-in-1995/2626)
**Author:** [@tedroc](https://discuss.httparchive.org/u/tedroc)\ **Replies:** 1\ **Last updated:** [September 18, 2023, 3:23pm UTC](https://discuss.httparchive.org/t/page-weight-in-1995/2626 "2023-09-18T15:23:43Z")
It’s very nice to see here the evolution of page weight (and other elements) over time. I’m interesting in having the same data before 2010. Here’s what wrote WebSiteOptimization in 2014 for the average page size : 19… --- ## [Comparing Core Web Vital Performance across WordPress versions](https://discuss.httparchive.org/t/comparing-core-web-vital-performance-across-wordpress-versions/2256)
**Author:** [@adamsilverstein](https://discuss.httparchive.org/u/adamsilverstein)\ **Replies:** 2\ **Last updated:** [August 14, 2023, 9:32am UTC](https://discuss.httparchive.org/t/comparing-core-web-vital-performance-across-wordpress-versions/2256 "2023-08-14T09:32:52Z")
It is exciting to see the new Core Web Vitals technology report Rick Viscomi created, even though it shows an abysmal rate of 22% of WordPress powered origins pass the Core Web Vitals “Good” threshold . However, part of… --- ## [Missing websites in April 2022](https://discuss.httparchive.org/t/missing-websites-in-april-2022/2493)
**Author:** [@schoki](https://discuss.httparchive.org/u/schoki)\ **Replies:** 4\ **Last updated:** [December 19, 2022, 11:15am UTC](https://discuss.httparchive.org/t/missing-websites-in-april-2022/2493 "2022-12-19T11:15:45Z")
Dear Http Archive Comunity, We are researchers investigating trends in website request data using Http Archive and came across a strange issue we would like to have more clarity about. It has been observed that there a… --- ## [How Many Sites Are Still Using AppCache?](https://discuss.httparchive.org/t/how-many-sites-are-still-using-appcache/1443)
**Author:** [@paulcalvano](https://discuss.httparchive.org/u/paulcalvano)\ **Replies:** 3\ **Last updated:** [October 5, 2021, 4:20pm UTC](https://discuss.httparchive.org/t/how-many-sites-are-still-using-appcache/1443 "2021-10-05T16:20:37Z")
The Application Cache has been deprecated and removed from the web standards. While some browsers still support it - that support is going away. For example, starting with Firefox 44 a console warning advised developers… --- ## [Help finding list of home pages with specific http response header](https://discuss.httparchive.org/t/help-finding-list-of-home-pages-with-specific-http-response-header/2567)
**Author:** [@maxjoles](https://discuss.httparchive.org/u/maxjoles)\ **Replies:** 7\ **Last updated:** [June 7, 2023, 2:17pm UTC](https://discuss.httparchive.org/t/help-finding-list-of-home-pages-with-specific-http-response-header/2567 "2023-06-07T14:17:17Z")
I’m new to the HTTP Archive dataset as well as Big Query, but have some very basic SQL knowledge. I’m hoping to get help with a certain query. I’m looking to generate a list of home page URLs where the http response hea… --- ## [What is the most popular feed format: rss, atom, or JSON?](https://discuss.httparchive.org/t/what-is-the-most-popular-feed-format-rss-atom-or-json/2577)
**Author:** [@pawelgrzybek](https://discuss.httparchive.org/u/pawelgrzybek)\ **Replies:** 3\ **Last updated:** [May 27, 2023, 1:01am UTC](https://discuss.httparchive.org/t/what-is-the-most-popular-feed-format-rss-atom-or-json/2577 "2023-05-27T01:01:16Z")
Hi all. I am new to HTTP Archive and BigQuery world. I am trying to determine the popularity of feed formats (rss/atom/json)? First idea that comes to my mind is to scan the body of httparchive.response\_bodies for appl… --- ## [What is the least common colour used on web pages?](https://discuss.httparchive.org/t/what-is-the-least-common-colour-used-on-web-pages/1085)
**Author:** [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\ **Replies:** 6\ **Last updated:** [May 3, 2023, 3:19pm UTC](https://discuss.httparchive.org/t/what-is-the-least-common-colour-used-on-web-pages/1085 "2023-05-03T15:19:40Z")
On Twitter, @v21 asks: Does anyone want to take a stab at researching this one? :cowboy\_hat\_face: --- ## [Querying the HTTP Archive with DuckDB](https://discuss.httparchive.org/t/querying-the-http-archive-with-duckdb/2568)
**Author:** [@acme](https://discuss.httparchive.org/u/acme)\ **Replies:** 0\ **Last updated:** [May 3, 2023, 10:51am UTC](https://discuss.httparchive.org/t/querying-the-http-archive-with-duckdb/2568 "2023-05-03T10:51:04Z")
I needed to explore ETag response headers locally and I’ve come up with a workflow for querying HTTP Archive data on my laptop in Parquet format using DuckDB. You may have to create Google Cloud project IDs, Google BigQ… [Next page](https://discuss.httparchive.org/c/analysis/1.md?page=1)