\
**Category:** Analysis\
**Created:** [December 4, 2014, 11:49pm UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442 "2014-12-04T23:49:47Z")\
**Posts on this page:** 7\
**Page:** 1
**Author:**  [@igrigorik](https://discuss.httparchive.org/u/igrigorik)\
**Post date:** [December 4, 2014, 11:49pm UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/1 "2014-12-04T23:49:47Z")
Earlier this year HTTP Archive began saving response bodies of HTML, CSS, and JavaScript resources. The responses are available for desktop runs only, and binary responses are omitted due to size constraints – even with HTML/CSS/JS-only restriction, each run results in ~250~350GB of payload data.
If you want to run analysis on the response bodies, you can [download the raw data](https://github.com/HTTPArchive/httparchive/issues/6#issuecomment-32502849) and do science to it. Alternatively, as an experiment, I’ve imported the response body payloads directly into BigQuery, which allows you to run queries against them from the comfort of your browser (check out [BigQuery query reference docs](https://cloud.google.com/bigquery/query-reference)):
- If you’re new to BigQuery, [read these instructions](https://www.igvita.com/2013/06/20/http-archive-bigquery-web-performance-answers/).
- Head to: [https://bigquery.cloud.google.com/project/httparchive](https://bigquery.cloud.google.com/project/httparchive)
- Expand the “runs” dataset and look for following tables:
- _2014\_08\_01\_requests\_body_
- _2014\_08\_15\_requests\_body_
The schema is very simple: `page` maps to `XXXX_XX_XX_pages.url`, `url` maps to `XXXX_XX_XX_requests.url`, and rest is self-explanatory. The first two fields allow you to “join” against the requests and/or pages tables to filter the table based on arbitrary conditions - e.g. run analysis on top 1K sites. But enough handwaving…
A sample query courtesy of @stevesoudersorg that counts occurrences of async vs blocking Google Analytics scripts in HTML markup:
```sql
SELECT COUNT(*) as num,
CASE
WHEN REGEXP_MATCH(body, r'ga\.src.*\.google-analytics\.com/ga\.js') THEN "async"
WHEN REGEXP_MATCH(body, r'gaJsHost.*\.google-analytics\.com/ga\.js') THEN "sync"
ELSE "other"
END as stat
FROM [httparchive:runs.2014_08_15_requests_body]
WHERE mimeType CONTAINS 'html'
AND REGEXP_MATCH(body, r'\.google-analytics\.com/ga\.js')
GROUP BY stat
ORDER BY num desc
```
A few seconds later, after processing 200GB of HTML responses:
Note that the `body` tables are large (~200GB), so its easy to exceed [your free quota](https://cloud.google.com/bigquery/pricing#queries) if you’re not careful. With that in mind, a few tips:
- If you’re only interested in particular content-type, restrict your queries by `mimeType` to speed up processing - e.g. above query checks for “html”.
- While you’re iterating on your query, add a `LIMIT X` clause to avoid executing it against the full dataset.
_P.S. At Velocity EU I also [demoed “user defined functions” inside of BigQuery](https://www.youtube.com/watch?v=_CMcaYnBt-g) - i.e. ability to execute arbitrary JavaScript functions against each record. However, this feature is still in beta and is not yet publicly available. Stay tuned, I’ll share more information and example queries for it once we iron out a few outstanding issues._
---
**Author:**  [@kevinSuttle](https://discuss.httparchive.org/u/kevinSuttle)\
**Post date:** [September 1, 2016, 7:33pm UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/2 "2016-09-01T19:33:01Z")
Is there a way to query the actual content of these pages in terms of HTML, CSS, and JS?
---
**Author:**  [@patmeenan](https://discuss.httparchive.org/u/patmeenan)\
**Post date:** [September 2, 2016, 12:21pm UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/3 "2016-09-02T12:21:17Z")
Yes, if you run queries against the [HAR data set](https://discuss.httparchive.org/t/quickstart-guide-to-exploring-the-http-archive/682) it has the response bodies for text resources (HTML, CSS, JS)
---
**Author:**  [@kevinSuttle](https://discuss.httparchive.org/u/kevinSuttle)\
**Post date:** [September 2, 2016, 3:00pm UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/4 "2016-09-02T15:00:13Z")
Thanks, Pat. I’m looking to evaluate class names in HTML.
Also does this take into account markup rendered via JS?
---
**Author:**  [@patmeenan](https://discuss.httparchive.org/u/patmeenan)\
**Post date:** [September 2, 2016, 5:28pm UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/5 "2016-09-02T17:28:06Z")
No (well, unless you pull the code out of the js). It is providing the raw responses from the wire.
---
**Author:**  [@kevinSuttle](https://discuss.httparchive.org/u/kevinSuttle)\
**Post date:** [September 8, 2016, 1:12am UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/6 "2016-09-08T01:12:17Z")
Sorry, I mixed 2 questions—the first being the more important.
> I’m looking to evaluate class names in HTML.
Has someone already done this, and if not, do you know if it’s possible?
---
**Author:**  [@zcorpan](https://discuss.httparchive.org/u/zcorpan)\
**Post date:** [September 9, 2016, 10:31am UTC](https://discuss.httparchive.org/t/analyzing-html-css-and-javascript-response-bodies/442/7 "2016-09-09T10:31:58Z")
See e.g. [Usage of ARIA attributes](https://discuss.httparchive.org/t/usage-of-aria-attributes/778) for how you could write such a query.