# Downloading HAR-Datasets later than May 2022?

**URL:** <https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588>\
**Category:** Meta\
**Created:** [June 9, 2023, 10:46am UTC](https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588 "2023-06-09T10:46:48Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![researcher](https://avatars.discourse-cdn.com/v4/letter/r/edb3f5/32.png) [@researcher](https://discuss.httparchive.org/u/researcher)\
**Post date:** [June 9, 2023, 10:46am UTC](https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588/1 "2023-06-09T10:46:48Z")

</div>

Hi everyone,

for scientific research, we would like to download the HAR or summary data from the httparchive folder as stated in the tutorial [1](https://discuss.httparchive.org/t/how-to-download-the-http-archive-data/679).

However, when checking the available datasets for download [2](https://console.cloud.google.com/storage/browser/httparchive?prefix=chrome), it only shows the downloads until May 2022. Was this stopped somehow or is the data still available somewhere? I cannot use BigQuery to get the data, as the free tier seems not to be enough here.

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [June 24, 2023, 11:46pm UTC](https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588/2 "2023-06-24T23:46:52Z")

</div>

Hi @researcher 👋

Last year we moved from individual top-level crawl directories to a `crawls` parent directory. You can access later HARs in GCS here: [https://console.cloud.google.com/storage/browser/httparchive/crawls](https://console.cloud.google.com/storage/browser/httparchive/crawls)

---

<div class="post-metadata">

**Author:** ![kzarifis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/kzarifis/32/664_2.png) [@kzarifis](https://discuss.httparchive.org/u/kzarifis)\
**Post date:** [October 12, 2023, 11:10pm UTC](https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588/3 "2023-10-12T23:10:19Z")

</div>

Hi @rviscomi , I was looking for the same and found your answer here, but I have a followup question: is there a way to download the full ‘pages’ data for one month as a zipped single file? I think remember doing that years ago from some legacy links, but now I only see the option to download HARs as separate objects from the buckets you mentioned above.

Alternatively, is there a way to download e.g. only the HARs for the top 1K pages? I was looking at [Minimizing query costs](https://har.fyi/guides/minimizing-costs/) that suggests using ‘rank=1000’ for this, but looks like if I want to grab the payload from bigquery, it’s still a very expensive query

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [October 16, 2023, 4:43pm UTC](https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588/4 "2023-10-16T16:43:17Z")

</div>

Here’s an example query that extracts the HAR info for the top 1000 pages in September:

```sql
SELECT
  client,
  page,
  payload AS HAR
FROM
  `httparchive.all.pages`
WHERE
  date = '2023-09-01'
  AND is_root_page
  AND rank = 1000

```

The actual number of bytes billed for this query is only 941 MB (thanks to the `WHERE` clause). I think there’s a bug in BigQuery where the estimate is inaccurate for anyone who isn’t a project editor, but rest assured this is a relatively inexpensive query.

After you’ve run the query, you have a few options to save the data locally or to Google Drive.

---

<div class="post-metadata">

**Author:** ![kzarifis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/kzarifis/32/664_2.png) [@kzarifis](https://discuss.httparchive.org/u/kzarifis)\
**Post date:** [October 16, 2023, 7:09pm UTC](https://discuss.httparchive.org/t/downloading-har-datasets-later-than-may-2022/2588/5 "2023-10-16T19:09:57Z")

</div>

Indeed, it was the estimate that was scaring me. Thanks for clarifying that!