# Determining the frequency of apng

**URL:** <https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345>\
**Category:** Analysis\
**Created:** [May 1, 2018, 6:51am UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345 "2018-05-01T06:51:57Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![patrickkettner](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/patrickkettner/32/548_2.png) [@patrickkettner](https://discuss.httparchive.org/u/patrickkettner)\
**Post date:** [May 1, 2018, 6:51am UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/1 "2018-05-01T06:51:58Z")

</div>

While apng images do technically have their own mime-type (image/apng), it does not show up at all in the archive. Every instance I do find through googling seems to ship them as .png images, with image/png headers. You can determine if a png image is apng by checking whether or not the string “acTL” occurs in the file, but am not how or if I can write a query to find instances of apng images

Is it possible?

---

<div class="post-metadata">

**Author:** ![paulcalvano](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/paulcalvano/32/1601_2.png) [@paulcalvano](https://discuss.httparchive.org/u/paulcalvano)\
**Post date:** [May 1, 2018, 1:46pm UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/2 "2018-05-01T13:46:12Z")

</div>

Hi @patrickkettner. I haven’t seen APNG images shipped with an APNG extension either, and looking at the HTTP Archive requests table it looks like there are only 2 objects with an apng extension.

```sql
SELECT url
FROM `httparchive.summary_requests.2018_04_01_desktop`
WHERE ext = "apng"

```

The HTTP Archive does capture response payloads for some objects (html, css, js), but binary images are not included in that dataset. So there is no way to specifically search for the string `acTL` in the payload of the .png responses via BigQuery.

An alternative solution for you to find this information is to extract a list of all PNGs…

```sql
SELECT distinct url
FROM `httparchive.summary_requests.2018_04_01_desktop`
WHERE url LIKE "%.png%" and respBodySize > 1024

```

or a random sample of PNGs…

```sql
SELECT url
FROM (
   SELECT distinct url, rand() as random
   FROM `httparchive.summary_requests.2018_04_01_desktop`
   WHERE url LIKE "%.png%" and respBodySize > 1024
   ORDER BY random ASC
)
LIMIT 100000

```

And then download the results to a CSV file so that you can process them outside of BigQuery. For example, if I were to process this list using some bash commands, the following command would

- Loop through the data files outputted from the above queries
- perform a curl request, outputting the response body to stdout
- set a timeout on the curl request to 5s to avoid timeouts stalling you
- grep for the payload in the response
- output a url only if the “acTL” payload is detected

```sql
for i in $(cat png.csv); do curl -sk -m 5 $i 2>&1 | grep -q "acTL" && echo $i; done 

```

Hope that helps! Let us know if you find anything interesting 🙂

---

<div class="post-metadata">

**Author:** ![patmeenan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/patmeenan/32/605_2.png) [@patmeenan](https://discuss.httparchive.org/u/patmeenan)\
**Post date:** [May 1, 2018, 2:23pm UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/3 "2018-05-01T14:23:38Z")

</div>

If it would be useful in an ongoing basis I’d recommend filing an issue to report the “sniffed content type” or something like that and we can add support to WebPageTest to report content types based on the binary files (assuming it is sniffable from the first few bytes of the file).

The agents already sniff the image types as part of the optimization checks so it doesn’t feel like a stretch to add, it’s just not currently possible.

---

<div class="post-metadata">

**Author:** ![svgeesus](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/svgeesus/32/1145_2.png) [@svgeesus](https://discuss.httparchive.org/u/svgeesus)\
**Post date:** [May 1, 2018, 10:30pm UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/4 "2018-05-01T22:30:01Z")

</div>

image/apng has not been registered, although image/vnd.mozilla.apng has:  
[https://www.iana.org/assignments/media-types/media-types.xhtml#image](https://www.iana.org/assignments/media-types/media-types.xhtml#image)

I agree that knowing the usage of apng would be very useful.

---

<div class="post-metadata">

**Author:** ![patrickkettner](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/patrickkettner/32/548_2.png) [@patrickkettner](https://discuss.httparchive.org/u/patrickkettner)\
**Post date:** [May 2, 2018, 4:57am UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/5 "2018-05-02T04:57:33Z")

</div>

@paulcalvano - cheers. I downloaded the full list and am downloading the 4.8M pngs as I type.

@patmeenan It _would_ be useful. First few examples that come up when you google apng have the acTL magic string ~40 bytes into the file. Not sure if that counts as the first few bites.

@svgeesus ah shoot, my bad. I could have sworn it was registered. Thanks for the correction!

---

<div class="post-metadata">

**Author:** ![svgeesus](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/svgeesus/32/1145_2.png) [@svgeesus](https://discuss.httparchive.org/u/svgeesus)\
**Post date:** [May 2, 2018, 4:27pm UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/6 "2018-05-02T16:27:04Z")

</div>

It isn’t registered because it isn’t part of the PNG spec, having failed a vote years back.

More recently I asked for new discussion and a re-vote because of widespread adoption. Little interest.

---

<div class="post-metadata">

**Author:** ![patmeenan](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/patmeenan/32/605_2.png) [@patmeenan](https://discuss.httparchive.org/u/patmeenan)\
**Post date:** [May 2, 2018, 4:40pm UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/7 "2018-05-02T16:40:57Z")

</div>

apng specifically should be easy enough since the optimization check actually walks the entire PNG looking for non-image tags as bloat. Finding PNGs with the acTL chunk (anywhere in them) should be easy enough. May be able to have something for the 5/15 crawl, just need to figure out where to plumb it into the results.

---

<div class="post-metadata">

**Author:** ![patrickkettner](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/patrickkettner/32/548_2.png) [@patrickkettner](https://discuss.httparchive.org/u/patrickkettner)\
**Post date:** [May 20, 2018, 11:26am UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/8 "2018-05-20T11:26:54Z")

</div>

Processing finally finished.

[here is a list](https://drive.google.com/open?id=1Tldyhp7M7QG0l4yNo7gIGwcE7bbnxKIu) of the 4,437,360 files that hit on @paulcalvano’s [query](https://bigquery.cloud.google.com/savedquery/428216393065:a2b44cefb71843b3a4b9354962e2ee12) ([gziped version](https://drive.google.com/open?id=1_o3Ky8KSN5IQWcOqPtNUaa7IGNK3jUCb)).

I split the list into several chunks to parallelize, and then ran the following to filter

```
while read pngURL; do curl -sk -m 5 -r 0-99 $pngURL 2>&1 | grep -q acTL && echo $pngURL >> /tmp/results-apng ; done < png_list

```

The final result of that is available [here](https://gist.github.com/patrickkettner/c6ad4d63d5af8c7cb9ee61a95672090a).

The answer to my original question is that apngs are ~0.00275% (122 of the 4.4m) of PNG files.

---

<div class="post-metadata">

**Author:** ![patrickkettner](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/patrickkettner/32/548_2.png) [@patrickkettner](https://discuss.httparchive.org/u/patrickkettner)\
**Post date:** [May 20, 2018, 3:31pm UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/9 "2018-05-20T15:31:01Z")

</div>

Given the URL of a PNG file, how would I be able to get the domain of the page that requested the image?

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [May 21, 2018, 3:59am UTC](https://discuss.httparchive.org/t/determining-the-frequency-of-apng/1345/10 "2018-05-21T03:59:42Z")

</div>

> [@patrickkettner](#):
>
> Given the URL of a PNG file, how would I be able to get the domain of the page that requested the image?

For example:

```sql
SELECT
  DISTINCT NET.REG_DOMAIN(page) 
FROM
  `httparchive.requests.2018_04_01_desktop`
WHERE
  url = 'http://maps.gstatic.com/mapfiles/transparent.png'

```

Multiple pages may request the same image though, so expect multiple results.