# Sites that deliver Images using gzip/deflate encoding **URL:** \ **Category:** Analysis\ **Created:** [April 16, 2014, 6:27pm UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220 "2014-04-16T18:27:53Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![pganti](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/pganti/32/1299_2.png) [@pganti](https://discuss.httparchive.org/u/pganti)\ **Post date:** [April 16, 2014, 6:27pm UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/1 "2014-04-16T18:27:53Z")
From the standard [best practices for reducing payload](https://developers.google.com/speed/docs/best-practices/payload) we know the following rule: Don’t use gzip for image or other binary files. However lets see if this is true in practice and if not what are the sites that do use gzip/deflate for their images using the following sample query ``` select count(requestid) as ct, domain(url) as dom from [httparchive:runs.latest_requests] where (resp_content_type='image/png' or resp_content_type='image/jpeg' or resp_content_type='image/jpg') and resp_content_encoding in ('gzip', 'deflate') group by dom having ct > 333 order by ct desc limit 20 ``` shows up the following domains ![](https://yyz1.discourse-cdn.com/flex035/uploads/httparchive/167/73f7e941a78e3819.png) We can drill down into specific URLs for the domain using the following query ``` select url from [httparchive:runs.latest_requests] where resp_content_type='image/jpeg' and resp_content_encoding in ('gzip', 'deflate') and domain(url) = 'facebook.com' ``` Since the top one happens to ‘[external.ak.fbcdn.net](http://external.ak.fbcdn.net)’ which looks like Facebook but is served from Akamai. Funny thing is that we can verify that the “Content-Encoding:gzip” is actually added by the Akamai config not the original content publisher as seen here: Content Publisher: [http://www.webpagetest.org/result/140416\_1P\_64S/1/details/#request1](http://www.webpagetest.org/result/140416_1P_64S/1/details/#request1) FB Shared Content: [http://www.webpagetest.org/result/140416\_BB\_675/1/details/#request1](http://www.webpagetest.org/result/140416_BB_675/1/details/#request1) Lets pick Slideshare for example which does use Akamai as their CDN but their origin is the one setting gzip headers on the jpeg (maybe a misconfig on their end) Origin: [http://www.webpagetest.org/result/140416\_33\_69J/1/details/#request1](http://www.webpagetest.org/result/140416_33_69J/1/details/#request1) CDN : [http://www.webpagetest.org/result/140416\_6T\_69Q/1/details/#request1](http://www.webpagetest.org/result/140416_6T_69Q/1/details/#request1) Finally given that [facebook.com](http://facebook.com) existed on top20 domains doing this, I picked the profile pic url: [http://www.webpagetest.org/result/140416\_S2\_SSX/1/details/#request1](http://www.webpagetest.org/result/140416_S2_SSX/1/details/#request1) I was a wondering if we pay any decompression costs (high on mobile relative to desktop), so ran the same query for mobile dataset it returns 0 records which is good news as no sites in httparchive do gzip compression on images ``` select count(requestid) as ct, domain(url) as dom from [httparchive:runs.latest_requests] where (resp_content_type='image/png' or resp_content_type='image/jpeg' or resp_content_type='image/jpg') and resp_content_encoding in ('gzip', 'deflate') group by dom having ct > 333 order by ct desc limit 20 ``` Net net of all this is that I can conclude most of these just misconfigurations (as in the generating endpoint democratically compresses without regards to content type) Is there any other harm I am not aware of when compressing images? ---
**Author:** ![igrigorik](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/igrigorik/32/925_2.png) [@igrigorik](https://discuss.httparchive.org/u/igrigorik)\ **Post date:** [April 16, 2014, 10:27pm UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/2 "2014-04-16T22:27:43Z")
@pganti nice analysis. > Is there any other harm I am not aware of when compressing images? Unnecessary CPU overhead on the server and client… and in some cases, extra bytes due to gzip headers. 🙂 ---
**Author:** ![guypod](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/guypod/32/1301_2.png) [@guypod](https://discuss.httparchive.org/u/guypod)\ **Post date:** [April 16, 2014, 11:45pm UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/3 "2014-04-16T23:45:34Z")
Interesting finding. A couple of small comments: (1) It actually looks like Facebook adds the gzip at Facebook origin, as you can see from requesting the image through [www.facebok.com](http://www.facebok.com) (which is not, as far as I can tell, on Akamai): [http://www.webpagetest.org/result/140416\_P8\_8b5b21a97ca0921a78924ef2305bbdf4/1/details/#request1](http://www.webpagetest.org/result/140416_P8_8b5b21a97ca0921a78924ef2305bbdf4/1/details/#request1) (2) That said, you _can_ make Akamai gzip your images, but it’s not default (a customer can state which content types should be compressed) I’ve actually seen cases where gzipping jpeg images reduced their size. However, arguably that’s just the result of incorrect encoding of the images, for instance leaving metadata in there. I can envision cases where you have to keep metadata on your images for various reasons, and it’s plausible that in those cases gzip might help, but that seems like a very rare scenario. ---
**Author:** ![cryopng](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/cryopng/32/1298_2.png) [@cryopng](https://discuss.httparchive.org/u/cryopng)\ **Post date:** [April 17, 2014, 1:01am UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/4 "2014-04-17T01:01:55Z")
By nature some mandatory JPEG headers like the quantization tables are relatively easy to compress with gzip, but once you enter the body of the image which is huffman encoded gzip will no longer be able to work efficiently. This could be tested using gzthermal, look for DQT (Define Quantization Table) and SOFx (Start of Frame) markers in the compressed stream (edit: unlike PNG, JPEG markers are not in ASCII but their position in the file can be retrieved with tools like JPEGsnoop). Here is a sample heatmap view produced by gzthermal when fed with a .jpg.gz file: [http://encode.ru/threads/1889-gzthermal-pseudo-thermal-view-of-Gzip-Deflate-compression-efficiency?p=37584&viewfull=1#post37584](http://encode.ru/threads/1889-gzthermal-pseudo-thermal-view-of-Gzip-Deflate-compression-efficiency?p=37584&viewfull=1#post37584) It is possible to losslessly recompress JPEGs. The fastest solution (a part from the usual sequential to progressive scan conversion) would be to switch to arithmetic coding instead of Huffman encoding (saves about 7%), but arithmetic coding is not well supported (historically a patent issue). The huge advantage of this technique is that it is damn easy to reencode to Huffman if compatibility with legacy JPEG decoders is important. A more evolved one would be to use specific tools like PackJPG [http://www.elektronik.htw-aalen.de/packjpg/](http://www.elektronik.htw-aalen.de/packjpg/) Unfortunately Gzip/Deflate is way to old to include such a sophisticated recompression sheme, it does not even cope specifically with Base64 encoded (HTML/CSS inlined) blobs since it is only a generic data compressor. ---
**Author:** ![doug\_sillars](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/doug_sillars/32/1297_2.png) [@doug\_sillars](https://discuss.httparchive.org/u/doug_sillars)\ **Post date:** [April 17, 2014, 3:51am UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/5 "2014-04-17T03:51:52Z")
Paddy, cool study. If you cut out the ct\>333 limit for mobile, there are sites with Gzip for images: ![](https://yyz1.discourse-cdn.com/flex035/uploads/httparchive/168/c71d6c89170b2079.png) The mobile data set is so small that if you remove the ``` and resp_content_encoding in ('gzip', 'deflate') ``` line, the max # of responses from one domain is only ~800. Note above that I am showing the host (not the domain). The [external.ak.fbcdn.net](http://external.ak.fbcdn.net) is the culprit on the web side as well. Is it possible that since these images are used externally that they have this added header? Or perhaps tested differently? ---
**Author:** ![pganti](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/pganti/32/1299_2.png) [@pganti](https://discuss.httparchive.org/u/pganti)\ **Post date:** [April 17, 2014, 4:44am UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/6 "2014-04-17T04:44:25Z")
@doug_sillars: Good Catch. I did put the filter of 333 for certain traffic level but you are right in identifying mobile sites that do this. FB seems to be the main one. ---
**Author:** ![pganti](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/pganti/32/1299_2.png) [@pganti](https://discuss.httparchive.org/u/pganti)\ **Post date:** [April 17, 2014, 4:46am UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/7 "2014-04-17T04:46:48Z")
@guypod: Thank you for the clarification. It does seem clear that its an origin issue. I shall notify our friends at FB to take some action on this. ---
**Author:** ![josephscott](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/josephscott/32/1300_2.png) [@josephscott](https://discuss.httparchive.org/u/josephscott)\ **Post date:** [April 17, 2014, 5:33pm UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/8 "2014-04-17T17:33:47Z")
The two Facebook images you reference are the same image, but they aren’t the same size. That makes comparing them kind of pointless. ---
**Author:** ![pganti](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/pganti/32/1299_2.png) [@pganti](https://discuss.httparchive.org/u/pganti)\ **Post date:** [April 17, 2014, 6:03pm UTC](https://discuss.httparchive.org/t/sites-that-deliver-images-using-gzip-deflate-encoding/220/9 "2014-04-17T18:03:53Z")
@josephscott: I understand. But the key import of doing that example is to look at who is adding the header in the first place not so much as the content. Even if you change the size the header remains. FB has been notified of the issue and are actively investigating the fix and I hope by the next crawl we should see fb removed from the list