# How are JavaScript files being served and how many are there?

**URL:** <https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36>\
**Category:** Analysis\
**Created:** [July 1, 2013, 6:17pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36 "2013-07-01T18:17:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![igrigorik](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/igrigorik/32/925_2.png) [@igrigorik](https://discuss.httparchive.org/u/igrigorik)\
**Post date:** [July 1, 2013, 6:17pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36/1 "2013-07-01T18:17:25Z")

</div>

**How many scripts are being fetched from the non-HTML / non-root domain?**

```sql
SELECT
 NTH(50, quantiles(num_scripts,101)) median,
 NTH(75, quantiles(num_scripts,101)) seventy_fifth,
 NTH(90, quantiles(num_scripts,101)) ninetieth,
 NTH(95, quantiles(num_scripts,101)) ninety_fifth
FROM (
  SELECT req.pageid, COUNT(req.url) as num_scripts
  FROM [httparchive:runs.latest_requests] as req JOIN (  
    SELECT HOST(url) self, pageid
    FROM [httparchive:runs.latest_pages]
  ) as pages ON pages.pageid = req.pageid
  WHERE HOST(req.url) != pages.self
    AND LOWER(mimeType) CONTAINS "script"
  GROUP BY req.pageid
)

```

Based on June 2013 data:  
 ![](https://yyz1.discourse-cdn.com/flex035/uploads/httparchive/13/e261a17e7cfb9b0d.png)

We can also ask the reverse (just remove the ! in the WHERE clause), which is to say… **How many scripts are being loaded from the same host?**  
 ![](https://yyz1.discourse-cdn.com/flex035/uploads/httparchive/14/f0fffff888fcda7f.png)

**Are sites sharding scripts on sub-hostnames of HTML domain?**

```sql
SELECT
 NTH(50, quantiles(num_scripts,101)) median,
 NTH(75, quantiles(num_scripts,101)) seventy_fifth,
 NTH(90, quantiles(num_scripts,101)) ninetieth,
 NTH(95, quantiles(num_scripts,101)) ninety_fifth
FROM (
  SELECT req.pageid, COUNT(req.url) as num_scripts
  FROM [httparchive:runs.latest_requests] as req JOIN (  
    SELECT DOMAIN(url) self, pageid
    FROM [httparchive:runs.latest_pages]
  ) as pages ON pages.pageid = req.pageid
  WHERE DOMAIN(req.url) = pages.self
    AND LOWER(mimeType) CONTAINS "script"
  GROUP BY req.pageid
)

```

A few more scripts in the tail, showing that some sites are indeed sharding on root domain…  
 ![](https://yyz1.discourse-cdn.com/flex035/uploads/httparchive/16/e1e62700999e325e.png)

So the median website has 13 scripts, 5 from the same host as the main page, and 8 from some other host. We now know the number of scripts, but **from how many distinct hosts are they being fetched?**

```sql
SELECT
 NTH(50, quantiles(num_hosts,101)) median,
 NTH(75, quantiles(num_hosts,101)) seventy_fifth,
 NTH(90, quantiles(num_hosts,101)) ninetieth,
 NTH(95, quantiles(num_hosts,101)) ninety_fifth
FROM (
 SELECT req.pageid, COUNT(DISTINCT HOST(req.url), 1000000) as num_hosts
 FROM [httparchive:runs.latest_requests] as req JOIN (  
   SELECT HOST(url) self, pageid
   FROM [httparchive:runs.latest_pages]
 ) as pages ON pages.pageid = req.pageid
 WHERE HOST(req.url) != pages.self
   AND LOWER(mimeType) CONTAINS "script"
 GROUP BY req.pageid
)

```

![](https://yyz1.discourse-cdn.com/flex035/uploads/httparchive/15/5ccec6a7adc3136a.png)

Looking at the median, the 8 scripts requested on hosts different than the main page are requested across 5 different domains. Yikes!

---

<div class="post-metadata">

**Author:** ![andydavies](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/andydavies/32/1306_2.png) [@andydavies](https://discuss.httparchive.org/u/andydavies)\
**Post date:** [July 1, 2013, 8:22pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36/2 "2013-07-01T20:22:49Z")

</div>

It’s a pity there’s no easy way to dig into the pages to understand, when and how the scripts are being loaded i.e. are they async, are they being loaded late enough in the page to not affect the visitors experience.

I can probably guess the answers to both questions but would like a way of proving it.

---

<div class="post-metadata">

**Author:** ![igrigorik](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/igrigorik/32/925_2.png) [@igrigorik](https://discuss.httparchive.org/u/igrigorik)\
**Post date:** [July 1, 2013, 9:52pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36/3 "2013-07-01T21:52:10Z")

</div>

@andydavies actually, I think we could get some pretty good estimates. Not sure if the time resolution is good enough, but `pages` table contains the `startedDateTime`, plus `onload` and `fullyLoaded`. Similarly, we have the `startedDateTime` on requests… So, in theory, we could use the offsets to determine when the request is being dispatched - e.g. after onload or before.

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [April 27, 2017, 6:15pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36/4 "2017-04-27T18:15:32Z")

</div>

> [@igrigorik](#):
>
> NTH(50, quantiles(num\_hosts,101)) median

Sanity check, wouldn’t the median be the 51st record of 101 records?

---

<div class="post-metadata">

**Author:** ![igrigorik](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/igrigorik/32/925_2.png) [@igrigorik](https://discuss.httparchive.org/u/igrigorik)\
**Post date:** [April 27, 2017, 9:19pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36/5 "2017-04-27T21:19:37Z")

</div>

In retrospect, the whole 101 bit is silly, you should probably ignore it. The context is:  
[https://cloud.google.com/bigquery/docs/reference/legacy-sql#nth](https://cloud.google.com/bigquery/docs/reference/legacy-sql#nth)

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [May 2, 2017, 8:30pm UTC](https://discuss.httparchive.org/t/how-are-javascript-files-being-served-and-how-many-are-there/36/6 "2017-05-02T20:30:53Z")

</div>

I think it still makes sense. The QUANTILES docs say to use 101 for percentiles. I’ve typically used NTH(501, QUANTILES(expr, 1001)) for added accuracy.