Did a quick search through the request_bodies table, which contains HTML, JS, CSS, etc. Simple search for the pattern `"%document.domain%". The query is below, but a quick warning that this wlll process 762GB of data (free tier is 1TB / mo).
SELECT count(distinct page) pages, count(distinct url) unique_urls, count(*) requests
FROM `httparchive.har.2017_10_15_chrome_requests_bodies`
WHERE body LIKE "%document.domain%"
It’s amazing that you can process this much data in 14 seconds :). Results are below and show that there are 278,398 pages that contain at least 1 reference to document.domain. There are 276,865 unique URLs and 1,031,136 requests total - which hints that there are quite a few requests shared across sites (ie, common third parties) -
I refined this query to show what some of the most common urls were that contain document.domain. The query for this is below (and will also process 762GB of data)
SELECT url, count(distinct page) pages, count(*) requests
FROM `httparchive.har.2017_10_15_chrome_requests_bodies`
WHERE body LIKE "%document.domain%"
GROUP BY url
ORDER BY requests DESC
And the results show requests to facebook, google ads, youtube, twitter, doubleclick, etc -
1 Like
The above query is telling us where document.domain is used. But using REGEXP_EXTRACT_ALL we can extract parts of the JavaScript where document.domain is being used to get an idea of how it is used in the wild.
This query uses a regular expression to extract the text between document.domain and either a ; or ) character. I’ve also padded the regex match with 10 characters on either side to give us an idea of how this is being used. As with the other examples above, this query will process 760GB of data…
SELECT docdomain, count(*) as freq
FROM (
SELECT REGEXP_EXTRACT_ALL(LOWER(body), r'(.{10}document\.domain[^;|\)]+.{10})') dd
FROM `httparchive.har.2017_10_15_chrome_requests_bodies`
WHERE body LIKE "%document.domain%"
) docdom_array
CROSS JOIN UNNEST(docdom_array.dd) docdomain
GROUP BY docdomain
ORDER BY freq DESC
Here are some of the results -
And a longer excerpt containing the top 200 results -
y><script>document.domain = 'facebook.com';__transfo
='<script>document.domain="'+document.domain+'";\x3c/scri
"nction() {document.domain = \""""+
document.domain+'"";var conti"
nction() {document.domain = \""+document.domain+'";var conti
onion)$/);document.domain=captures?captures[1]:"facebook.com";(function
tion(){if(document.domain==i)return tr
e,f,g){if(document.domain.tolowercase().match(/(
ction j(){document.domain=i}f.exports=j}),18);
tch (e) { document.domain="'+a.domain+'"; }',"wind
/"+window.document.domain+"/post_login";a&&(c=og(
ascript\">document.domain = \""+window.document.domain+'";'+g+"<\\/
e(); };":'document.domain = "'+window.document.domain+'";'+g+"docu
='<script>document.domain="'+e+'"\x3c/script>'}d+="</body></html>";c=g.zc(g.
ostname===document.domain&&e.port===t.port;return r.
n(){var b=document.domain.split(/\./);var a=""
"cation if document.domain has been set
try {
ajaxlocation = location.href;"
t.domain||document.domain.search(/\.gov/)>0){retur
t.open(); document.domain="'+document.domain+'"; window.p
ostname===document.domain&&n.port===e.port;return r.
\"<script>document.domain='"+document.domain+"';parent['"
==am){try{document.domain=az}catch(bl){}bi=av(a
===0){try{document.domain=az}catch(be){}x=av(ay
bh,bo;try{document.domain=az}catch(bu){}for(;bi
}else{try{document.domain=az}catch(be){}af={};a
.msie6 || document.domain != locdomain) document
nt.open();document.domain='{domain}';document.
\'<script>document.domain=\\\""+document.domain+"\\\"<\\\\/script>\\');document
" domain: document.domain,
port: location.port ? ':' + location.port : ''"
nt.open();document.domain='"+document.domain+"';document.
var n=0,e=document.domain,i=e.split('.'),t='_gd'+
ew regexp(document.domain,"gi");if(b.tes
(),domain=document.domain,test=false,sep="/",delim="/",autofire=false;var portn
urn eval("document.domain")},functio
/"+window.document.domain+"/post_login";a&&(c=pg(
domain='"+document.domain+"';"),b.push
"<body>"),document.domain!==location.hostname&&(b.push(a.open),b.push("
).replace(document.domain,"").replace(
n(t){t=t||document.domain,0===t.indexof("www.")&&(t=t.su
ostname!==document.domain&&e.push("document.domain overridden"),window.f
='<script>document.domain="'+e+'"\x3c/script>'}d+="</body></html>";c=zc(vc("
ostname===document.domain&&e.port===n.port;return r.
"cation if document.domain has been set
try {
ajaxlocation = location.href;
"
"nt.open();document.domain='""+
document.domain+""';document."
ocol+"//"+document.domain+"/post_login";a=df(a,"m
t.domain||document.domain.search(/\.gov/)>0)return
n(){var e=document.domain.split(/\./),t="";ret
/",u.isie?document.domain:"")}function
ostname===document.domain&&e.port===n.port&&e.protocol===n.protocol,r}},{}],13:[function(t,e){function
nt.open();document.domain="'+document.domain+'";document.
s._window.document.domain+e;var a=nul
ol,domain:document.domain,port:location.port?':'+location.port:''}));},gethe
y><script>document.domain=\\'"+q.domain+"\\';<\\/scrip
var c="."+document.domain.split("www.").join("")
atch(e){n=document.domain,r.src='javascript:var d=document.open();d.domain
(a,b,c,d){document.domain!=window.location.hostname&&d&&!f(11)?cq(a,b,c
(a.parent.document.domain!=a.document.domain)return c.
"nt.open();document.domain=""'+
window.document.domain+""\"";document."
;else{try{document.domain=document.domain}catch(p){}n=encod
c.com"}if(document.domain.indexof(".ssl.cf2.rackcdn.com")>-1){docu
slave){if(document.domain.indexof(".cxpublic.com")>-1){docu
(b.parent.document.domain!=b.document.domain)return a.
var t=0,e=document.domain,n=e.split("."),r="_gd"+
('<script>document.domain="'+f+'"\x3c/script>'),d.close(
{},ao=0,h=document.domain,d,q=function(){var bc=n
indow.top.document.domain==window.document.domain)return a.
domain='"+document.domain+"';d.write('
}catch(e){document.domain="'+document.domain+'";}\r');a.s
lse try{b.document.domain==f.document.domain&&(d=!0)}catch(l)
d><script>document.domain=\\'"+ntv.postrelease.gettopwindow().document
.d){var c=document.domain,d=(lb?"https://":"http://")+za("","p
if(typeof document.domain=="undefined"||typeof d
{a=window.document.domain==this.ia.document.domain}catch(d){a=!1}a&&
j(){var a=document.domain.split("."),q=a.leng
.query,b=(document.domain.split(".").length>2
eption) { document.domain = document.domain.replace(/^www./,""); }}funct
"um=""+n+""; document.domain=\""""
+d.domain+""\"";""+s+""<\/s"
"l)):(d.bs=document.domain,d.ebptcl=this._getprotocol(document.location.protocol));
"
gdomain)e=document.domain,f=e+location.pathname,e&&0<e.length&&(g=0<g.indexof(f)?g.replac
('<script>document.domain="'+t+'"<\/script>');u.close(
('<script>document.domain=\""+document.domain+"\"</script>')",e.conte
ascript\">document.domain = \""+document.domain+'";'+g+"<\\/
l><script>document.domain="'+document.domain+'";</script>
(a,b,c,d){document.domain!=window.location.hostname&&d&&!f(11)?eq(a,b,c
e(); };":'document.domain = "'+document.domain+'";'+g+"docu
ostname===document.domain&&t.port===e.port;return r.
ht,domain:document.domain,id:n.response.callback_uid},a={ssp:"appnexus",winbid:number((n.response.result.cpm/1e4).tofixed(
ascript\">document.domain = \""+document.domain+'";'+r+"<\\/
" modified document.domain: ""+
ma.message)}finally{"
e(); };":'document.domain = "'+document.domain+'";'+r+"docu
)&&dc(b)!=document.domain?new re:new te(void 0,void 0);k(c,"suc
)&&yg(b)!=document.domain?new fn:new hn(void 0,c)};functio
nt.open();document.domain="'+document.domain+'";var ed =
{a=window.document.domain==this.y.document.domain}catch(b){a=!1}a?t
('<script>document.domain="'+f+'"</script>'),e.close(
ol,domain:document.domain,port:location.port?":"+location.port:""}))},gethea
('<script>document.domain="'+domain+'"<\/script>'),iframe_d
atch(c){n=document.domain,r.src='javascript:var d=document.open();d.domain
"t.open(); document.domain=""'+
document.domain+'""; window.p"
/",domain:document.domain},!0);if(d.che
document.domain = 'fout.jp';
nt.open();document.domain="'+window.document.domain+"\";document.
ht,domain:document.domain};this.done
ze,domain:document.domain,auctionid:e.result.reqid,dmtagid:e.result.id},a='<img src="'+n+'" width="1" height="1" style="display:none;">',o=new
ang_frame.document.domain!==boomr.window.document.domain&&(boomr.boomerang_frame.document.domain=boomr.window.document.domain)}catch(c)
ain=" + w.document.domain + (expires ? " ; expires=
indow.top.document.domain!==window.document.domain}catch(t){}return!
atch(a){e=document.domain,g.src="javascript:var d=document.open();d.domain
;else{try{document.domain=document.domain}catch(y){}var i=e
domain==k.document.domain}catch(va){p=!1}}p=
}catch(c){document.domain=a}try{b=window.parent.document;return}ca
infix")}a=document.domain}if(-1!==a.indexof(".")){try{b=w
domain='"+document.domain+"';void(0);"
entdomain=document.domain.tolowercase(),referdom
='<script>document.domain="'+document.domain+'";<\/script
tdomain = document.domain.tolowercase(),
ol,domain:document.domain,port:location.port?":"+location.port:""})},gethead
main||"."+document.domain.replace(/^www\./,""),!0===n.s
('<script>document.domain="'+f+'"<\/script>'),e.close(
('<script>document.domain="'+g+'"\x3c/script>'),c.close(
('<script>document.domain="'+g+'"\x3c/script>'),d.close(
.listname,document.domain,t.blacklist.map(function(t){return n
?"<script>document.domain='"+document.domain+"';</script>
nt.open();document.domain='"+h.domain+"';",a.src=e
="";try{d=document.domain}catch(e){}a.index
ostname===document.domain&&i.port===t.port;return a.
<strong>'+document.domain+"</strong>. clear your browser cache and reload this page afterwards.",d="the domain "+document.domain+" is not allowed to include this script. please check your oneall site security settings.";
{a=window.document.domain==this.t.document.domain}catch(b){a=!1}a&&
n(){var e=document.domain,t=e;try{for(v
unction(){document.domain=document.domain;function
\x26\x26 document.domain.search(/ok-magazin.de/) \x3d\x3d
\x26\x26 document.domain.search(/menshealth.de/) \x3d\x3d
\x26\x26 document.domain.search(/winfuture.de/) \x3d\x3d
\x26\x26 document.domain.search(/jolie.de/) \x3d\x3d
if (document.domain.search(/hamburg-airport.de/) \x3d\x3d
t.domain)?document.domain:/[-\w]+\.(?:[-\w]+\.xn--[-\w]+|[-\w]{3,}
(c)&&-1!==document.domain.indexof(c))for(var
] + '.' + document.domain.split('.').reverse(
/",domain:document.domain})},del:fun
/"+window.document.domain+"/post_login";a&&(c=ng(
/",domain:document.domain})}};functi
+ "'; if(document.domain!=newdomain)document.
ref.match(document.domain.split('.').reverse(
{a=window.document.domain==this.u.document.domain}catch(b){a=!1}a&&
n.href||f.document.domain==f.top.document.domain)return!0}
alhost"!==document.domain.tolowercase()&&(n+=";
var i=0,d=document.domain,p=d.split('.'),s='_vapi
return!!e.document.domain}catch(t){return!1
\'<script>document.domain=\\\""+document.domain+"\\\";<\\\\/scr
b.domain)?document.domain:b.domain)+((!b||!b
te_domain=document.domain,this.__page_url=document.location.href,this.__page_domain=document.location.hostname,this.__browser_useragent=window.navigator.useragent,this.__browser_timezone=(new date).gettimez
",{domain:document.domain,language:l(e,["userlang"]),mode:"un
omain_cmp(document.domain,turlnameindex))&&(domai
sie")>-1&&document.domain!=location.hostname&&(n+=l),n+=m;var
domain="'+document.domain+'";d.write("
id,domain:document.domain,publisher:{id:a}},t.site={id:a,domain:window.location.hostname,page:document.url,ref:document.referrer,publisher:{id:a,domain:window.location.hostname}},t.publisher={id:a,domain:window.location.hostname},json.stringify(t)}return a
nt.open();document.domain=\'' + this.document.domain + '\';document.
sie")>-1&&document.domain!=location.hostname&&(w.src=y,x.src=y);var z=m.
sie")>-1&&document.domain!=location.hostname&&(n.src=o);var p=m.
)));var d=document.domain,e=(d?d:"a").split(/q
"domain="+document.domain+";expires="
"main!==h){document.domain=h
}b(a(""parent""));"
d(){var j=document.domain,h=j,k=/(lmodules)|(linkedi
n b(f){if(document.domain==f.domain){var g=["
etdomain){document.domain=me.domain;!me.feedb
var a=0,b=document.domain,c=b.split("."),d="_gd"+
[href*="'+document.domain+'"]').each(fun
ull;try{t=document.domain}catch(n){}return
.dmn||"."+document.domain.replace(/^www\./,"");x=e.type
/",domain:document.domain},!0)}if(a(c.t
/",domain:document.domain})}},genera
/",domain:document.domain},!0),"object"
.env.air&&document.domain==window.location.hostname&&!(b.ie&&(8>b.version||b.quirks
/",domain:document.domain})}},checka
"http://"+document.domain+"/global/video/worldnowasx.asp?playertype=native&"+e;else var
replace");document.domain = "'+i.domain+'";document.
eturn top.document.domain===document.domain?"js":"js_framed";}catch(e)
atch(g){b=document.domain,d.src="javascript:var d=document.open();d.domain
ocol+"//"+document.domain+"/post_login";a=je(a,"m
nt.open();document.domain='"+encodeuricomponent(window.document.domain)+"';docum
ry{window.document.domain=(0,_.default)(window.d
pagetitle:document.domain},getchattranscriptwidth:function(){return d
" domain: document.domain,
port: location.port ? ':' + location.port : ''
"
ined(t)?i=document.domain:n.isnull(t)?i="":(i=
('<script>document.domain="'+u+'"</script>'),o.close(
='<script>document.domain="'+document.domain+'";</script>
tion f(a){document.domain==a.domain&&(["platform","www","pemberly.www"].some(function(b){if(a.sub
n(){var a=document.domain,b=a,d=/(lmodules)|(linkedi
ml")&&e();document.domain!==b&&(document.domain=b);f(g("par
if(window.document.domain==window.top.document.domain){var a=/g
ol,domain:document.domain,port:location.port?':'+ location.port:''}));},gethe
"',window.document.domain,'";',"void(0
ostwindow.document.domain,"';"]}var o=
"cation if document.domain has been set
try {
ajaxlocation = document.location.href;"
ef).split(document.domain+"/")[1]),r=s;
document.domain = full_domain;
document.domain = main_domain;
,"")+"$");document.domain.match(t)||f.apply
ined(e)?n=document.domain:p.isnull(e)?n="":(n=
sd!="" && document.domain.indexof(sd)<0 ) retu
sd!="" && document.domain.indexof(sd)<0 ){ sd=
1 Like