# Queries returning numbers larger than size of tables?

**URL:** <https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021>\
**Category:** Analysis\
**Created:** [July 10, 2017, 10:55pm UTC](https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021 "2017-07-10T22:55:44Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![amirian](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@amirian](https://discuss.httparchive.org/u/amirian)\
**Post date:** [July 10, 2017, 10:55pm UTC](https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021/1 "2017-07-10T22:55:44Z")

</div>

I’m using the tables from the June 15, 2017 run and running into some strange numbers.

For example, when I run `select count(*) from httparchive:runs.2017_06_15_pages` I get 474,696 rows. I ran `select count(distinct(pageid)) from httparchive:runs.2017_06_15_pages` to see if there were any pageid’s that were duplicate, and received 498599 as an answer. My understanding is that there is one pageid per row.

I’m pretty stuck at this point. The second query shouldn’t be returning a number larger than the first, at least based on my understanding from the docs. What am I missing?

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [July 11, 2017, 5:03pm UTC](https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021/2 "2017-07-11T17:03:16Z")

</div>

Yeah that is strange. I rewrote your second query to group duplicate pageids, if they exist:

```sql
SELECT
  COUNT(0),
  pageid
FROM
  [httparchive:runs.2017_06_15_pages]
GROUP BY
  2
ORDER BY
  1 DESC

```

And there were no duplicates. Also the number of rows in the result was 474696.

---

<div class="post-metadata">

**Author:** ![amirian](https://avatars.discourse-cdn.com/v4/letter/a/e19b73/32.png) [@amirian](https://discuss.httparchive.org/u/amirian)\
**Post date:** [July 12, 2017, 12:04am UTC](https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021/3 "2017-07-12T00:04:45Z")

</div>

Thanks for the modified query!

I’m really curious about the underlying reason though…any guesses as to why that original problem is? Does it have something to do with the querying syntax with BigQuery?

---

<div class="post-metadata">

**Author:** ![rviscomi](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/rviscomi/32/835_2.png) [@rviscomi](https://discuss.httparchive.org/u/rviscomi)\
**Post date:** [July 12, 2017, 1:06am UTC](https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021/4 "2017-07-12T01:06:42Z")

</div>

> [@amirian](#):
>
> Does it have something to do with the querying syntax with BigQuery?

Yeah I think so. Just trying to do `SELECT DISTINCT(pageid) ...` yielded an error so it seems `GROUP BY` is the right way to go.

cc @fhoffa in case he has any advice

---

<div class="post-metadata">

**Author:** ![fhoffa](https://yyz1.discourse-cdn.com/flex035/user_avatar/discuss.httparchive.org/fhoffa/32/750_2.png) [@fhoffa](https://discuss.httparchive.org/u/fhoffa)\
**Post date:** [July 12, 2017, 1:22am UTC](https://discuss.httparchive.org/t/queries-returning-numbers-larger-than-size-of-tables/1021/5 "2017-07-12T01:22:07Z")

</div>

Switch to #standardSQL - COUNT DISTINCT is exact there.

And to see the advantages of an approximate method:

- [https://medium.freecodecamp.org/counting-uniques-faster-in-bigquery-with-hyperloglog-5d3764493a5a](https://medium.freecodecamp.org/counting-uniques-faster-in-bigquery-with-hyperloglog-5d3764493a5a)

With #standardSQL, the approximate results are better too:

```
#standardSQL
select count(distinct(pageid)) 
from `httparchive.runs.2017_06_15_pages`

474696	

#standardSQL
select APPROX_COUNT_DISTINCT(pageid) from `httparchive.runs.2017_06_15_pages`

473038
```
