Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/17592 )
Change subject: IMPALA-7635: Reducing HashTable size by packing it's buckets
efficiently.
......................................................................
Patch Set 5:
(1 comment)
I grabbed this change and was doing some local testing, and I ran into a
scenario that crashes Impala.
Here's what I did:
1. I loaded a larger TPC-H dataset in parquet (takes quite a while and will use
up some disk space):
bin/load-data.py -w tpch -s 42 --table_formats=text/none/none,parquet/none/none
2. Ran the following multiple times (it is intermittent):
use tpch42_parquet;
SELECT l_orderkey,
count(*) AS cnt
FROM lineitem
GROUP BY l_orderkey
HAVING count(*) > 9999999999999;
That intermittently crashes with this stack:
C [impalad+0x27e8546] long impala::HashTable::Probe<true,
false>(impala::HashTable::Bucket*, long, impala::HashTableCtx*, unsigned int,
bool*, impala::HashTable::BucketData*)+0x28c
C [impalad+0x27e2141] impala::HashTable::ResizeBuckets(long,
impala::HashTableCtx*, bool*)+0x7d1
C [impalad+0x27e194d] impala::HashTable::CheckAndResize(unsigned long,
impala::HashTableCtx*, bool*)+0xcb
C [impalad+0x27c8a48]
impala::GroupingAggregator::CheckAndResizeHashPartitions(bool, int,
impala::HashTableCtx*)+0x176
This may reproduce on smaller datasets.
http://gerrit.cloudera.org:8080/#/c/17592/5/be/src/benchmarks/hash-table-benchmark.cc
File be/src/benchmarks/hash-table-benchmark.cc:
http://gerrit.cloudera.org:8080/#/c/17592/5/be/src/benchmarks/hash-table-benchmark.cc@332
PS5, Line 332: vector<int> num_tuples { 65536, 262144 };
One area of performance testing is looking at what happens when the hash table
no longer fits in CPU cache and requires more accesses to main memory. This
will interact heavily with the memory layout and prefetching.
That kind of testing requires a lot of care if we do it outside the usual
Impala code, because it will be much more sensitive to the specific algorithms
we use. In particular, we process tuples in batches and do a round of
prefetching for the whole batch before the insert/probe. This hides the latency
of fetching things from memory.
I'm commenting here because if this benchmark wants to handle that case, these
values need to go much higher. 1 million, 5 million, 10 million, 100 million.
Other aspects would also need to change.
Another approach for testing the hashtable is to write custom hashtable
intensive queries. The general shape would be like this:
1. Run a single Impalad
2. Create tables with a single column with different data distributions (i.e.
different number of distinct values, duplication). These are used for building
hash tables.
3. Create tables with different data distributions for probing the hash table.
4. Testing hashtable build involves a groupby like this targeted perf query:
https://github.com/apache/impala/blob/master/testdata/workloads/targeted-perf/queries/primitive_groupby_bigint_highndv.test
5. Testing hashtable probe involves writing SQLs with JOINs that stress the
hashtable. The probe tables can be as big as we want, so that the probe is the
dominant part of query execution. STRAIGHT_JOIN can override any join ordering
optimizations to force a particular join order. We may have to disable runtime
filters.
--
To view, visit http://gerrit.cloudera.org:8080/17592
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I72912ae9353b0d567a976ca712d2d193e035df9b
Gerrit-Change-Number: 17592
Gerrit-PatchSet: 5
Gerrit-Owner: Amogh Margoor <[email protected]>
Gerrit-Reviewer: Amogh Margoor <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
Gerrit-Comment-Date: Thu, 08 Jul 2021 02:40:10 +0000
Gerrit-HasComments: Yes
