Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24587
to look at the new patch set (#3).
Change subject: IMPALA-15185: Increase Hive query parallelism from 2 to 8
......................................................................
IMPALA-15185: Increase Hive query parallelism from 2 to 8
The combination of Yarn/Tez/Hive configs limited the number
of parallel Hive queries to 2, which slowed down Hive heavy
tests and the dataload. Specifically each Hive query needs
a Yarn application master (AM), and mem settings limited AM
count to 2.
This change tweaks some configs. The math on typical Impala
test job (m6i-4xl):
total mem: 64 GB
Yarn mem: 44 GB (unchanged)
AM mem%: 10% -> 20%
AM mem: 4 GB -> 8 GB
mem/AM: 2 GB -> 1 GB
max AMs: 2 -> 8
worker mem: 40 GB -> 36 GB
mem/worker: 2 GB (unchanged)
max workers: 20 -> 18
worker/AM: 10 -> 3
The main goal is to speed up dataload, which is Hive heavy and
tries to do multiple jobs in parallel. ACID compactions also
seem to become much faster (>1m -> 10-20s), not sure why.
The max 8 AM matches NUM_CONCURRENT_TESTS=8 (cores/2) so
tests that use Hive are unlikely to be blocked waiting for
AM.
3 worker/AM looks small, but this is a worst (or best...)
case scenario where 8 Hive queries do heavy work in parallel.
Most tables are small and should be fine with 1 worker. Hive
queries during tests are usually minimal (like insterting 1 row).
If the change brings instability, then the number of AMs can
be reduced.
>From dataload logs before change:
Loading TPC-H data OK (Took: 5 min 39 sec)
Loading TPC-DS data OK (Took: 22 min 33 sec)
Loading functional-query data OK (Took: 30 min 43 sec)
After Change:
Loading TPC-H data OK (Took: 4 min 50 sec)
Loading functional-query data OK (Took: 16 min 34 sec)
Loading TPC-DS data OK (Took: 21 min 2 sec)
functional-query is almost halved and TPC-DS became the
longest leg.
Total dataload time is decreased from ~35m to ~27m.
Core test suite seem to become another 20-30m faster. Did
limited investigation, but generally Hive using tests look
a bit faster while tests that do ACID compactions are much
faster (test_acid.py: 23m->10m).
Assisted-by: Clause Opus 4.8
Change-Id: I0b17c99b4f925bcce4604f8e595de33bd87236c4
---
M fe/src/test/resources/hive-site.xml.py
M testdata/cluster/node_templates/common/etc/hadoop/conf/capacity-scheduler.xml
M testdata/cluster/node_templates/common/etc/hadoop/conf/yarn-site.xml.py
3 files changed, 9 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/87/24587/3
--
To view, visit http://gerrit.cloudera.org:8080/24587
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0b17c99b4f925bcce4604f8e595de33bd87236c4
Gerrit-Change-Number: 24587
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
