Hello Joe McDonnell, Michael Smith, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24586
to look at the new patch set (#3).
Change subject: IMPALA-15184: Switch tpcds json from bzip to zstd
......................................................................
IMPALA-15184: Switch tpcds json from bzip to zstd
bzip is very slow, and it is enough to test it on
smaller data (already covered in functional text).
Switching to zstd, which did not have its own db.
>From dataload logs before change:
Loading TPC-H data OK (Took: 5 min 39 sec)
Loading TPC-DS data OK (Took: 22 min 33 sec)
Loading functional-query data OK (Took: 30 min 43 sec)
After change:
Loading TPC-H data OK (Took: 5 min 22 sec)
Loading TPC-DS data OK (Took: 13 min 49 sec)
Loading functional-query data OK (Took: 27 min 30 sec)
Total dataload time is decreased from ~35m to ~30m.
The longest leg functional-query also decreased, likely
due to less contention on the max 2 parallel Hive queries
(see IMPALA-15185 for details on Hive parallelism).
Change-Id: Ia5e787619cc9f2b1cd30d40440aecd20fcef4117
---
M testdata/bin/generate-schema-statements.py
M testdata/workloads/tpcds/tpcds_core.csv
2 files changed, 2 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/86/24586/3
--
To view, visit http://gerrit.cloudera.org:8080/24586
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia5e787619cc9f2b1cd30d40440aecd20fcef4117
Gerrit-Change-Number: 24586
Gerrit-PatchSet: 3
Gerrit-Owner: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
