Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24586 )
Change subject: IMPALA-15184: Switch tpcds json from bzip to zstd ...................................................................... IMPALA-15184: Switch tpcds json from bzip to zstd bzip is very slow, and it is enough to test it on smaller data (already covered in functional text). Switching to zstd, which did not have its own db. >From dataload logs before change: Loading TPC-H data OK (Took: 5 min 39 sec) Loading TPC-DS data OK (Took: 22 min 33 sec) Loading functional-query data OK (Took: 30 min 43 sec) After change: Loading TPC-H data OK (Took: 5 min 22 sec) Loading TPC-DS data OK (Took: 13 min 49 sec) Loading functional-query data OK (Took: 27 min 30 sec) Total dataload time is decreased from ~35m to ~30m. The longest leg functional-query also decreased, likely due to less contention on the max 2 parallel Hive queries (see IMPALA-15185 for details on Hive parallelism). Change-Id: Ia5e787619cc9f2b1cd30d40440aecd20fcef4117 Reviewed-on: http://gerrit.cloudera.org:8080/24586 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M testdata/bin/generate-schema-statements.py M testdata/workloads/tpcds/tpcds_core.csv 2 files changed, 2 insertions(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24586 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia5e787619cc9f2b1cd30d40440aecd20fcef4117 Gerrit-Change-Number: 24586 Gerrit-PatchSet: 5 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Laszlo Gaal <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]>
