Hello Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18233
to look at the new patch set (#2).
Change subject: IMPALA-11124: Reuse local TPCH/TPCDS data in testdata loading
......................................................................
IMPALA-11124: Reuse local TPCH/TPCDS data in testdata loading
When loading testdata for TPC-H/TPC-DS, we first run a preload script to
generate local data, and then upload them to HDFS to be used by Hive.
The preload script currently always generates the data, which is
time-consuming in large scale factors.
This patch modifies the preload scripts to check if the last run
succeeded, and reuse the data if it does. Otherwise, generate the data
and leave a success marker in the data directory.
Tests:
- Verified the scripts locally.
Change-Id: Ied40e599cda009ae0ad88ad13385e7bb86428bb4
---
M testdata/datasets/tpcds/preload
M testdata/datasets/tpch/preload
2 files changed, 14 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18233/2
--
To view, visit http://gerrit.cloudera.org:8080/18233
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ied40e599cda009ae0ad88ad13385e7bb86428bb4
Gerrit-Change-Number: 18233
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
