Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18233
Change subject: IMPALA-11124: Reuse local TPCH/TPCDS data in testdata loading ...................................................................... IMPALA-11124: Reuse local TPCH/TPCDS data in testdata loading When loading testdata for TPC-H/TPC-DS, we first run a preload script to generate local data, and then upload them to HDFS to be used by Hive. The preload script currently always re-generate the data, which is time-consuming in large scale factors. This patch modifies the preload scripts to check if the last run succeeded, and reuse the data if it does. Otherwise, generate the data and leave a success marker in the data directory. Tests: - Verified the scripts locally. Change-Id: Ied40e599cda009ae0ad88ad13385e7bb86428bb4 --- M testdata/datasets/tpcds/preload M testdata/datasets/tpch/preload 2 files changed, 14 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/33/18233/1 -- To view, visit http://gerrit.cloudera.org:8080/18233 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ied40e599cda009ae0ad88ad13385e7bb86428bb4 Gerrit-Change-Number: 18233 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
