This is an automated email from the ASF dual-hosted git repository. joemcdonnell pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/impala.git
commit ad915ca58eaa004925d545057e9ebdba5d62131b Author: Riza Suminto <[email protected]> AuthorDate: Thu May 19 15:08:12 2022 -0700 IMPALA-11306: Create symlink for dataset of scale factor 1 single_node_perf_run.py and load-data.py can fail if user set scale factor argument 1. This is because generate-schema-statements.py will insert the scale factor into the database name (ie., "tpch1"), but the preload script omit the scale factor when creating dataset directory (ie., "tpch"). This patch fix the issue by additionally creating symlink for scale factor 1. Testing: - Manual test by running the following script: ./bin/load-data.py --scale_factor=1 --workloads=targeted-perf \ --table_formats=text/none/none Change-Id: I76c9c90b243df6213626e11652cfed59643aed2c Reviewed-on: http://gerrit.cloudera.org:8080/18545 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- testdata/datasets/tpcds/preload | 6 ++++++ testdata/datasets/tpch/preload | 6 ++++++ 2 files changed, 12 insertions(+) diff --git a/testdata/datasets/tpcds/preload b/testdata/datasets/tpcds/preload index 99b1f7bc3..f4a9c2ec8 100755 --- a/testdata/datasets/tpcds/preload +++ b/testdata/datasets/tpcds/preload @@ -42,6 +42,12 @@ echo "Generating TPC-DS data into ${TPC_DS_DATA}" # Delete any preexisting data or symlinks rm -rf ${TPC_DS_DATA} mkdir -p ${TPC_DS_DATA} +# Create symlink if scale factor is 1 +if [ ${SCALE_FACTOR} -eq 1 ] +then + rm -rf ${TPC_DS_DATA}${SCALE_FACTOR} + ln -s ${TPC_DS_DATA} ${TPC_DS_DATA}${SCALE_FACTOR} +fi cd ${TPC_DS_DATA} # dsdgen uses fixed size buffers that cause bizarre issues if the path to the diff --git a/testdata/datasets/tpch/preload b/testdata/datasets/tpch/preload index 4bbfe0620..2ecbfd692 100755 --- a/testdata/datasets/tpch/preload +++ b/testdata/datasets/tpch/preload @@ -50,6 +50,12 @@ echo "Generating TPC-H data into ${TPC_H_DATA}" chmod +w ${TPC_H_DATA} || true rm -rf ${TPC_H_DATA} mkdir -p ${TPC_H_DATA} +# Create symlink if scale factor is 1 +if [ ${SCALE_FACTOR} -eq 1 ] +then + rm -rf ${TPC_H_DATA}${SCALE_FACTOR} + ln -s ${TPC_H_DATA} ${TPC_H_DATA}${SCALE_FACTOR} +fi cd ${TPC_H_DATA} if [ -t 1 ]
