Repository: incubator-impala Updated Branches: refs/heads/master 2100b3a34 -> 73595d8f4
IMPALA-3737: Local filesystem build failed loading custom schemas When a LOCATION that does not have the scheme specified is used, the default FS is used as the filesystem scheme. The default FS is set as 'file:/tmp' for localFS runs, however the Hadoop library seems to ignore the '/tmp' part of the defaultFS for locations without schemes and just uses 'file:'. So the test warehouse is in: 'file:/tmp/test-warehouse' However, the scripts access '/test-warehouse' without the scheme which hadoop translates to: 'file:/test-warehouse' which does not exist. This change disables metadata loading on local filesystem if there is a schema change detected just as it is done in S3 and Isilon too. Change-Id: Ie404079aeb2f837ac8b03244b2019e2c8ee9f221 Reviewed-on: http://gerrit.cloudera.org:8080/3384 Reviewed-by: Sailesh Mukil <[email protected]> Tested-by: Sailesh Mukil <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/73595d8f Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/73595d8f Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/73595d8f Branch: refs/heads/master Commit: 73595d8f40dd5ac67cf6f225dd55f1297d06e3ff Parents: 2100b3a Author: Sailesh Mukil <[email protected]> Authored: Tue Jun 14 16:08:21 2016 -0700 Committer: Tim Armstrong <[email protected]> Committed: Thu Jun 16 17:34:34 2016 -0700 ---------------------------------------------------------------------- testdata/bin/create-load-data.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/73595d8f/testdata/bin/create-load-data.sh ---------------------------------------------------------------------- diff --git a/testdata/bin/create-load-data.sh b/testdata/bin/create-load-data.sh index abeefab..2ee7d56 100755 --- a/testdata/bin/create-load-data.sh +++ b/testdata/bin/create-load-data.sh @@ -79,9 +79,10 @@ elif [ $SKIP_SNAPSHOT_LOAD -eq 0 ]; then ${IMPALA_HOME}/testdata/bin/load-test-warehouse-snapshot.sh "$SNAPSHOT_FILE" # Don't skip the metadata load if a schema change is detected. if ! ${IMPALA_HOME}/testdata/bin/check-schema-diff.sh; then - if [[ "${TARGET_FILESYSTEM}" == "isilon" || "${TARGET_FILESYSTEM}" == "s3" ]] ; then + if [[ "${TARGET_FILESYSTEM}" == "isilon" || "${TARGET_FILESYSTEM}" == "s3" || \ + "${TARGET_FILESYSTEM}" == "local" ]] ; then echo "ERROR in $0 at line $LINENO: A schema change has been detected in the" - echo "metadata, but it cannot be loaded on isilon or s3 and the" + echo "metadata, but it cannot be loaded on isilon, s3 or local and the" echo "target file system is ${TARGET_FILESYSTEM}. Exiting." exit 1 fi
