IMPALA-4299: add buildall.sh option to start test cluster A previous commit "IMPALA-4259: build Impala without any test cluster setup" altered some undocumented side-effects of buildall.sh.
Previously the following commands reconfigured and restarted the test cluster. It worked because buildall.sh unconditionally regenerated the test cluster configs. ./buildall.sh -notests && ./testdata/bin/run-all.sh ./buildall.sh -noclean -notests && ./testdata/bin/run-all.sh Instead of restoring the old behaviour and continuing to encourage mixing use of low and high level scripts like testdata/bin/run-all.sh as part of the "standard" workflow, this commit adds another high-level option to buildall.sh, -start_minicluster, that accomplishes the high-level task of restarting a minicluster with fresh configs. The above commands can be replaced with: ./buildall.sh -notests -start_minicluster ./buildall.sh -notests -noclean -start_minicluster Change-Id: I0ab3461f8ff3de49b3f28a0dc22fa0a6d5569da5 Reviewed-on: http://gerrit.cloudera.org:8080/4734 Reviewed-by: Alex Behm <[email protected]> Tested-by: Internal Jenkins Project: http://git-wip-us.apache.org/repos/asf/incubator-impala/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-impala/commit/ef762b73 Tree: http://git-wip-us.apache.org/repos/asf/incubator-impala/tree/ef762b73 Diff: http://git-wip-us.apache.org/repos/asf/incubator-impala/diff/ef762b73 Branch: refs/heads/master Commit: ef762b73a1ddb8842bb5bac5bfed733b9a71fc1b Parents: 0480253 Author: Tim Armstrong <[email protected]> Authored: Mon Oct 17 10:04:39 2016 -0700 Committer: Internal Jenkins <[email protected]> Committed: Mon Oct 17 22:19:06 2016 +0000 ---------------------------------------------------------------------- buildall.sh | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-impala/blob/ef762b73/buildall.sh ---------------------------------------------------------------------- diff --git a/buildall.sh b/buildall.sh index ebd3eb3..a7858a3 100755 --- a/buildall.sh +++ b/buildall.sh @@ -46,6 +46,7 @@ TESTS_ACTION=1 FORMAT_CLUSTER=0 FORMAT_METASTORE=0 FORMAT_SENTRY_POLICY_DB=0 +NEED_MINICLUSTER=0 START_IMPALA_CLUSTER=0 IMPALA_KERBERIZE=0 SNAPSHOT_FILE= @@ -133,6 +134,9 @@ do METASTORE_SNAPSHOT_FILE="$(readlink -f "$METASTORE_SNAPSHOT_FILE")" shift; ;; + -start_minicluster) + NEED_MINICLUSTER=1 + ;; -start_impala_cluster) START_IMPALA_CLUSTER=1 ;; @@ -169,6 +173,10 @@ do echo "[-asan] : Address sanitizer build [Default: False]" echo "[-skiptests] : Skips execution of all tests" echo "[-notests] : Skips building and execution of all tests" + echo "[-start_minicluster] : Start test cluster including Impala and all"\ + " its dependencies. If already running, all services are restarted."\ + " Regenerates test cluster config files. [Default: True if running "\ + " tests or loading data, False otherwise]" echo "[-start_impala_cluster] : Start Impala minicluster after build"\ " [Default: False]" echo "[-testpairwise] : Run tests in 'pairwise' mode (increases"\ @@ -191,8 +199,12 @@ Examples of common tasks: # Build and skip tests ./buildall.sh -skiptests - # Incrementally rebuild and skip tests. Keeps existing Hadoop services running. - ./buildall.sh -skiptests -noclean + # Build, then restart the minicluster and Impala with fresh configs. + ./buildall.sh -notests -start_minicluster -start_impala_cluster + + # Incrementally rebuild and skip tests. Keeps existing minicluster services running + # and restart Impala. + ./buildall.sh -skiptests -noclean -start_impala_cluster # Build, load a snapshot file, run tests ./buildall.sh -snapshot_file <file> @@ -256,7 +268,6 @@ if [[ -z "$METASTORE_SNAPSHOT_FILE" && "${TARGET_FILESYSTEM}" != "hdfs" && exit 1 fi -NEED_MINICLUSTER=0 if [[ $TESTS_ACTION -eq 1 || $TESTDATA_ACTION -eq 1 || $FORMAT_CLUSTER -eq 1 || $FORMAT_METASTORE -eq 1 || $FORMAT_SENTRY_POLICY_DB -eq 1 || -n "$SNAPSHOT_FILE" || -n "$METASTORE_SNAPSHOT_FILE" ]]; then
