IMPALA-3971, IMPALA-3229: Bootstrap an Impala dev environment

This script bootstraps an Impala dev environment on Ubuntu 14.04. It
is not hermetic -- it changes some config files for the user and for
the OS.

It is green on Jenkins, and it runs in about 6.5 hours. The intention
is to have this script run in a CI tool for post-commit testing, with
the hope that this will make it easier for new developers to get a
working development environment. Previously, the new developer
workflow lived on wiki pages and tended to bit-rot.

Still left to do: migrating the install script into the official
Impala repo.

Change-Id: If166a8a286d7559af547da39f6cc09e723f34c7e
Reviewed-by: Jim Apple <>
Tested-by: Internal Jenkins


Branch: refs/heads/hadoop-next
Commit: 784716f776492ed648833f2717b3784900b6f090
Parents: 2a04b0e
Author: Jim Apple <>
Authored: Fri Oct 7 20:34:28 2016 -0700
Committer: Internal Jenkins <>
Committed: Fri Oct 14 06:07:46 2016 +0000

 bin/ | 80 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
diff --git a/bin/ b/bin/
new file mode 100755
index 0000000..8c4f742
--- /dev/null
+++ b/bin/
@@ -0,0 +1,80 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# This script bootstraps a development environment from almost nothing; it is 
known to
+# work on Ubuntu 14.04, and it definitely clobbers some local environment, so 
it's best to
+# run this in a sandbox first, like a VM or docker.
+# The intended user is a person who wants to start contributing code to 
Impala. This
+# script serves as an executable reference point for how to get started.
+# At this time, it completes in about 6.5 hours. It generates and loads the 
test data and
+# metadata without using a snapshot (which takes about 3 hours) and it then 
runs the full
+# testsuite (frontend, backend, end-to-end, JDBC, and custom cluster) in "core"
+# exploration mode.
+set -eux -o pipefail
+if [[ ! -d "${HOMEDIR}" ]]
+    echo "${HOMEDIR} is needed for installing Impala dependencies"
+    exit 1
+if [[ -z "${JAVA_HOME}" ]]
+    echo "JAVA_HOME must be set to install Impala dependencies"
+    exit 1
+if ! sudo true
+    echo "Passwordless sudo is needed for this script"
+    exit 1
+# Place to download setup scripts
+TMPDIR=$(mktemp -d)
+function cleanup {
+    rm -rf "${TMPDIR}"
+trap cleanup EXIT
+# Install build and test pre-reqs
+pushd "${TMPDIR}"
+git clone "${IMPALA_SETUP_REPO_URL}" impala-setup
+cd impala-setup
+chmod +x ./
+sudo ./
+# HDFS bug workaround
+echo " $(hostname -s) $(hostname)" | sudo tee -a /etc/hosts
+echo "NoHostAuthenticationForLocalhost yes" > ~/.ssh/config
+pushd "$(dirname $0)/.."
+export IMPALA_HOME="$(pwd)"
+source bin/
+./ -noclean -format -testdata

Reply via email to