Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3223: Supports download of CDH components from S3. ......................................................................
IMPALA-3223: Supports download of CDH components from S3. This change updates the toolchain bootstrapping script to download the CDH components (hadoop, hbase, hive, llama, llama-minikdc and sentry) from the toolchain S3 bucket to the toolchain directory if the environment variable $DOWNLOAD_CDH_COMPONENTS is true. By default, it is false which means the CDH components in the thirdparty directory will be used instead. To build the ASF tree(https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git), set $DOWNLOAD_CDH_COMPONENTS to true. Currently, the CDH components in S3 are snapshots from the thirdparty directory at 688d0efcd38731e8e27a8236dbdca21c8fd571a1. Once the integration jenkins job (impala-cdh5-trunk-core-integration) is modified to upload the latest stable builds to the S3 buckets, we can remove the thirdparty directory and always use the CDH components in the toolchain directory. Note that bootstrap_toolchain.py will not overwrite existing directories in the toolchain directory. To force a refresh of cpmponents in the toolchain directory, a user should delete the cached copy in the toolchain directory and execute bootstrap_toolchain.py again. This behavior allows users to develop locally without network connection once the toolchain has been bootstrapped. Change-Id: I16fa79db0005554cc0a116e74775647ba99f8dda Reviewed-on: http://gerrit.cloudera.org:8080/3333 Reviewed-by: Michael Ho <[email protected]> Tested-by: Internal Jenkins --- M CMakeLists.txt M bin/bootstrap_toolchain.py M bin/impala-config.sh M buildall.sh M testdata/cluster/node_templates/cdh5/etc/init.d/llama-application 5 files changed, 107 insertions(+), 61 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/3333 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I16fa79db0005554cc0a116e74775647ba99f8dda Gerrit-PatchSet: 8 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Jim Apple <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
