David Knupp has uploaded a new patch set (#4).
Change subject: Enabling end-to-end tests on a remote cluster
......................................................................
Enabling end-to-end tests on a remote cluster
This patch enables data loading and running end-to-end tests on a remote
cluster. The requirements to run the tests on a remote cluster are
- CDH cluster that is CM managed
- KMS and KeyTrustee installed and available as service
- Hive warehouse dir points to /test-warehouse
The new remote_load_data.py script takes a CM host as argument and will
load the test warehouse snapshot on the first cluster managed by this
instance of CM. It will automatically pick the necessary configuration
needed to perform the data load process.
Usage: remote_data_load.py [options] cm_host
Options:
-h, --help show this help message and exit
--cm-user=CM_USER Cloudera Manager admin user
--cm-pass=CM_PASS Cloudera Manager admin user password
--gateway=GATEWAY Gateway host to upload the data from. If not set, uses
the CM host as gateway.
--ssh-user=SSH_USER System user on the remote machine with passwordless
SSH configured.
--no-load Do not try to load the snapshot
--exploration-strategy=EXPLORATION_STRATEGY
--test Run end-to-end tests against cluster
Change-Id: I1f443a1728a1d28168090c6f54e82dec2cb073e9
---
M bin/load-data.py
A bin/remote_data_load.py
M testdata/bin/compute-table-stats.sh
M testdata/bin/create-load-data.sh
M testdata/bin/create-table-many-blocks.sh
M testdata/bin/generate-schema-statements.py
M testdata/bin/load-test-warehouse-snapshot.sh
M testdata/bin/load_nested.py
M testdata/bin/run-step.sh
M testdata/bin/setup-hdfs-env.sh
10 files changed, 575 insertions(+), 60 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/69/4769/4
--
To view, visit http://gerrit.cloudera.org:8080/4769
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I1f443a1728a1d28168090c6f54e82dec2cb073e9
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: David Knupp <dkn...@cloudera.com>
Gerrit-Reviewer: David Knupp <dkn...@cloudera.com>
Gerrit-Reviewer: Harrison Sheinblatt <h...@hotmail.com>
Gerrit-Reviewer: Martin Grund <grundprin...@gmail.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
