[
https://issues.apache.org/jira/browse/IMPALA-7988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746685#comment-16746685
]
ASF subversion and git services commented on IMPALA-7988:
---------------------------------------------------------
Commit 236b9194d345bbfdfb177dd7ef4908170eaff259 in impala's branch
refs/heads/master from Tim Armstrong
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=236b919 ]
IMPALA-7988: support loading data with dockerized Impalas
This patch does the work to load data and run some end-to-end
query tests on a dockerised cluster. Changes were required
in start-impala-cluster.py/ImpalaCluster and in some configuration
files.
ImpalaCluster is used for various things, including discovering
service ports and testing for cluster readiness. This patch adds
basic support and uses it from start-impala-cluster.py to check
for cluster readiness. Some logic is moved from
start-impala-cluster.py to ImpalaCluster.
Limitations:
* We're fairly inconsistent about whether services listen only on
a single interface (e.g. loopback, traditionally) or whether it
listens on all interfaces. This doesn't fix all of those issues.
E.g. HDFS datanodes listen on all interfaces to work around
some issues.
* Many tests don't pass yet, particularly those using
ImpalaCluster(), which isn't initialised with the appropriate
docker arguments.
Testing:
Did a full data load locally using a dockerised Impala cluster:
START_CLUSTER_ARGS="--docker_network=impala-cluster" \
TEST_START_CLUSTER_ARGS="--docker_network=impala-cluster" \
./buildall.sh -format -testdata -ninja -notests -skiptests -noclean
Ran a selection of end-to-end tests touching HDFS, Kudu and HBase
tables after I loaded data locally.
Ran exhaustive tests with non-dockerised impala cluster.
Change-Id: I98fb9c4f5a3a3bb15c7809eab28ec8e5f63ff517
Reviewed-on: http://gerrit.cloudera.org:8080/12189
Reviewed-by: Tim Armstrong <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Support loading data into a dockerised minicluster
> --------------------------------------------------
>
> Key: IMPALA-7988
> URL: https://issues.apache.org/jira/browse/IMPALA-7988
> Project: IMPALA
> Issue Type: Sub-task
> Components: Infrastructure
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Major
> Labels: docker
>
> This JIRA tracks getting data load to work against dockerised impala daemons.
> * Fix start-impala-cluster.py to wait for cluster to become ready using
> ImpalaCluster
> * Fix test configuration to work against all table formats (HDFS, HBase, Kudu)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
