Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21867 )
Change subject: IMPALA-13408: use a specific flag for the topic prefix cluster identifier. ...................................................................... IMPALA-13408: use a specific flag for the topic prefix cluster identifier. The cluster_id flag was introduced in IMPALA-12426 to identify Impala clusters in systems where a single query_log table could be shared. In IMPALA-13208 the cluster_id flag was reused as a prefix to topic names for backend membership, to allow sub-clusters of backends within a Statestore service. There have been some problems with the interaction of these two usages. An important difference is that the query_log cluster_id must be set only on coordinators, whereas the topic prefix cluster_id must be set simultaneously on coordinators, executors, and admission daemons (if present). If a system is started with cluster_id set only on coordinators then there are split-brain problems where coordinators and executors are tracked in different topics. In addition, the query_log cluster_id is more likely to be user-settable as it is used for data in query_log which will be read by humans, who may want to write queries selecting data from their ‘production’ or ‘dev’ clusters. Avoid these problems by using a separate flag for the topic prefix cluster_id ‘cluster_membership_topic_id’. Change-Id: Icd3f7e1c73c00a7aaeee79ecb461209e3939c422 Reviewed-on: http://gerrit.cloudera.org:8080/21867 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/runtime/exec-env.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/admissiond-env.cc M be/src/scheduling/cluster-membership-mgr.cc M tests/custom_cluster/test_shared_catalogd.py 5 files changed, 21 insertions(+), 14 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/21867 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Icd3f7e1c73c00a7aaeee79ecb461209e3939c422 Gerrit-Change-Number: 21867 Gerrit-PatchSet: 9 Gerrit-Owner: Andrew Sherman <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
