Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10705 )
Change subject: IMPALA-6816: minimise calls to GetMinSubscriberTopicVersion() ...................................................................... IMPALA-6816: minimise calls to GetMinSubscriberTopicVersion() min_subscriber_topic_version is expensive to compute (requires iterating over all subscribers to compute) but is only used by one subscriber/topic pair: Impalads receiving catalog topic updates. This patch implements a simple fix - only compute it if a subscriber asks for it. A more complex alternative would be to maintain a priority queue of subscriber versions, but that didn't seem worth the the complexity and risk of bugs. Testing: Add a statestore test to validate the versions. It looks like we had a pre-existing test gap for validating min_subscriber_topic_version so the test is mainly focused on adding that coverage. Ran core tests with DEBUG and ASAN. Change-Id: I8ee7cb2355ba1049b9081e0df344ac41aa4ebeb1 Reviewed-on: http://gerrit.cloudera.org:8080/10705 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/catalog/catalog-server.cc M be/src/scheduling/admission-controller.cc M be/src/scheduling/scheduler.cc M be/src/service/impala-server.cc M be/src/statestore/statestore-subscriber.cc M be/src/statestore/statestore-subscriber.h M be/src/statestore/statestore.cc M be/src/statestore/statestore.h M common/thrift/StatestoreService.thrift M tests/statestore/test_statestore.py 10 files changed, 160 insertions(+), 34 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/10705 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8ee7cb2355ba1049b9081e0df344ac41aa4ebeb1 Gerrit-Change-Number: 10705 Gerrit-PatchSet: 8 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tianyi Wang <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
