Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/10705 )
Change subject: IMPALA-6816: minimise calls to GetMinSubscriberTopicVersion() ...................................................................... Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/10705/4/tests/statestore/test_statestore.py File tests/statestore/test_statestore.py: http://gerrit.cloudera.org:8080/#/c/10705/4/tests/statestore/test_statestore.py@635 PS4, Line 635: assert min_subscriber_topic_version >= min_to_version - 1,\ > If the update for the consumer comes late for some reason, won't the to_ver Let me go over my reasoning again, I'm not totally confident now. We have 3 versions. * a is the consumer's version tracked in last_to_versions * b is the consumer's version in the statestore (TopicSubscription::last_version) * c is last minimum version seen by the producer * a >= b, because the version is only updated after the Update() RPC. * b >= c, because the update with the latest version may not yet have been delivered to the producer. Getting bounds on how much c can be behind a is harder. I'm assuming that updates aren't delayed by more than 1 second or so, but I think there could still be two producer updates in-between a consumer update or vice-versa. And you're right - this means that to_version could advance by two at a time on the consumer. So I'll loosen the check. I think checking for a delta of 2 is sufficient to check that the version is advancing but will be less vulnerable to small delays. I'll also add a check for the other invariant -- To view, visit http://gerrit.cloudera.org:8080/10705 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8ee7cb2355ba1049b9081e0df344ac41aa4ebeb1 Gerrit-Change-Number: 10705 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Tianyi Wang <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Tue, 26 Jun 2018 14:51:42 +0000 Gerrit-HasComments: Yes
