Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/14897 )
Change subject: WIP [master] KUDU-3016 don't lump together updates on all tablet reports ...................................................................... Patch Set 1: (1 comment) > (2 comments) > > Seems like a reasonable approach to me. I'd like to understand the > perf impact though; could you deploy it to a cluster with a large > number of replicas and gauge the impact on a beefy TSHeartbeat? Thank you for the feedback! I'll work on a couple of tests to cover this functionality and to gauge the impact of this change. http://gerrit.cloudera.org:8080/#/c/14897/1/src/kudu/master/catalog_manager.cc File src/kudu/master/catalog_manager.cc: http://gerrit.cloudera.org:8080/#/c/14897/1/src/kudu/master/catalog_manager.cc@4282 PS1, Line 4282: std::move(actions), FLAGS_consensus_max_batch_size_bytes); > This seems rather low (1 MB). How does the consensus queue behave if this i Yes, sxactly -- the default value for --consensus_max_batch_size_bytes is just 1MB and it seems be a bit low. However this is what used in the path of ConsensusUpdate(), so I tried to stay close to that here as well. Yep, and a single ConsensusUpdate message might be arbitrarily large if that's the only message pushed to followers. And yes: all these estimates are fuzzy, so suing --rpc_max_message_size doesn't seem save enough. Maybe, using (rpc_max_message_size / 2) or (3 * rpc_max_message_size / 4) would be a safe threshold? From the other side, why do we use --consensus_max_batch_size_bytes at all then? Is it a good time to revise the --consensus_max_batch_size_bytes vs --rpc_max_message_size approach even for the consensus messages itself? -- To view, visit http://gerrit.cloudera.org:8080/14897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I83e8ca4bc8db7cab8fee6b4a40f48adc8752e7c5 Gerrit-Change-Number: 14897 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Comment-Date: Fri, 13 Dec 2019 00:24:31 +0000 Gerrit-HasComments: Yes
