Will Berkeley has uploaded a new patch set (#2) to the change originally
created by Todd Lipcon. ( http://gerrit.cloudera.org:8080/12615 )
Change subject: KUDU-2711 (part 3). Only send one TSInfoPB per server in
GetTableLocations
......................................................................
KUDU-2711 (part 3). Only send one TSInfoPB per server in GetTableLocations
This changes the response for GetTableLocations to send a top-level list
of TSInfoPB, and then each replica refers into that list by index. This
avoids having to copy a TSInfoPB per replica, and instead bounds the
number of serializations to the number of TS in the cluster. In the
worst case, this may slightly regress performance if the number of
replicas in the response is much fewer than the number of TS in the
cluster, since there is some extra overhead of building the index
mapping. In the best case (large number of replicas in the response)
there is a big improvement.
To handle wire compatibility, the server only uses the new response
format in the case that the client indicates it expects it by passing
a new boolean in the request.
The client handles old servers by handling a response in either format.
This patch only implements the new protocol in the C++ client. The Java client
will be updated in a follow-up patch.
Benchmarked a ~5x improvement with:
table_locations-itest --gtest_filter=\*Bench\* \
--benchmark_num_tablets 300 --benchmark_num_threads 30 \
--benchmark_runtime_secs=10
with rwlock (previous patch):
Count: 22179
Mean: 3901.19
Percentiles:
0% (min) = 52
25% = 3600
50% (med) = 3904
75% = 4192
95% = 4608
99% = 4896
99.9% = 5760
99.99% = 10048
100% (max) = 10661
with these improvements:
Count: 109590
Mean: 569.492
Percentiles:
0% (min) = 31
25% = 512
50% (med) = 556
75% = 604
95% = 716
99% = 912
99.9% = 1232
99.99% = 2528
100% (max) = 6336
Change-Id: Ief65d0825e919f681b7ade6a856c103201f8305c
---
M java/kudu-client/src/main/java/org/apache/kudu/client/AsyncKuduClient.java
M src/kudu/client/meta_cache.cc
M src/kudu/client/meta_cache.h
M src/kudu/consensus/quorum_util-test.cc
M src/kudu/consensus/quorum_util.cc
M src/kudu/consensus/quorum_util.h
M src/kudu/integration-tests/registration-test.cc
M src/kudu/integration-tests/table_locations-itest.cc
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.proto
M src/kudu/master/master_service.cc
12 files changed, 234 insertions(+), 50 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/15/12615/2
--
To view, visit http://gerrit.cloudera.org:8080/12615
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ief65d0825e919f681b7ade6a856c103201f8305c
Gerrit-Change-Number: 12615
Gerrit-PatchSet: 2
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Will Berkeley <[email protected]>
