Hello Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22867
to look at the new patch set (#2).
Change subject: [WIP] Add option to send no-op heartbeat operations batched
......................................................................
[WIP] Add option to send no-op heartbeat operations batched
Due to the periodically sent heartbeat messages, a Kudu cluster having
thousands of tablets still uses significant CPU and networking even
without any user activity.
When we send multiple messages to the same host within a short
time-frame, they can be batched to reduce CPU impact.
This would result in fewer RPC calls and some fields can also be shared
between the no-op messages.
Measurement:
I launched an AWS cluster with a 1 master, 4 tserver configuration
(t3.xlarge). Then put 2000 tablets (RF=3) on them. cpu_stime seems to
be decreased by 10-15% (the same number of no-op messages arrive).
Here is one result over 500 sec for the flag being turned on or off
(change in metrics over 500 sec):
Metric: no_op_heartbeat_count, old: 920739, new: 919910, inc: -0.0900%
Metric: heartbeat_batch_count, old: 0, new: 30664, inc: n/a%
Metric: cpu_stime, old: 81208, new: 64079, inc: -21.093%
Metric: cpu_utime, old: 176558, new: 170430, inc: -3.471%
WIP cause:
Concurrent write and reads need some performance measurement
I am not sure if the heartbeat_batch_count and no_op_heartbeat_count
metrics should be added to the final code.
Change-Id: Ie92ba4de5eae00d56cd513cb644dce8fb6e14538
---
M src/kudu/client/client-test.cc
M src/kudu/consensus/CMakeLists.txt
M src/kudu/consensus/consensus.proto
M src/kudu/consensus/consensus_peers-test.cc
M src/kudu/consensus/consensus_peers.cc
M src/kudu/consensus/consensus_peers.h
A src/kudu/consensus/multi_raft_batcher.cc
A src/kudu/consensus/multi_raft_batcher.h
M src/kudu/consensus/peer_manager.cc
M src/kudu/consensus/peer_manager.h
M src/kudu/consensus/raft_consensus.cc
M src/kudu/consensus/raft_consensus.h
M src/kudu/consensus/raft_consensus_quorum-test.cc
M src/kudu/master/sys_catalog.cc
M src/kudu/master/sys_catalog.h
M src/kudu/tablet/tablet_replica-test-base.cc
M src/kudu/tablet/tablet_replica.cc
M src/kudu/tablet/tablet_replica.h
M src/kudu/tserver/tablet_copy_source_session-test.cc
M src/kudu/tserver/tablet_service.cc
M src/kudu/tserver/tablet_service.h
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
23 files changed, 765 insertions(+), 36 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/67/22867/2
--
To view, visit http://gerrit.cloudera.org:8080/22867
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie92ba4de5eae00d56cd513cb644dce8fb6e14538
Gerrit-Change-Number: 22867
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Martonka <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
