raft: set threadpool timeout to be longer than heartbeat interval Looking at a cluster where the heartbeat interval has been set to 1.5 seconds, I see that there are a lot of threads starting and stopping. The issue seems to be that the Raft threadpool (on which requests are scheduled) has a default 500ms timeout, but the requests are less frequent. Thus, every request is starting and stopping a thread.
Change-Id: Ic1b11d3d9230af8fe8cc3490fed51738ae7e81a1 Reviewed-on: http://gerrit.cloudera.org:8080/4635 Tested-by: Kudu Jenkins Reviewed-by: David Ribeiro Alves <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/0ee10ea3 Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/0ee10ea3 Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/0ee10ea3 Branch: refs/heads/master Commit: 0ee10ea34cb3f68ba911524b873773a45031aa68 Parents: 4d1b1e9 Author: Todd Lipcon <[email protected]> Authored: Wed Oct 5 14:50:55 2016 -0700 Committer: Todd Lipcon <[email protected]> Committed: Thu Oct 6 01:15:19 2016 +0000 ---------------------------------------------------------------------- src/kudu/consensus/raft_consensus.cc | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kudu/blob/0ee10ea3/src/kudu/consensus/raft_consensus.cc ---------------------------------------------------------------------- diff --git a/src/kudu/consensus/raft_consensus.cc b/src/kudu/consensus/raft_consensus.cc index 3e4a89b..f09fda8 100644 --- a/src/kudu/consensus/raft_consensus.cc +++ b/src/kudu/consensus/raft_consensus.cc @@ -173,6 +173,7 @@ scoped_refptr<RaftConsensus> RaftConsensus::Create( gscoped_ptr<ThreadPool> thread_pool; CHECK_OK(ThreadPoolBuilder(Substitute("$0-raft", options.tablet_id.substr(0, 6))) .set_trace_metric_prefix("raft") + .set_idle_timeout(MonoDelta::FromMilliseconds(FLAGS_raft_heartbeat_interval_ms * 2)) .Build(&thread_pool)); DCHECK(local_peer_pb.has_permanent_uuid());
