Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15113 )
Change subject: KUDU-3046: deflake TabletServerQuiescingITest ...................................................................... Patch Set 4: (3 comments) http://gerrit.cloudera.org:8080/#/c/15113/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/15113/3//COMMIT_MSG@10 PS3, Line 10: - Slowness in TSAN mode along with a low Raft timeout meant workloads : would fail to even create tablets. : - Addressed this by increasing the heartbeat interval in TSAN mode. > This happens to us quite often. How do you feel about adding a DCHECK or so I think it'd be pretty difficult to enforce this is doing the right thing. Sometimes the super low timeout is exactly what we want, eg in stress tests. http://gerrit.cloudera.org:8080/#/c/15113/3/src/kudu/integration-tests/tablet_server_quiescing-itest.cc File src/kudu/integration-tests/tablet_server_quiescing-itest.cc: http://gerrit.cloudera.org:8080/#/c/15113/3/src/kudu/integration-tests/tablet_server_quiescing-itest.cc@a197 PS3, Line 197: > Yup, we have some test which run with ranges 25ms -- 200ms hearbeat interva It wasn't really necessary once we began more aggressively transferring leadership. Though I suppose it doesn't hurt to increase the frequency of Raft communication since it still means waiting 100ms instead of 500ms. Done http://gerrit.cloudera.org:8080/#/c/15113/3/src/kudu/integration-tests/tablet_server_quiescing-itest.cc@699 PS3, Line 699: // step it down, but it's hard to guarantee that the found leader is of the > Indeed, what if the tablet servers were in election cycle already when a re I think the 2x wait time might still be insufficient because there still might be multiple leaders of different terms when we go out to find a leader. I opted to go with Adar's approach rather than sleeping. I also wrapped this in an ASSERT_EVENTUALLY for good measure. -- To view, visit http://gerrit.cloudera.org:8080/15113 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I3f9ef531062c4b66648840e04962070768fbad5d Gerrit-Change-Number: 15113 Gerrit-PatchSet: 4 Gerrit-Owner: Andrew Wong <[email protected]> Gerrit-Reviewer: Adar Dembo <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Tue, 28 Jan 2020 21:07:03 +0000 Gerrit-HasComments: Yes
