Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9897 )
Change subject: KUDU-2394. Fix flaky RpcStubTest.TestDontHandleTimedOutCalls ...................................................................... KUDU-2394. Fix flaky RpcStubTest.TestDontHandleTimedOutCalls This test was flaky under stress+ASAN with three cases of failures: (1) most commonly, the assertion to check that an RPC had timed out in the queue would run too quickly, just before the actual metric was incremented. Changing it to ASSERT_EVENTUALLY got rid of most of the failures. (2) in a few cases, the 100ms sleeps were processed completely before the later RPC was processed, so the RPC that was supposed to time out in the queue actually had a chance to run. Switching to 1000ms sleeps got rid of the remaining failure case. (3) in a few other cases, the call that was supposed to time out (with a 1ms timeout) timed out before it was even sent to the server. This patch bumps the timeout to 5ms and retries that until it succeeds with a server-side timeout. Ran 5000x with the following command: ./build-support/dist_test.py loop -n 500 build/latest/bin/rpc_stub-test \ --gtest_filter=\*DontHandle\* --gtest_repeat=10 --stress-cpu-threads=8 and got no failures after this patch, whereas I'd usually get 5-10 failures prior. Change-Id: If8cfebb5e9160fb11586abdbba6fc1ae06574a2c Reviewed-on: http://gerrit.cloudera.org:8080/9897 Tested-by: Kudu Jenkins Reviewed-by: Adar Dembo <a...@cloudera.com> --- M src/kudu/rpc/rpc_stub-test.cc 1 file changed, 23 insertions(+), 9 deletions(-) Approvals: Kudu Jenkins: Verified Adar Dembo: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/9897 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: If8cfebb5e9160fb11586abdbba6fc1ae06574a2c Gerrit-Change-Number: 9897 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Andrew Wong <andrew.w...@cloudera.com> Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
