David Ribeiro Alves has submitted this change and it was merged. Change subject: Add integration tests for duplicate keys ......................................................................
Add integration tests for duplicate keys This splits the "crashy nodes" and "churny elections" test of raft_consensus-itest into unique and duplicate key variants. This change is meant to stress any possible deadlock scenarios related to transaction commit/abort and 2-phase locking for which we didn't have much coverage. Additionally this also disallows timeouts on writes and requires an exact count of the rows at the end. This is now possible due to exactly once semantics. Finally this changes the cluster verifier to use snapshot scans and changes the timeout of another scan in that test. These two changes deflaked the test from 27/1000 to 3/1000 with asan, slow mode, and 1 stress thread (any more and the test becomes much more flaky, as before). Of the 3 failures two are unrelated (inability to start the webserver and a timeout on another, unrelated test). The one failure that is related to this patch is a snapshot scan anomaly and should be solved by the safe time patches. The coverage of these kinds of workloads is now much better. An example of that is that revision 12 of [1] caused a deadlock while aborting transactions out of order. Looping raft_consensus-itest in slow mode, asan, 1 stress thread would previously not fail with the buggy code and now it fails 127/1000. Results: With master (3/1000 failures): http://dist-test.cloudera.org//job?job_id=david.alves.1480767369.16205 With buggy patch[1] (127/1000 failures): http://dist-test.cloudera.org//job?job_id=david.alves.1480805849.13039 [1] - https://gerrit.cloudera.org/#/c/5294/12 Change-Id: I6d018281d412ae034bd7b70c8311077a52b2795d Reviewed-on: http://gerrit.cloudera.org:8080/5349 Reviewed-by: Todd Lipcon <[email protected]> Tested-by: Kudu Jenkins --- M src/kudu/integration-tests/cluster_verifier.cc M src/kudu/integration-tests/raft_consensus-itest.cc 2 files changed, 126 insertions(+), 75 deletions(-) Approvals: Todd Lipcon: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/5349 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I6d018281d412ae034bd7b70c8311077a52b2795d Gerrit-PatchSet: 10 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: David Ribeiro Alves <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: David Ribeiro Alves <[email protected]> Gerrit-Reviewer: Jean-Daniel Cryans <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]>
