[kudu-CR] Fix flaky disk reservation-itest
Todd Lipcon has posted comments on this change. Change subject: Fix flaky disk_reservation-itest .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/3652 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] Fix flaky disk reservation-itest
Todd Lipcon has submitted this change and it was merged. Change subject: Fix flaky disk_reservation-itest .. Fix flaky disk_reservation-itest There are two fixes in this patch for two separate types of failures seen on Jenkins for this test: 1. Fix a data race in DiskReservationITest.TestFillMultipleDisks We can't override gflag strings at runtime in a thread-safe manner, although this test was attempting to. Take what used to be a single parsed string gflag and replace it with 2 path strings and 2 integer overrides, one for each path. That makes 4 new test-only gflags total. Only the integer flags are modified at runtime. 2. Fix a startup race between the TestWorkload client thread and SetFlags() in DiskReservationITest.TestWalWriteToFullDiskAborts We need to wait for some rows to be written after starting up the TestWorkload threads in TestWalWriteToFullDiskAborts before we allow the TS to crash by setting gflags. If we don't, the test gets confused because the TestWorkload client thread may not be able to resolve where the tablet is located. The previous failures were because we sometimes managed to crash the TS before it sent its tablet report to the master. After applying these changes, I looped disk_reservation-itest 1000x in TSAN mode and got no failures. Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 Reviewed-on: http://gerrit.cloudera.org:8080/3652 Tested-by: Mike Percy Reviewed-by: Todd Lipcon --- M src/kudu/integration-tests/disk_reservation-itest.cc M src/kudu/util/env_util.cc 2 files changed, 61 insertions(+), 29 deletions(-) Approvals: Mike Percy: Verified Todd Lipcon: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/3652 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 Gerrit-PatchSet: 2 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon
[kudu-CR] Fix flaky disk reservation-itest
Mike Percy has posted comments on this change. Change subject: Fix flaky disk_reservation-itest .. Patch Set 1: Verified+1 Overriding Jenkins failure due to flaky Java test filed as KUDU-1531 -- To view, visit http://gerrit.cloudera.org:8080/3652 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy Gerrit-Reviewer: Mike Percy Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No
[kudu-CR] Fix flaky disk reservation-itest
Hello Todd Lipcon, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/3652 to review the following change. Change subject: Fix flaky disk_reservation-itest .. Fix flaky disk_reservation-itest There are two fixes in this patch for two separate types of failures seen on Jenkins for this test: 1. Fix a data race in DiskReservationITest.TestFillMultipleDisks We can't override gflag strings at runtime in a thread-safe manner, although this test was attempting to. Take what used to be a single parsed string gflag and replace it with 2 path strings and 2 integer overrides, one for each path. That makes 4 new test-only gflags total. Only the integer flags are modified at runtime. 2. Fix a startup race between the TestWorkload client thread and SetFlags() in DiskReservationITest.TestWalWriteToFullDiskAborts We need to wait for some rows to be written after starting up the TestWorkload threads in TestWalWriteToFullDiskAborts before we allow the TS to crash by setting gflags. If we don't, the test gets confused because the TestWorkload client thread may not be able to resolve where the tablet is located. The previous failures were because we sometimes managed to crash the TS before it sent its tablet report to the master. After applying these changes, I looped disk_reservation-itest 1000x in TSAN mode and got no failures. Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 --- M src/kudu/integration-tests/disk_reservation-itest.cc M src/kudu/util/env_util.cc 2 files changed, 61 insertions(+), 29 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/3652/1 -- To view, visit http://gerrit.cloudera.org:8080/3652 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy Gerrit-Reviewer: Todd Lipcon
[kudu-CR] Fix flaky disk reservation-itest
Kudu Jenkins has posted comments on this change. Change subject: Fix flaky disk_reservation-itest .. Patch Set 1: Build Started http://104.196.14.100/job/kudu-gerrit/2439/ -- To view, visit http://gerrit.cloudera.org:8080/3652 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1 Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-HasComments: No