Todd Lipcon has submitted this change and it was merged.

Change subject: Fix flaky disk_reservation-itest
......................................................................


Fix flaky disk_reservation-itest

There are two fixes in this patch for two separate types of failures
seen on Jenkins for this test:

1. Fix a data race in DiskReservationITest.TestFillMultipleDisks

We can't override gflag strings at runtime in a thread-safe manner,
although this test was attempting to.

Take what used to be a single parsed string gflag and replace it with 2
path strings and 2 integer overrides, one for each path. That makes 4
new test-only gflags total. Only the integer flags are modified at
runtime.

2. Fix a startup race between the TestWorkload client thread and
   SetFlags() in DiskReservationITest.TestWalWriteToFullDiskAborts

We need to wait for some rows to be written after starting up the
TestWorkload threads in TestWalWriteToFullDiskAborts before we allow the
TS to crash by setting gflags. If we don't, the test gets confused
because the TestWorkload client thread may not be able to resolve where
the tablet is located. The previous failures were because we sometimes
managed to crash the TS before it sent its tablet report to the master.

After applying these changes, I looped disk_reservation-itest 1000x in
TSAN mode and got no failures.

Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1
Reviewed-on: http://gerrit.cloudera.org:8080/3652
Tested-by: Mike Percy <mpe...@apache.org>
Reviewed-by: Todd Lipcon <t...@apache.org>
---
M src/kudu/integration-tests/disk_reservation-itest.cc
M src/kudu/util/env_util.cc
2 files changed, 61 insertions(+), 29 deletions(-)

Approvals:
  Mike Percy: Verified
  Todd Lipcon: Looks good to me, approved



-- 
To view, visit http://gerrit.cloudera.org:8080/3652
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ica86390b49d459e079807d777e97c47fa35134d1
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>

Reply via email to