Repository: kudu Updated Branches: refs/heads/master be62673d9 -> 50c7d3249
maintenance_manager: allow runtime disabling via flag This should fix a flake in alter_table-test that was causing failures about 0.3% of the time when I looped it (and worse than that in the normal test environment). The issue was that this test sets FLAGS_enable_maintenance_manager to false at runtime, and then manually triggers compaction operations and asserts on the results. In commit 40aa4c3c271c9df20a1 I "fixed" the flag to only be checked at startup rather than in each iteration of the maintenance scheduler. Several other tests also set this flag to false, so it's possible this will reduce other flakiness as well. After the fix I looped alter_table-test 2000 times in ASAN with no failures. Prior to it, it failed 3/1000. Change-Id: I2207c9674fc1016253af038fc8fa8a14d2bb45a0 Reviewed-on: http://gerrit.cloudera.org:8080/7164 Tested-by: Kudu Jenkins Reviewed-by: Mike Percy <[email protected]> Project: http://git-wip-us.apache.org/repos/asf/kudu/repo Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/9533c57e Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/9533c57e Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/9533c57e Branch: refs/heads/master Commit: 9533c57e8b487f09d2a933545280e9ff1e42fbef Parents: be62673 Author: Todd Lipcon <[email protected]> Authored: Mon Jun 12 17:52:29 2017 -0700 Committer: Todd Lipcon <[email protected]> Committed: Tue Jun 13 02:06:45 2017 +0000 ---------------------------------------------------------------------- src/kudu/util/maintenance_manager.cc | 7 ++++++- src/kudu/util/maintenance_manager.h | 4 ++++ 2 files changed, 10 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/kudu/blob/9533c57e/src/kudu/util/maintenance_manager.cc ---------------------------------------------------------------------- diff --git a/src/kudu/util/maintenance_manager.cc b/src/kudu/util/maintenance_manager.cc index b410646..4bcd888 100644 --- a/src/kudu/util/maintenance_manager.cc +++ b/src/kudu/util/maintenance_manager.cc @@ -209,6 +209,10 @@ void MaintenanceManager::UnregisterOp(MaintenanceOp* op) { op->manager_.reset(); } +bool MaintenanceManager::disabled_for_tests() const { + return !ANNOTATE_UNPROTECTED_READ(FLAGS_enable_maintenance_manager); +} + void MaintenanceManager::RunSchedulerThread() { if (!FLAGS_enable_maintenance_manager) { LOG(INFO) << "Maintenance manager is disabled. Stopping thread."; @@ -227,7 +231,8 @@ void MaintenanceManager::RunSchedulerThread() { // 1) there are no free threads available to perform a maintenance op. // or 2) we just tried to schedule an op but found nothing to run. // However, if it's time to shut down, we want to do so immediately. - while ((running_ops_ >= num_threads_ || prev_iter_found_no_work) && !shutdown_) { + while ((running_ops_ >= num_threads_ || prev_iter_found_no_work || disabled_for_tests()) && + !shutdown_) { cond_.TimedWait(polling_interval); prev_iter_found_no_work = false; } http://git-wip-us.apache.org/repos/asf/kudu/blob/9533c57e/src/kudu/util/maintenance_manager.h ---------------------------------------------------------------------- diff --git a/src/kudu/util/maintenance_manager.h b/src/kudu/util/maintenance_manager.h index 1d2419a..2509d46 100644 --- a/src/kudu/util/maintenance_manager.h +++ b/src/kudu/util/maintenance_manager.h @@ -289,6 +289,10 @@ class MaintenanceManager : public std::enable_shared_from_this<MaintenanceManage typedef std::map<MaintenanceOp*, MaintenanceOpStats, MaintenanceOpComparator> OpMapTy; + // Return true if tests have currently disabled the maintenance + // manager by way of changing the gflags at runtime. + bool disabled_for_tests() const; + void RunSchedulerThread(); // find the best op, or null if there is nothing we want to run
