Mike Percy has submitted this change and it was merged. Change subject: block_manager: various changes to disk space reservation checking ......................................................................
block_manager: various changes to disk space reservation checking I was trying to move log block container preallocation out of CreateBlock(), but it was very difficult to reason about its behavior due to how wrapped up it was with disk space checking. So here's some more yak shaving. First, disk space checking is now done in DataDirManager. I like this for several reasons: - DataDir already models a data directory; disk space checking just adds a couple fields. - It feels natural for DataDirManager to incorporate the "ignore full data directories" behavior into GetNextDataDir(). - It makes disk space checking available to all block managers. Second, its semantics are now slightly different. We no longer check if we're about to exceed the reserved space; instead, we just check if we've already exceeded it. This is simpler but it preserves the "soft" nature of the reservation behavior. The main advantage is that we can now run the check from anywhere instead of just before a file grows, which makes it easier to reason about and reduces the coupling on preallocation. Third, the published metric is now a function of data directories, not containers. I don't think a container-based metric is particularly useful here because technically all containers hosted on a data directory are "unavailable" if the underlying filesystem is full; it's more relevant to count the data directory itself. Change-Id: Ica2bb710a246ef09890554d1d6ff6da528145a6e Reviewed-on: http://gerrit.cloudera.org:8080/4831 Tested-by: Kudu Jenkins Reviewed-by: Mike Percy <mpe...@apache.org> --- M src/kudu/fs/block_manager-test.cc M src/kudu/fs/data_dirs.cc M src/kudu/fs/data_dirs.h M src/kudu/fs/file_block_manager.cc M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/integration-tests/disk_reservation-itest.cc M src/kudu/util/metrics.h 8 files changed, 268 insertions(+), 238 deletions(-) Approvals: Mike Percy: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/4831 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ica2bb710a246ef09890554d1d6ff6da528145a6e Gerrit-PatchSet: 6 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <mpe...@apache.org> Gerrit-Reviewer: Tidy Bot Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
