Hello Mike Percy, Todd Lipcon, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4831
to look at the new patch set (#2).
Change subject: block_manager: various changes to disk space reservation
checking
......................................................................
block_manager: various changes to disk space reservation checking
I was trying to move log block container preallocation out of CreateBlock(),
but it was very difficult to reason about its behavior due to how wrapped up
it was with disk space checking. So here's some more yak shaving.
First, disk space checking is now done in DataDirManager. I like this for
several reasons:
- DataDir already models a data directory; disk space checking just adds a
couple fields.
- It feels natural for DataDirManager to incorporate the "ignore full data
directories" behavior into GetNextDataDir().
- It makes disk space checking available to all block managers.
Second, its semantics are now slightly different. We no longer check if
we're about to exceed the reserved space; instead, we just check if we've
already exceeded it. This is simpler but it preserves the "soft" nature of
the reservation behavior. The main advantage is that we can now run the
check from anywhere instead of just before a file grows, which makes it
easier to reason about and reduces the coupling on preallocation.
Third, the published metric is now a function of data directories, not
containers. I don't think a container-based metric is particularly useful
here because technically all containers hosted on a data directory are
"unavailable" if the underlying filesystem is full; it's more relevant to
count the data directory itself.
Change-Id: Ica2bb710a246ef09890554d1d6ff6da528145a6e
---
M src/kudu/fs/block_manager-test.cc
M src/kudu/fs/data_dirs.cc
M src/kudu/fs/data_dirs.h
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/fs/log_block_manager.h
M src/kudu/integration-tests/disk_reservation-itest.cc
M src/kudu/util/env_util.cc
M src/kudu/util/metrics.h
9 files changed, 266 insertions(+), 239 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/31/4831/2
--
To view, visit http://gerrit.cloudera.org:8080/4831
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ica2bb710a246ef09890554d1d6ff6da528145a6e
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Tidy Bot
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
