Mike Percy has submitted this change and it was merged.
Change subject: Allow for reserving disk space for non-Kudu processes
Allow for reserving disk space for non-Kudu processes
Adds gflags to reserve disk space such that Kudu will not use more than
specified. Hadoop calls this functionality "du.reserved".
If a WAL preallocation is attempted while the log disk is past its
reservation limit, the log write will fail. As a result, RaftConsensus
will cause the process to crash.
The log block manager will use non-full disks if possible until all of
the disks are full. If a flush or compaction is attempted when all disks
are beyond their configured capacity then the LogBlockManager will
return an error. As a result, the maintenance manager task will cause
the process to crash.
This initial implementation provides a "best effort" approach. Disk
space checks are only done at preallocation time, and if writes continue
beyond the preallocated point (for both a WAL segment and a data block)
those writes will not be prevented. This makes it easier to provide a
"friendly" option where the block manager will divert new writes to
non-full disks, avoiding a hard crash when only one disk is past its
In the future, we may want to add "hard" and "soft" limits, such that
going beyond the soft limit will do what we do today, and going beyond
the hard limit (say, by writing a very large data block past its
preallocation point) will result in a crash.
This patch includes:
* Unit tests.
* End-to-end test for flushing / compaction falling back to non-full
disks due to disk space backpressure and finally crashing when there
is no space left in any data dir.
* End-to-end test for writes failing due to WAL disk space backpressure,
causing a crash.
Reviewed-by: Adar Dembo <a...@cloudera.com>
Tested-by: Mike Percy <mpe...@apache.org>
Reviewed-by: Jean-Daniel Cryans <jdcry...@apache.org>
Reviewed-by: Todd Lipcon <t...@apache.org>
14 files changed, 613 insertions(+), 76 deletions(-)
Jean-Daniel Cryans: Looks good to me, but someone else must approve
Mike Percy: Verified
Adar Dembo: Looks good to me, approved
Todd Lipcon: Looks good to me, but someone else must approve
To view, visit http://gerrit.cloudera.org:8080/3135
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Owner: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Adar Dembo <a...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jdcry...@apache.org>
Gerrit-Reviewer: Mike Percy <mpe...@apache.org>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>