Hello Adar Dembo, Bankim Bhavsar, Todd Lipcon,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/14910
to review the following change.
Change subject: KUDU-3002: prioritize WAL unanchoring when under memory pressure
......................................................................
KUDU-3002: prioritize WAL unanchoring when under memory pressure
When under memory pressure, we currently prioritize performing the op
that will free the most memory. In theory this seems reasonable -- if
Kudu's using way too much memory, we should try to use less memory as
quickly as possible. In practice, this meant that, while under memory
pressure and under a constant, memory-consuming workload (e.g. inserts
to the MRS), Kudu would starve operations that anchor relatively little
memory (e.g. DMS flushes).
This patch updates the behavior so that we prioritize operations that
unanchor the most WAL bytes, breaking ties by prioritizing the ops that
use more memory. This seems reasonable because:
- Ops that anchor WALs also anchor memory. In performing an op that
unanchor WALs, we are performing an op that frees memory, so we'll
still prefer MRS and DMS flushing over compactions when under memory
pressure.
- We already use this heuristic when _not_ under memory pressure, but it
is only used when the ops under consideration anchor too many WAL
bytes (per --log_target_replay_size_mb), lending some credibility to
it when used conditionally.
- It becomes much more difficult to think of a scenario in which we're
"stuck" using too much space for WALs or too much memory.
Change-Id: Ibd85e8f2904a36b74cd6a3038c9ec49bb1ff9844
---
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
3 files changed, 54 insertions(+), 15 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/10/14910/1
--
To view, visit http://gerrit.cloudera.org:8080/14910
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibd85e8f2904a36b74cd6a3038c9ec49bb1ff9844
Gerrit-Change-Number: 14910
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Bankim Bhavsar <[email protected]>
Gerrit-Reviewer: Todd Lipcon <[email protected]>
