Hello Ashwani Raina, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19413
to look at the new patch set (#2).
Change subject: KUDU-3406 corrected estimate for ancient UNDO delta size
......................................................................
KUDU-3406 corrected estimate for ancient UNDO delta size
When looking into micro-benchmark results produced by the
$KUDU_HOME/src/kudu/scripts/benchmarks.sh script, I noticed that
dense_node-itest showed 9-fold increase in the number of blocks under
management. Even if the test disables GC of the ancient UNDO deltas
(i.e. it runs with --enable_undo_delta_block_gc=false), that's not the
expected behavior. It turned out the issue was in the way how
DeltaTracker::EstimateBytesInPotentiallyAncientUndoDeltas() operated: it
always treated a delta to be ancient if no stats were present. So, if a
delta file was lazily loaded without stats being read, DeltaTracker
assumed all its deltas were ancient. With the new behavior introduced
in 1556a353e, it lead to rowset merge compactions skipping the newly
generated UNDO deltas since the estimate reported 100% of the deltas
being ancient.
While this was not manifesting itself in real-world scenarios involving
some tangible amount of data being written, the artificial
"metadata only" workload exercised by dense_node-itest allowed the issue
to manifest itself.
This patch addresses the issue, introducing different estimate modes for
the method mentioned above and using proper modes in various contexts.
I verified that with this patch added, the benchmark based on
dense_node-itest now reports the number of blocks as it has been
reporting for the longest span of its history. So I'm not adding any
new tests with this patch.
This is a follow-up to 1556a353e60c5d555996347cbd46d5e5a6661266.
Change-Id: I17bddae86f84792caf14fb1e11a6e1c0d7a92b56
---
M src/kudu/tablet/delta_tracker.cc
M src/kudu/tablet/delta_tracker.h
M src/kudu/tablet/diskrowset.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/memrowset.h
M src/kudu/tablet/mock-rowsets.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/tablet.cc
8 files changed, 57 insertions(+), 28 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/19413/2
--
To view, visit http://gerrit.cloudera.org:8080/19413
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I17bddae86f84792caf14fb1e11a6e1c0d7a92b56
Gerrit-Change-Number: 19413
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Ashwani Raina <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
