Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19413
Change subject: KUDU-3406 corrected estimate for ancient UNDO delta size ...................................................................... KUDU-3406 corrected estimate for ancient UNDO delta size When looking into micro-benchmark results produced by the $KUDU_HOME/src/kudu/scripts/benchmarks.sh script, I noticed that dense_node-itest showed 9-fold increase in the number of blocks under management. Even if the test disables GC of the ancient UNDO deltas (i.e. it runs with --enable_undo_delta_block_gc=false), that's not the expected behavior. It turned out the issue was in the way how DeltaTracker::EstimateBytesInPotentiallyAncientUndoDeltas() operated: it always treated a delta to be ancient if no stats were present. So, if a delta file was lazily loaded without stats being read, DeltaTracker assumed all its deltas were ancient. With the new behavior introduced in 1556a353e, it lead to rowset merge compactions skipping the newly generated UNDO deltas since the estimate reported 100% of the deltas being ancient. While this was not manifesting itself in real-world scenarios involving some tangible amount of data being written, the artificial "metadata only" workload exercised by dense_node-itest allowed the issue to manifest itself. This patch addresses the issue, introducing different estimate modes for the method mentioned above and using proper modes in various contexts. I verified that with this patch added, the benchmark based on dense_node-itest now reports the number of blocks as it has been reporting for the longest span of its history. So I'm not adding any new tests with this patch. This is a follow-up to 1556a353e60c5d555996347cbd46d5e5a6661266. Change-Id: I17bddae86f84792caf14fb1e11a6e1c0d7a92b56 --- M src/kudu/tablet/delta_tracker.cc M src/kudu/tablet/delta_tracker.h M src/kudu/tablet/diskrowset.cc M src/kudu/tablet/diskrowset.h M src/kudu/tablet/memrowset.h M src/kudu/tablet/mock-rowsets.h M src/kudu/tablet/rowset.h M src/kudu/tablet/tablet.cc 8 files changed, 56 insertions(+), 28 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/13/19413/1 -- To view, visit http://gerrit.cloudera.org:8080/19413 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I17bddae86f84792caf14fb1e11a6e1c0d7a92b56 Gerrit-Change-Number: 19413 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin <[email protected]>
