Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/19197 )
Change subject: KUDU-3406: add --memory_flush_prioritization_prob flag ...................................................................... Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/19197/1/src/kudu/util/maintenance_manager.cc File src/kudu/util/maintenance_manager.cc: http://gerrit.cloudera.org:8080/#/c/19197/1/src/kudu/util/maintenance_manager.cc@532 PS1, Line 532: peformance degrages nit: spell correction -> performance degrades http://gerrit.cloudera.org:8080/#/c/19197/1/src/kudu/util/maintenance_manager.cc@538 PS1, Line 538: if (!best_perf_improvement_op || best_perf_improvement <= 0 || Out of curiosity, is it possible that best_perf_improvement_op itself is a data flush op for most iterations because of the logic inside SetPerfImprovementForFlush that may set the perf improvement result for flush op to a higher number (than even compaction) if anchored size exceeds the threshold size: ++++ if (anchored_mb >= threshold_mb) { // If we're over the user-specified flush threshold, then consider the perf // improvement to be 1 for every extra MB (at least 1). This produces perf_improvement // results which are much higher than most compactions would produce, and means that, // when there is an MRS over threshold, a flush will almost always be selected instead of // a compaction. That's not necessarily a good thing, but in the absence of better // heuristics, it will do for now. double extra_mb = anchored_mb - threshold_mb; DCHECK_GE(extra_mb, 0); stats->set_perf_improvement(std::max(1.0, extra_mb)); +++++ If that is the case, we would keep running flush ops and ignore compaction even with this random skip logic. Although, it is less likely that we will keep hitting the threshold size so frequently. But, just wanted to mention it out there. It would be worth testing a high speed flush activity scenario to see compaction is chosen as best perf op on some occasions if not frequently. http://gerrit.cloudera.org:8080/#/c/19197/1/src/kudu/util/maintenance_manager.cc@549 PS1, Line 549: prioritizing perf improvement nit: prioritisation of perf improvement ops -- To view, visit http://gerrit.cloudera.org:8080/19197 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I07e33bcebe7284da20e57dfbc9d87070f92fd6f3 Gerrit-Change-Number: 19197 Gerrit-PatchSet: 1 Gerrit-Owner: Alexey Serbin <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Comment-Date: Thu, 03 Nov 2022 11:48:41 +0000 Gerrit-HasComments: Yes
