Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/23348 )
Change subject: [compaction] include undo size while picking rowsets ...................................................................... Patch Set 2: (6 comments) http://gerrit.cloudera.org:8080/#/c/23348/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/23348/2//COMMIT_MSG@7 PS2, Line 7: [compaction] > Please reference an appropriate upstream ticket for this bug. This patch is independent of the functionality introduced with KUDU-3406. The calculation is always going to take into consideration the size of UNDO deltas along with existing REDO and base data sizes. This new calculated size is used at the time of deciding whether the rowset candidate is to be picked for compaction. With this change, we don't need to get into complex steps of enabling the budgeted compaction introduced in KUDU-3406. http://gerrit.cloudera.org:8080/#/c/23348/2//COMMIT_MSG@22 PS2, Line 22: With this patch, rowset compaction never hit OOM and the resident : memory kept well within limits (~1GB). > I guess the tests were run with the compaction memory budgeting turned on? No memory budgeting configuration needs to be turned on for this. This change simply includes size of UNDO deltas(which is not being done currently) into the usual budgeting (not to be confused with budgeting code added in KUDU-3406) calculation while picking the rowsets for compaction. This essentially fixes the day-one issue of incorrect way of adding rowset candidates. Currently, only base data and redo delta sizes are taken into consideration while creating min-heap. With that, we end up adding rowsets with huge UNDO deltas as well that are eventually going to push the compaction op beyond budget limits. http://gerrit.cloudera.org:8080/#/c/23348/2//COMMIT_MSG@37 PS2, Line 37: [<redacted>,<redacted>] > nit for here and below: if this isn't adding any useful information, maybe Done http://gerrit.cloudera.org:8080/#/c/23348/2/src/kudu/tablet/rowset.h File src/kudu/tablet/rowset.h: http://gerrit.cloudera.org:8080/#/c/23348/2/src/kudu/tablet/rowset.h@206 PS2, Line 206: or UNDO deltas > here and elsewhere: please revise and update the comments where necessary Done http://gerrit.cloudera.org:8080/#/c/23348/2/src/kudu/tablet/rowset.h@207 PS2, Line 207: RedosAndUndos > here and elsewhere: change 'RedosAndUndos' into 'Deltas' for brevity? Done http://gerrit.cloudera.org:8080/#/c/23348/2/src/kudu/tablet/rowset.h@449 PS2, Line 449: not including UNDOs > ditto Done -- To view, visit http://gerrit.cloudera.org:8080/23348 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I351c0ba96a02e6ded5153adf9d981083a8c40592 Gerrit-Change-Number: 23348 Gerrit-PatchSet: 2 Gerrit-Owner: Ashwani Raina <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Attila Bukor <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Comment-Date: Thu, 08 Jan 2026 13:27:24 +0000 Gerrit-HasComments: Yes
