Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/22079 )
Change subject: [compaction] Add memory tracking for better OOM issues triaging ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/22079/3/src/kudu/tablet/delta_compaction.cc File src/kudu/tablet/delta_compaction.cc: http://gerrit.cloudera.org:8080/#/c/22079/3/src/kudu/tablet/delta_compaction.cc@103 PS3, Line 103: parent_tracker_ = MemTracker::FindOrCreateGlobalTracker(-1, "major_compaction"); > Instead of creating a new parent tracker, could we just use the tablet's tr I was thinking of having a separate tracker to get a sense of how much of memory is consumed in just this type of operation rather than recording it in terms of tablet ids. I guess your point of having it recorded per tablet also makes sense since there could be some certain tablets that have higher memory consumption than others during compaction op. http://gerrit.cloudera.org:8080/#/c/22079/3/src/kudu/tablet/delta_compaction.cc@221 PS3, Line 221: size_t arena_memory_footprint_before_mutations = mem.arena.memory_footprint(); > Why do we need to update memory tracker before and after mutations? This statement is not updating memory tracker. Instead, it just stores the memory footprint before mutations are applied. After applying the mutations, it checks whether any further memory was allocated. If yes, only then it updates the memory tracker at line:232 with the difference. We are just interested in the difference of increase in memory footprint in every subsequent iteration of ApplyMutations. -- To view, visit http://gerrit.cloudera.org:8080/22079 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic2037582d433730884212e83359bb75bad0d5394 Gerrit-Change-Number: 22079 Gerrit-PatchSet: 3 Gerrit-Owner: Ashwani Raina <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy <[email protected]> Gerrit-Reviewer: Yifan Zhang <[email protected]> Gerrit-Comment-Date: Mon, 02 Dec 2024 18:23:40 +0000 Gerrit-HasComments: Yes
