Ashwani Raina has posted comments on this change. ( http://gerrit.cloudera.org:8080/19278 )
Change subject: [tools] Add test to generate heavy rowset compaction ...................................................................... Patch Set 9: (2 comments) http://gerrit.cloudera.org:8080/#/c/19278/9/src/kudu/benchmarks/compaction-highmem.cc File src/kudu/benchmarks/compaction-highmem.cc: http://gerrit.cloudera.org:8080/#/c/19278/9/src/kudu/benchmarks/compaction-highmem.cc@165 PS9, Line 165: RunLoadgen > Does it make sense to increase the number of maintenance manager threads an We could do that if the test requirement is to make compaction run more frequently. Here, the main goal is to create enough deltas so that we could see to it that compaction ends up consuming lot of memory to compact those deltas. The issue at hand is that compaction (when it starts working on a set of deltas) didn't care about the memory budget and ended up consuming too much memory by allocating too many data structures in the process, that sometimes it led to OOM on certain nodes. http://gerrit.cloudera.org:8080/#/c/19278/9/src/kudu/benchmarks/compaction-highmem.cc@270 PS9, Line 270: RunLoadgen > Same as above. Same comment as above with one difference. Here, I am enabling memory budgeting flags to ensure that compaction known if there is going to be too much memory demand, it skips compaction. This helps in avoiding OOM situation. -- To view, visit http://gerrit.cloudera.org:8080/19278 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ief4ec03f1bf0f7936f8fb054948f87e71333f824 Gerrit-Change-Number: 19278 Gerrit-PatchSet: 9 Gerrit-Owner: Ashwani Raina <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Ashwani Raina <[email protected]> Gerrit-Reviewer: Attila Bukor <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Mahesh Reddy <[email protected]> Gerrit-Reviewer: Zoltan Chovan <[email protected]> Gerrit-Comment-Date: Fri, 24 Mar 2023 06:33:13 +0000 Gerrit-HasComments: Yes
