Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17743 )
Change subject: KUDU-1954 [maintenance] Add an optional flush-only thread pool ...................................................................... Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/17743/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17743/3//COMMIT_MSG@7 PS3, Line 7: KUDU-1954 > For the RocksDB's example, although now a single 'max_background_jobs' is r I see, thanks for explaining. If I'm not mistaken, the --memory_pressure_percentage only determines the when we'll start prioritizing flushes vs compactions. There are other flags that determine throttling (e.g. --memory_limit_soft_percentage, --tablet_transaction_memory_limit_mb, etc.) http://gerrit.cloudera.org:8080/#/c/17743/3//COMMIT_MSG@9 PS3, Line 9: In some special environments, > Yes, this kind of environment has only 1 disk, maybe HDD, and --maintenance I guess compared to an approach in which we tune existing configs, this approach wins out when there are on-going compactions when we run into memory pressure, in which case we can flush ASAP and hope that there's enough IO bandwidth to serve both the compaction and the flush in time to free up the memory. I'm not sure how likely that is. Another thought is that perhaps for such IO- and memory- constrained environments we should reduce the tablet compaction budget (--tablet_compaction_budget_mb) to something smaller, to reduce the window during which compactions can hog IO. -- To view, visit http://gerrit.cloudera.org:8080/17743 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6dca9d87e7287cc04482a1c18b1e0eefc5f9bcb1 Gerrit-Change-Number: 17743 Gerrit-PatchSet: 3 Gerrit-Owner: Yingchun Lai <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Bankim Bhavsar <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Comment-Date: Fri, 13 Aug 2021 03:38:44 +0000 Gerrit-HasComments: Yes
