[
https://issues.apache.org/jira/browse/HDDS-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18108611#comment-18108611
]
Chia-Chuan Ho commented on HDDS-12834:
--------------------------------------
Hi [~tejaskriya09]
I looked into HDDS-12834 and propose making the background compaction service
CPU-aware as follows: * Use the OM JVM process CPU utilization as the load
signal.
* Before compacting each column family, compare the current utilization with a
configurable threshold (proposed default: 0.8).
* If CPU utilization is above the threshold, retain the remaining column
families and retry after a configurable interval (proposed default: 1 minute).
* Keep compaction single-threaded. An already-running RocksDB compaction will
not be interrupted.
* Once a delayed cycle completes, wait for the normal run interval before
starting another cycle; missed cycles will not be replayed.
* If CPU utilization is unavailable, fail open and preserve the current
compaction behavior.
* Do not force compaction after a maximum deferral period, to avoid adding
load while OM remains busy.
* This applies only to the background service; on-demand admin compaction
remains unchanged.
Does this match the intended behavior, particularly using OM process CPU rather
than system CPU and the proposed defaults? Thanks!
> Improve performance of auto-compaction of tables by the background service.
> ---------------------------------------------------------------------------
>
> Key: HDDS-12834
> URL: https://issues.apache.org/jira/browse/HDDS-12834
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Tejaskriya Madhan
> Priority: Major
>
> HDDS-12819 aims to introduce a background service that bottommost-compacts a
> set of column families. This process can be optimized if it is aware of the
> CPU utilization of the service that that point in time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
