[
https://issues.apache.org/jira/browse/HDDS-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18108649#comment-18108649
]
Ivan Andika edited comment on HDDS-12834 at 8/27/26 5:57 AM:
-------------------------------------------------------------
Please be careful when doing anything related to RocksDB, be careful not to
introduce issues like HDDS-15990. RocksDB is the least understood component of
Ozone community.
Ideally all RocksDB related logic need to stay in RocksDB, Ozone does not need
to come up with a new solution like a background service to compensate. You can
start by doing tuning some RocksDB configuration.
If anyone still want to go ahead on improving RocksDB compaction, I expect a
deep design doc that consider precedence in other production systems using
RocksDB.. The design need to also prove why the logic cannot be done in RocksDB
instead. I also want to see precedence in other production systems (e.g TiKV /
FoundationDB, etc) on why it's a good idea to do it, if so it's better to
imitate their implementation than invent our own. The designer need to be
RocksDB subject matter expert and understand RocksDB inside out since you are
dealing with RocksDB internals (it is not enough to say that compaction causes
CPU overhead and that is bad). Unable to do this and I will -1 the design by
default.
Also I want to see a reproducible testing setup and results for this.
So the default is that we should not implement anything related to compaction
(CompactionService arguably is not necessary).
The burden of proof of anything related to RocksDB must be high since I want to
dissuade anyone to actually doing this.
was (Author: JIRAUSER298977):
Please be careful when doing anything related to RocksDB, be careful not to
introduce issues like HDDS-15990. RocksDB is the least understood component of
Ozone community.
Ideally all RocksDB related logic need to stay in RocksDB, Ozone does not need
to come up with a new solution like a background service to compensate. You can
start by doing tuning some RocksDB configuration.
If we still want to go ahead on improving RocksDB compaction, I want to see a
deep design doc that consider precedence in other production systems using
RocksDB. Please do not come up with your own solution without checking other
production systems (e.g TiKV / FoundationDB, etc). The design need to also
prove why the logic cannot be done in RocksDB instead. The designer need to be
RocksDB subject matter expert and understand RocksDB inside out since you are
dealing with RocksDB internals (it is not enough to say that compaction causes
CPU overhead and that is bad). Unable to do this and I will -1 the design by
default.
So the default is that we should not implement anything related to compaction.
So CompactionService arguably is not necessary.
> Improve performance of auto-compaction of tables by the background service.
> ---------------------------------------------------------------------------
>
> Key: HDDS-12834
> URL: https://issues.apache.org/jira/browse/HDDS-12834
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Tejaskriya Madhan
> Priority: Major
>
> HDDS-12819 aims to introduce a background service that bottommost-compacts a
> set of column families. This process can be optimized if it is aware of the
> CPU utilization of the service that that point in time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
