hemantk-12 opened a new pull request, #5303: URL: https://github.com/apache/ozone/pull/5303
## What changes were proposed in this pull request? In this change, a new columnFamily: `compactionLogTable` is added to keep the compaction log entries. Currently simple text file is used to keep compaction entries which has following problems: * There is no proper way to serialize and deserialize the compaction entry. It is based on different token ` `, `,` and `:`. Which makes it backward incompatible whenever a new parameter gets added to logs. e.g. 1). when dbSequence number was added to the log in [HDDS-8652](https://issues.apache.org/jira/browse/HDDS-8652), 2). when it is needed to add SST file details to fix [HDDS-8940](https://issues.apache.org/jira/browse/HDDS-8940). After migrating to column family, Protobuf's serialization will be used to store it in RocksDB column family. It will be easier to add new parameter with protobuf. * There is no way to deal with potential (silent) corruption against the compaction log because it is a simple text file. Using RocksDB we can simply rely on RocksDB's built-in [per-key checksum](https://rocksdb.org/blog/2022/07/18/per-key-value-checksum.html) to deal with it rather than handling it ourselves. Migrating compaction logs to RocksDB column family will solve the above problems. Note: this is first patch to solve [HDDS-7601](https://issues.apache.org/jira/browse/HDDS-7601) and [HDDS-8940](https://issues.apache.org/jira/browse/HDDS-8940). Initially I raised single [PR-5236](https://github.com/apache/ozone/pull/5236) to fix both the issues. But over the time it is getting bigger and bigger. So I decided to create independent PRs to make it easier to review. Follow up PR would be to migrate from compaction text file to new columnFamily: `compactionLogTable`. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-7601 ## How was this patch tested? Existing unit tests `TestOMDBDefinition#testDBDefinition` and `TestOmMetadataManager#testAllTablesAreProperInOMMetadataManagerImpl`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
