hemantk-12 opened a new pull request, #5070: URL: https://github.com/apache/ozone/pull/5070
## What changes were proposed in this pull request? Currently it is possible that RocksDB compaction adds the newer log entry to the compaction log while tarball creation is in progress because it is unaware of tarball creation. This lead to the situation when compaction log has new log entries after checkpoint was created for the follower. And when follower construct the DAG based on the compaction log, new entries nodes will become dangling nodes and will never be reached by snap diff or removed from the DAG unless OM is restarted or another bootstrap kicks in. In this change, it is proposed to wait for tarball completion before appending new log entries to the compaction log. This way follower will never get new entries. Whenever OM gets tarball request, it increases the tarball request count in the `RocksDBCheckpointDiffer` and once request finishes it decreases the request count and notify all the thread. `CompactionCompletedListener` waits till all the tarball requests finish and request count reaches to zero. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-9004 ## How was this patch tested? * Existing unit and integration tests for now. New integration test will be added as a separate PR because it would be tricky to start the compaction at the same time tarball creation kicks in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
