hemantk-12 opened a new pull request, #4553: URL: https://github.com/apache/ozone/pull/4553
## What changes were proposed in this pull request? Currently, it is possible that when snapDiff job is running we may loose some of compaction DAG if snapshots timing is overlapping with time by which snapshot becomes stale in compaction DAG. Other thing could happen is that DAG returned some SST file/s as diff but those files are removed by [RocksDBCheckpointDiffer#pruneOlderSnapshotsWithCompactionHistory](https://github.com/apache/ozone/blob/f3e6f443447318854b3e5ce000a68dae5237165f/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/RocksDBCheckpointDiffer.java#L1142) while reading them to generate diff report. For example, if one or both of the snapshots of snapDiff job are 30 days old and compaction DAG pruning service is removing [30 days](https://github.com/apache/ozone/blob/0ebb555fab01117dd568cafa70275acea398894c/hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneConfigKeys.java#L598-L604) older snapshots, we could be in that situation. This change is to achieve the synchronization between compaction DAG update (appending and pruning) and SnapDiff job so that snapDiff report is complete and correct instead of partial and incorrect. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-8314 ## How was this patch tested? Existing tests as of now. More tests will be added as part of [HDDS-8315](https://issues.apache.org/jira/browse/HDDS-8315). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
