SaketaChalamchala opened a new pull request, #11296: URL: https://github.com/apache/ozone/pull/11296
## What changes were proposed in this pull request? Developed with the help of Cursor AI. This PR is intended as the foundation for a more efficient snapshot diff ([HDDS-9154](https://issues.apache.org/jira/browse/HDDS-9154)). In the proposed approach the sequential readers [HDDS-15394](https://issues.apache.org/jira/browse/HDDS-15394) and [HDDS-15393](https://issues.apache.org/jira/browse/HDDS-15393) persist sorted(by objectId) new/old diff candidate lists and parent/child edge indexes(FSO only) in temporary per-job RocksDB column families. This change completes the optimized snapshot diff pipeline: - Classifies the diff candidates as CREATE/DELETE/RENAME?MODIFY using merge-join. - FSO only: - Filters out, resolves full path and reports top-level deletes. - Resolves the full paths of the remaining report entries. - If dependency ordering is enabled, topologically sorts the entries using rules defined in [HDDS-15390](https://issues.apache.org/jira/browse/HDDS-15390). - Writes the report to the report table. - **`MergeJoinSnapDiffWriter`** — orchestrates merge join, delete retention, path resolution, optional dependency ordering, and report write. - **`SnapDiffJobStore`** — extended with classified column families, report batching, spill for deleted/renamed directory IDs and diff candidates (bounded by `maxInMemoryEntries`), RocksDB iterators for lists/classified rows/dependency nodes, and inlined wire-format helpers for classified entries. - **`SnapDiffPathResolver`** — resolves bucket-relative paths by walking reverse edge links with LRU memoization. - **`SnapDiffDependencyGraph`** — consumes an iterator of dependency nodes and persists topological order back to the job store for batched report lookup. - **`SnapshotDiffManager`** — adds `generateDiffReportOptimized(...)`, exposes `getReportKeyForIndex(...)`, and refactors `hasDeletedAncestors(...)` behind small functional interfaces for store-backed directory checks. Memory is bounded during FSO processing: ancestor memo, deleted-directory IDs, and renamed-directory IDs spill to RocksDB after `maxInMemoryEntries`. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-15391 ## How was this patch tested? Unit Tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
