smiroslav opened a new pull request, #3301: URL: https://github.com/apache/jackrabbit-oak/pull/3301
https://issues.apache.org/jira/browse/OAK-12533 When an archive has no binary references entry (`.brf`), or it can't be read, `TarReader.getBinaryReferences()` returns `null` (OAK-12527). Data store GC then reports no references for that archive, and compaction cleanup rewrites it with an empty `.brf`, losing the references permanently. `SegmentStoreMigrator` and `AwsSegmentStoreMigrator` produced such archives by skipping missing binary references silently. This PR also covers that case and replaces #3274 (OAK-12516). ## Change - New `BinaryReferencesRecovery` (`file.tar`) recovers the binary references of segments with the store's `TarRecovery`, without writing anything. Long blob ids stored in other segments are resolved through the store. - `TarReader.collectBlobReferences` (data store GC) recovers the references from the segments of an archive without `.brf`, with a WARN. - `TarReader.sweep` (compaction cleanup) recovers the binary references of the remaining segments while rewriting such an archive, so the new archive has a complete `.brf`. - `ReadOnlyFileStore.recoverBinaryReferences(segments)` builds the binary references index of an archive from its segments. - `SegmentStoreMigrator` (Azure) and `AwsSegmentStoreMigrator` recover the missing binary references of a source archive with a `ReadOnlyFileStore` of the source, opened only when needed so the source is never modified, and always write a `.brf`. ## Tests - `ReadOnlyStoreBlobReferencesTest`: data store GC collects short and long references (incl. bulk segments and values stored in a previous archive) of archives without `.brf`; cleanup recovers the binary references of a rewritten archive. - `SegmentStoreMigratorTestBase` (`oak-segment-tar` test jar), run by `SegmentStoreMigratorTest` (Azurite) and `AwsSegmentStoreMigratorTest` (S3 mock), remote ↔ TAR: closed archive, archive without `.brf`, long reference with the value in a previous archive, open archive. - `TarFileTest`: sweep of an archive with `.brf` doesn't run recovery. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
