smiroslav opened a new pull request, #3301:
URL: https://github.com/apache/jackrabbit-oak/pull/3301

   https://issues.apache.org/jira/browse/OAK-12533
   
   When an archive has no binary references entry (`.brf`), or it can't be 
read, `TarReader.getBinaryReferences()` returns `null` (OAK-12527). Data store 
GC then reports no references for that archive, and compaction cleanup rewrites 
it with an empty `.brf`, losing the references permanently. 
`SegmentStoreMigrator` and `AwsSegmentStoreMigrator` produced such archives by 
skipping missing binary references silently. This PR also covers that case and 
replaces #3274 (OAK-12516).
   
   ## Change
   - New `BinaryReferencesRecovery` (`file.tar`) recovers the binary references 
of segments with the store's `TarRecovery`, without writing anything. Long blob 
ids stored in other segments are resolved through the store.
   - `TarReader.collectBlobReferences` (data store GC) recovers the references 
from the segments of an archive without `.brf`, with a WARN.
   - `TarReader.sweep` (compaction cleanup) recovers the binary references of 
the remaining segments while rewriting such an archive, so the new archive has 
a complete `.brf`.
   - `ReadOnlyFileStore.recoverBinaryReferences(segments)` builds the binary 
references index of an archive from its segments.
   - `SegmentStoreMigrator` (Azure) and `AwsSegmentStoreMigrator` recover the 
missing binary references of a source archive with a `ReadOnlyFileStore` of the 
source, opened only when needed so the source is never modified, and always 
write a `.brf`.
   
   ## Tests
   - `ReadOnlyStoreBlobReferencesTest`: data store GC collects short and long 
references (incl. bulk segments and values stored in a previous archive) of 
archives without `.brf`; cleanup recovers the binary references of a rewritten 
archive.
   - `SegmentStoreMigratorTestBase` (`oak-segment-tar` test jar), run by 
`SegmentStoreMigratorTest` (Azurite) and `AwsSegmentStoreMigratorTest` (S3 
mock), remote ↔ TAR: closed archive, archive without `.brf`, long reference 
with the value in a previous archive, open archive.
   - `TarFileTest`: sweep of an archive with `.brf` doesn't run recovery.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to