smiroslav opened a new pull request, #3274:
URL: https://github.com/apache/jackrabbit-oak/pull/3274

   https://issues.apache.org/jira/browse/OAK-12516
   
   ## Problem
   For remote segment stores the binary references (`.brf`) of an archive are 
written only when the archive is closed. They are missing while the archive is 
still open, or when the Oak process terminated before closing it. 
`SegmentStoreMigrator` skipped the missing binary references silently, so the 
migrated TAR archive had no `.brf` entry and 
`SegmentTarReader.getBinaryReferences()` failed with 
`InvalidBinaryReferencesIndexException: Unrecognized magic number`.
   
   ## Change
   - `SegmentStoreMigrator` (Azure) reads the binary references of the source 
archive first. If they are missing, it rebuilds them from the migrated segments 
(with a WARN) and writes them after the segments, so the TAR layout stays 
`[segments][.brf][.gph][.idx]`.
   - New `BinaryReferencesCollector` (`oak-segment-tar`, package 
`file.tar.binaries`, not exported) rebuilds the index from the segments of an 
archive. Long blob ids whose value is stored in another archive are read from a 
fallback store. The migrator opens the source as a `ReadOnlyFileStore` for 
that, lazily and only when needed, so the source is never recovered or modified.
   - `BinaryReferencesCollector.readBinaryReferences(Segment, Map)` is now the 
single place that reads the blob ids of a segment; `FileStore.writeSegment` and 
TAR recovery (`AbstractFileStore.populateTarBinaryReferences`) use it as well. 
No behaviour change there.
   - The error log of a failed archive migration now names the archive.
   
   ## Tests
   - `SegmentStoreMigratorTest` (Azurite): closed archive (binary references 
copied unchanged), open archive, archive not closed before termination, and a 
long blob id whose value is in a previous archive (also asserts the source 
archive was not recovered).
   - `BinaryReferencesCollectorTest`: rebuilt index equals the written index 
for short ids, long ids, ids stored in bulk segments and ids whose value is in 
a previous segment; behaviour with and without fallback store.
   - `LongIdMappingBlobStore` accepts an id length to produce these cases.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to