leaves12138 opened a new pull request, #9835:
URL: https://github.com/apache/paimon/pull/9835
### Purpose
Expose the original Avro object-container header and physical block
offsets/lengths through `RawBlockReader`, `AvroBlockReader`, and
`ManifestAvroReader`. These locations allow callers to index existing manifest
blocks and reconstruct valid Avro streams from selected blocks. Offsets account
for decoder read-ahead, and encoded lengths include the block header and sync
marker.
This is the first standalone piece extracted from the manifest sidecar work
in #9743. It only adds block metadata APIs and tests; sidecar generation, scan
pruning, and caching will follow separately.
### Tests
- Added boundary checks against `DataFileWriter.sync()` for null, deflate,
Snappy, and Zstandard codecs, including large headers, one-byte/short reads,
repeated look-ahead, borrowed-block reuse, and empty files.
- Reconstructed each block with the original header and verified its records
using the standard Avro reader.
- Extended manifest integration coverage to verify header preservation and
contiguous physical block coverage.
- Passed 109 tests on Java 8, with the normal Maven validation checks
enabled:
```bash
mvn -B -ntp -nsu -pl paimon-core -am \
-DwildcardSuites=none -DfailIfNoTests=false \
-Dtest=AvroBlockReaderTest,AvroFileFormatTest,ManifestFileTest clean test
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
