leaves12138 opened a new pull request, #9897:
URL: https://github.com/apache/paimon/pull/9897
### Purpose
Wire manifest sidecars into Java scans, following the format (#9845),
lifecycle (#9886), and writer (#9889) changes. Extracted from #9743 after
rebasing the tracking PR onto current master.
- Reuse `manifest.sidecar.enabled`; no additional read option is introduced.
- Select physical Avro blocks using partition, row-ID and conservative
bucket predicates before reading manifest entries. Empty selections skip the
manifest body entirely.
- Preserve normal entry filters, statistics filtering, conversion and
ADD/DELETE reconciliation.
- Follow only explicit sidecar references. Missing, corrupt or unsupported
sidecars fall back to ordinary manifest reads. Disabled sidecars and scans
without relevant filters skip sidecar access.
- Share the manifest byte cache with sidecar and selected-block reads,
keeping block keys separate from whole-manifest entry keys. Partial results
cannot populate the whole-manifest entry cache.
- Preserve the existing `ManifestFile.read` overloads for callers that do
not supply a block selection.
No file-format, writer, lifecycle or Python changes are included.
### Tests
Java 8: 195 targeted tests passed with Checkstyle and Spotless enabled.
Tests cover row-ID, partition-only and bucket-only pruning; nullable
coverage; real seek/byte-read accounting; repeated partial/all-block cache
reads; explicit/custom references; disabled and unfiltered reads;
missing/corrupt sidecar fallback while preserving deletes and overlapping
row-ID groups; and existing data-evolution, bucket-filter, primary-key and
global-index scans.
```sh
mvn -B -ntp -nsu -pl paimon-core -am -DwildcardSuites=none
-DfailIfNoTests=false -Dflink.forkCount=2 \
-Dtest=ManifestFileTest,ManifestSidecarTest,ManifestSidecarWriteTest,ManifestBlockIndexTest,DataEvolutionFileStoreScanTest,BucketFilterScanTest,KeyValueFileStoreScanTest,DataEvolutionBatchScanTest
test
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
