huwh opened a new pull request, #9003:
URL: https://github.com/apache/paimon/pull/9003
### Purpose
`VortexRecordsReader` derives `returnedPosition()` from a monotonically
increasing iterator, but the Vortex `DataSource` / `Scan` API is unordered by
default. If internal scan partitions complete out of order, dedicated Vortex
vector rows can be paired with the wrong rows from the main data file.
This change explicitly requests ordered Vortex scans and adds a
deterministic test for the scan contract.
This PR is kept as a draft while waiting for issue assignment and maintainer
feedback.
Fixes #9002.
### Tests
- `mvn -pl paimon-vortex/paimon-vortex-format -am -Pfast-build
-DfailIfNoTests=false -DwildcardSuites=none
-Dtest=VortexRecordsReaderTest#testScanPreservesPhysicalRowOrder test`
- Verified the regression test fails before the fix with `expected: <true>
but was: <false>`.
- The same test passes after the fix.
- `mvn -pl paimon-vortex/paimon-vortex-format -am -Pfast-build
-DfailIfNoTests=false -DwildcardSuites=none
-Dtest=VortexRecordsReaderTest,VortexReaderWriterTest test` (15 tests passed)
- `mvn -pl paimon-vortex/paimon-vortex-format -am -DskipTests package`
- `git diff --check`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
