zhangyue19921010 opened a new pull request, #11151:
URL: https://github.com/apache/paimon/pull/11151
### Purpose
Fixes #11150.
When a Java batch commit collects normal-only files before files with
vectors, the vector row-ID cursor still points at the earlier rows. Reading IDs
0–3 can then return vectors `[22, 33, NULL, NULL]` instead of `[NULL, NULL, 22,
33]`.
Advance the vector cursor to each new normal file's start when it lags
behind, keeping dedicated vector files aligned with their rows without
rewinding an already-ahead copied-file cursor.
### Tests
Added a table regression using two writers and one commit, with no manually
constructed metadata. It covers JSON and Parquet vector files and normal-file
rolling. An existing copied-file fixture also verifies that an already-ahead
vector cursor is preserved. Both table cases fail on unchanged production at
`57233dbc1` with vectors attached to the wrong IDs and pass with the change.
55 related tests passed on JDK 11, including dedicated-file rolling, vector
updates/compaction and copied-file row tracking. Checkstyle, Spotless, Enforcer
and RAT passed without `fast-build`. The Maven `verify` lifecycle ran the
selected suite twice.
```sh
mvn -pl paimon-core -am -DwildcardSuites=none -DfailIfNoTests=false \
'-Dtest=VectorStoreTableTest,DedicatedFormatRollingFileWriterVectorTest,RowTrackingCopiedFilesTest,CopiedDataFilesTest,DataEvolutionTableTest#testPartitionGroupedRowIdsWithBlobFiles+testDataEvolutionWriteRollsByRows+testCompactPreservesConcurrentPartialUpdateWithinCandidateRange'
verify
```
AI-generated with OpenAI Codex; independently reviewed by an AI agent. No
human review is claimed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]