zhangyue19921010 opened a new issue, #11150:
URL: https://github.com/apache/paimon/issues/11150

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   ### Paimon version
   
   Master at `57233dbc1` (`2.2-SNAPSHOT`).
   
   ### Compute Engine
   
   Java batch-write API with a Data Evolution append table and dedicated JSON 
or Parquet vector files.
   
   ### Minimal reproduce step
   
   Create a table `(id INT, embedding VECTOR<FLOAT, 2>)` with 
`row-tracking.enabled=true`, `data-evolution.enabled=true` and 
`vector.file.format=json`.
   
   Using one `BatchWriteBuilder`, prepare two writers and collect their 
messages into one `BatchTableCommit`:
   
   1. A writer with 
`withWriteType(table.rowType().project(Collections.singletonList("id")))` 
writes IDs 0 and 1.
   2. A full-row writer writes `(2, [22, 0])` and `(3, [33, 0])`.
   3. Commit the first writer's messages followed by the second writer's 
messages. Read the table ordered by `id`.
   
   No file metadata or row IDs need to be modified by the application. The 
behavior also reproduces when normal files roll after each row and with 
`vector.file.format=parquet`.
   
   ### What doesn't meet your expectations?
   
   The returned vector values are `[22, 0]`, `[33, 0]`, `NULL`, `NULL` for IDs 
0–3. They should be `NULL`, `NULL`, `[22, 0]`, `[33, 0]`.
   
   The commit's vector row-ID cursor advances only for vector files. 
Normal-only files advance the normal cursor without advancing the vector 
cursor, so subsequent vectors become associated with earlier rows. Advance the 
vector cursor to at least each new normal file's start, preserving an 
already-ahead cursor for copied files.
   
   ### Anything else?
   
   The table regression fails on unchanged production for both formats. 
Validation results will accompany the PR.
   
   AI-generated with OpenAI Codex.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to