JingsongLi opened a new issue, #1316:
URL: https://github.com/apache/paimon-rust/issues/1316

   Native upsert can lose the row count when selecting non-null 
`FixedSizeBinary(0)` values, including values nested in ROW/STRUCT columns.
   
   The native Python CI for apache/paimon#10970 built paimon-rust main at 
`5bb42a110eeb22229851e9e36205376bedb729f8` and failed four cases in 
`native_update_encoded_inputs_test.py::test_scalar_input_encodings_keep_values`:
 `empty_fixed_binary` with root/row inputs and batch/stream native upsert. The 
Python execution path passes.
   
   Failure:
   
   ```text
   cannot build selected upsert rows: Invalid argument error: all columns in a 
record batch must have the same length
   ```
   
   `TableUpsert::selected_rows` uses Arrow's `take` kernel. For zero-width 
fixed binary, that kernel constructs the result without an explicit length. 
When the selected values are all valid, the value buffer is empty and there is 
no null buffer to carry the row count, so a nonempty selection becomes a 
zero-length array. Enclosing structs can inherit the incorrect length or 
contain children with inconsistent lengths.
   
   Reproduced locally with the same failing Python test and a Rust regression 
covering row selections, nulls and nested structs. Affected columns can 
preserve their row counts by concatenating selected row slices.
   
   CI evidence: 
https://github.com/apache/paimon/actions/runs/41605967684/job/125817770399
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to