JingsongLi opened a new issue, #1316: URL: https://github.com/apache/paimon-rust/issues/1316
Native upsert can lose the row count when selecting non-null `FixedSizeBinary(0)` values, including values nested in ROW/STRUCT columns. The native Python CI for apache/paimon#10970 built paimon-rust main at `5bb42a110eeb22229851e9e36205376bedb729f8` and failed four cases in `native_update_encoded_inputs_test.py::test_scalar_input_encodings_keep_values`: `empty_fixed_binary` with root/row inputs and batch/stream native upsert. The Python execution path passes. Failure: ```text cannot build selected upsert rows: Invalid argument error: all columns in a record batch must have the same length ``` `TableUpsert::selected_rows` uses Arrow's `take` kernel. For zero-width fixed binary, that kernel constructs the result without an explicit length. When the selected values are all valid, the value buffer is empty and there is no null buffer to carry the row count, so a nonempty selection becomes a zero-length array. Enclosing structs can inherit the incorrect length or contain children with inconsistent lengths. Reproduced locally with the same failing Python test and a Rust regression covering row selections, nulls and nested structs. Affected columns can preserve their row counts by concatenating selected row slices. CI evidence: https://github.com/apache/paimon/actions/runs/41605967684/job/125817770399 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
