leekeiabstraction commented on code in PR #3899:
URL: https://github.com/apache/fluss/pull/3899#discussion_r3741558383
##########
fluss-rust/crates/fluss/src/record/arrow.rs:
##########
@@ -590,15 +590,15 @@ fn estimate_arrow_ipc_overhead(
) -> Result<usize> {
use arrow::array::new_null_array;
- // Create a 1-row batch of nulls. Null arrays have minimal, predictable
- // data: no validity bitmap, no variable-length data, just fixed-width
- // zero buffers. This lets us compute raw data size exactly.
- let null_arrays: Vec<ArrayRef> = schema
- .fields()
- .iter()
- .map(|field| new_null_array(field.data_type(), 1))
- .collect();
- let batch = RecordBatch::try_new(schema.clone(), null_arrays)?;
+ let fields = schema.fields();
Review Comment:
Fix looks right to me. Worth noting the the fix unlocks the prebuilt
arrow-batch path, which does not validate the caller's batch schema against the
table's so nulls can still get written into a NOT NULL column and then fail to
decode later.
Are you OK to fix that part in this PR too or follow up with new issue/PR?
Much appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
