advancedxy commented on code in PR #2635:
URL: https://github.com/apache/iceberg-rust/pull/2635#discussion_r3412109242
##########
crates/iceberg/src/arrow/value.rs:
##########
@@ -628,23 +627,25 @@ pub(crate) fn create_primitive_array_single_element(
data_type: &DataType,
prim_lit: &Option<PrimitiveLiteral>,
) -> Result<ArrayRef> {
+ // With no value, the single element is NULL. `new_null_array` supports
every
+ // Arrow type, including nested ones (list/map/struct), which matters for
+ // columns added by schema evolution after a data file was written (#2618).
+ if prim_lit.is_none() {
Review Comment:
I think this fix is valid short-term fix. We can get it merged first, and
refactor it in a follow-up PR to address the naming issue and default value
support in Iceberg V3.
##########
crates/iceberg/src/arrow/value.rs:
##########
@@ -628,23 +627,25 @@ pub(crate) fn create_primitive_array_single_element(
data_type: &DataType,
prim_lit: &Option<PrimitiveLiteral>,
) -> Result<ArrayRef> {
+ // With no value, the single element is NULL. `new_null_array` supports
every
+ // Arrow type, including nested ones (list/map/struct), which matters for
+ // columns added by schema evolution after a data file was written (#2618).
+ if prim_lit.is_none() {
Review Comment:
Hi @viirya thanks for the fix. We encountered the same issue with a schema
evolution case: a list of binary was added, and the original code cannot handle
it. We fixed that internally and are about to contribute it back.
Our fix is similar with yours. However during the internal code review
process, we noticed the function name `create_primitive_array_single_element`
is no longer valid, it's not just creating primitive array any more, it also
creating list/struct/map arrays now. Considering the default value in Iceberg
V3, a struct could have default values. I think it would be best to change the
signature to create_array_single_element snd passes prim_lit as
`&Option<Literal>`, WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
