parthchandra opened a new issue, #2699:
URL: https://github.com/apache/iceberg-rust/issues/2699
follow up from #2668
The `ColumnSource` enum in `RecordBatchTransformer` currently has two
separate variants for constant columns:
```rust
Add {
target_type: DataType,
value: Option<PrimitiveLiteral>,
},
AddStructConstant {
fields: Fields,
child_values: Vec<Option<PrimitiveLiteral>>,
},
```
`AddStructConstant` was added in #2668 for the `_partition` metadata
column because Add only supports primitive values. However, iceberg-rust
already has Literal::Struct in crates/iceberg/src/spec/values/literal.rs which
can represent struct values natively.
### Proposed refactoring
1. Change ColumnSource::Add to use Option<Literal> instead of
Option<PrimitiveLiteral>:
Add {
target_type: DataType,
value: Option<Literal>,
},
2. Remove ColumnSource::AddStructConstant — struct constants become a
Literal::Struct inside Add.
3. Update create_column (and create_struct_column) to handle
Literal::Struct by recursively constructing child arrays, similar to how
AddStructConstant works today.
4. This also opens the door for constant_fields: HashMap<i32, Datum> to
eventually support struct-valued constants (e.g., if Datum gains a struct
variant), which would allow the _partition column to use the same
constant_fields path as _file and identity-partitioned columns.
### Benefits
- Eliminates a special-case enum variant
- Aligns the constant-column infrastructure with iceberg-rust's existing
Literal type system
- Makes future metadata columns or complex-typed constants straightforward
to add
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
