huan233usc opened a new pull request, #941:
URL: https://github.com/apache/iceberg-cpp/pull/941
## Summary
Iceberg v3 write-default values must be materialized when a writer input
schema omits a table field. The current Writer API only carries one schema, so
format writers cannot distinguish the incoming Arrow layout from the target
file layout.
This PR introduces the shared API and Arrow alignment layer before wiring it
into Parquet and Avro writers:
- add an optional WriterOptions input_schema while preserving the current
same-schema default
- align top-level and nested struct fields by Iceberg field ID
- materialize missing write-default values as constant Arrow arrays
- materialize missing optional fields as null and reject missing required
fields
- preserve explicit nulls and reject incompatible or unexpected input fields
Parquet and Avro integration is intentionally deferred to follow-up PRs, so
this PR does not change writer behavior yet.
## Test Plan
- [x] Added unit coverage for reordered fields, distinct initial/write
defaults, optional null filling, and explicit null preservation
- [x] Added nested struct alignment coverage
- [x] Added rejection coverage for missing required fields, incompatible
types, and extra input fields
- [x] Compiled the new implementation and test translation units with
warnings as errors
- [x] Ran WriteSchemaUtilTest: 5 tests passed
- [ ] Run the normal clean CMake CI matrix; local CMake regeneration is
currently blocked by an existing utf8proc/Arrow target conflict
## Verification Commands
cmake --build build --target data_test
./build/src/iceberg/test/data_test --gtest_filter=WriteSchemaUtilTest.*
Tracking: #637
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
