RussellSpitzer opened a new pull request, #18935:
URL: https://github.com/apache/iceberg/pull/18935

   Part 2 of #17919, following #18252. 
   
   A file column's six nested fields take the IDs immediately after its own, so 
the type must stay in step with the field that holds it and those IDs must 
never go to another column. This PR enforces that in schema construction and 
evolution. `UpdateSchema.addColumn` now gives a file column, at any depth, a 
base ID that matches its field and reserves the derived block so later columns 
cannot collide. Replacing a table reserves the block when a column becomes a 
file. Committing file columns still needs schema JSON serialization, which 
comes in Part 3.
   
   
   **API changes** (no new public methods)
   - `Schema` rejects a file column whose type's base ID differs from its field 
ID. A column that takes a reserved ID is rejected by the existing duplicate-ID 
check.
   - `CheckCompatibility` accepts reading a file as a file and reports any 
other type as incompatible.
   **Core changes**
   - `UpdateSchema` rejects renaming, deleting, promoting, or changing the 
requirement, doc, or default of a file's nested fields.
   - `MetricsConfig` and `SchemaOrder` visit a file like a struct. Table 
metadata creation reaches both visitors after #18269 and #18331.
   
   
   ### Series
   - Part 1 (merged): #18252
   - Part 2: this PR
   The following parts are draft branches in my fork. They are not open PRs and 
will change as this one is reviewed. Each link shows only that part's diff 
against the previous part.
   - Part 3: [Core: Serialize the file type in schema 
JSON](https://github.com/RussellSpitzer/iceberg/compare/file-type-2-id-block...file-type-3-schema-json)
   - Part 4: [API, Core, Data: Handle the file type in type switches and struct 
views](https://github.com/RussellSpitzer/iceberg/compare/file-type-3-schema-json...file-type-4-type-switches)
   - Part 5: [Core: Read and write file columns in 
Avro](https://github.com/RussellSpitzer/iceberg/compare/file-type-4-type-switches...file-type-5-avro)
   - Part 6: [Parquet: Map the file type to a 
group](https://github.com/RussellSpitzer/iceberg/compare/file-type-5-avro...file-type-6-parquet)
   - Part 7: [ORC: Support the file type in schema conversion and 
visitors](https://github.com/RussellSpitzer/iceberg/compare/file-type-6-parquet...file-type-7-orc)
   - Part 8: [Hive, Kafka Connect, AWS, Arrow: Render a file column as a 
struct](https://github.com/RussellSpitzer/iceberg/compare/file-type-7-orc...file-type-8-peripheral)
   ---
   **AI Disclosure**
   - Model: Claude Opus 5.5
   - Platform/Tool: Cursor
   - Human Oversight: Reviewed and Guided
   - Prompt Summary: Split the file type implementation into a reviewable 
series; this part reserves the derived field ID block and guards schema 
evolution for file columns.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to