DeviousCardi opened a new pull request, #3451: URL: https://github.com/apache/iceberg-rust/pull/3451
## Which issue does this PR close? - Closes #3403. ## What changes are included in this PR? The spec reserves field ids above 2147483447 (`Integer.MAX_VALUE - 200`) for metadata columns such as `_file`. Table metadata accepted schemas that used them, and the reader then resolved such a user column as the metadata column with the same id. `TableMetadataBuilder::add_schema` now rejects a schema whose highest field id is above the limit, and `try_normalize` (run after JSON deserialization and in `build()`) applies the same check to every schema in the metadata. The error quotes the limit. Things that may not be obvious from the diff: - The check is at the table metadata level, not in `SchemaBuilder::build`, because the crate builds internal schemas with reserved ids (position delete schema, scan projections with `_file`, `_pos`, etc.). Those never go through table metadata, so they are unaffected. - `Schema::highest_field_id()` already covers nested struct, list and map fields, so no new traversal is needed. - On load, all schemas are checked, not just the current one. Metadata written elsewhere with a reserved id in a historical schema will now fail to load. - Partition field ids are not checked. They are allocated from 1000 upward and live in the partition tuple, not the table schema; the issue scopes this to schemas. ## Are these changes tested? Unit tests: boundary id 2147483447 accepted and 2147483448 rejected through `add_schema` (top-level and nested field, v1-v3) and through JSON deserialization (v2, v3). Existing `_file`/`_pos` scan projection tests still pass. ## AI Disclosure This change was written with Claude Code and reviewed by the author. Areas a reviewer should focus on: - Whether rejecting reserved ids in historical (non-current) schemas on load is too strict for metadata written by other implementations, versus checking only the current schema. - Whether partition field ids should also be bounded. - The constant is private to `table_metadata.rs`; it could instead live next to the `RESERVED_FIELD_ID_*` constants in `metadata_columns.rs`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
