anoopj opened a new issue, #2842: URL: https://github.com/apache/iceberg-rust/issues/2842
### Apache Iceberg Rust version None ### Describe the bug The manifests metadata table panics (rather than returning an error) for tables where an old partition spec references a source column that no longer exists in the current schema. Note that this is a legal table state: in format v2+, a column may be dropped once it is no longer used by the default spec, but historical specs referencing it are retained in table metadata because existing manifests still use them. ### To Reproduce 1. Create a v2/v3 table with schema (id int, is_test boolean) and spec identity(is_test) (spec 0), and write at least one snapshot so a manifest with spec 0 exists. 2. Evolve the spec to remove the is_test partition field (spec 1 becomes default). 3. Drop the is_test column from the schema. 4. Load the table and collect table.inspect().manifests().scan(). Step 4 panics. ### Expected behavior At minimum, the inspect table should return an Err in the stream instead of panicking. An unreadable metadata table row should never abort the process as this can cause serious availability issues. Longer term, partition_type doesn't need to fail here at all: for most transforms (bucket, year/month/day/hour, void) the result type is independent of the source column and can be derived from the transform alone; only identity and truncate genuinely need the source type. iceberg-java hit this same problem while adding the v4 manifest reader (see the discussion in apache/iceberg#16958) and is fixing it in apache/iceberg#17262: derive the output type from the transform when it doesn't depend on the source, and fall back to the v3 unknown type only for source-dependent transforms. I intend to file follow-up issues to bring partition_type in line with that (which for rust also depends on adding the unknown type), plus a related scan-planning failure with the same root cause. This issue is scoped to the `panic`. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
