xanderbailey commented on issue #3411: URL: https://github.com/apache/iceberg-rust/issues/3411#issuecomment-6133203302
I got the Java side wrong above — thanks @mbutrovich for catching it on #3412. The name check does run on schema change. `Builder.add` calls `checkAndAddPartitionName` before `buildUnchecked()`, and `buildUnchecked()` only skips `checkCompatibility`: https://github.com/apache/iceberg/blob/4eb9cb7c3d53cb8080724eebf6e6898871ee23b0/api/src/main/java/org/apache/iceberg/PartitionSpec.java#L623-L628 So Java rejects the repro I wrote: `x` is still in the schema, so `sourceFieldExists` is true, `x_bucket` resolves to field id 2, and the source id is 1. "Java has no schema-side check at all" was wrong, and so was reading the `buildUnchecked()` comment as "no validation" — it means no *compatibility* check. The real divergence is narrower. Java skips the check when the partition field's source column is absent from the new schema: https://github.com/apache/iceberg/blob/4eb9cb7c3d53cb8080724eebf6e6898871ee23b0/api/src/main/java/org/apache/iceberg/PartitionSpec.java#L417-L426 We don't, so we reject there. Corrected repro: 1. v2 table, schema `x: long`, partitioned by `bucket(x)` as `x_bucket` 2. set the default spec to unpartitioned 3. add a schema that drops `x` and adds `x_bucket: string` Java accepts step 3, we reject it. Whether spec 0 is still the default isn't the axis, so my suggested fix of scoping to the current spec was wrong and #3412 is closed. #3420 has the fix: keep every spec in scope, skip a field whose source column is gone. One related divergence that PR leaves alone — our `build()` rebinds the default spec with `bind()`, which requires the source column to exist, so dropping the source column of the *current* spec still fails. Java uses `buildUnchecked()` there and tolerates it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
