mfellner commented on issue #3041: URL: https://github.com/apache/iceberg-rust/issues/3041#issuecomment-6112673308
Related to #2411 (v3 tracking). Related to #2418 (v2 tables accept v3-only schema types) and #2580 (RowDelta and deletion-vector writer, which will need this gate). Concrete cases a per-action version gate would need to cover, checked against main at 050ddfda8. "(run)" means reproduced with unit tests in a scratch copy of main. "(read)" means from code reading only. - **Position delete files on v3 (read).** The spec's Delete Formats section says "Position delete files must not be added to v3 tables, but existing position delete files are valid." The `PositionDeleteFileWriter` module docs say v3 "forbids adding new position delete files, so callers must not route v3 writes here", and that format-version gating "belongs at the transaction/commit layer" (`crates/iceberg/src/writer/base_writer/position_delete_writer.rs:29-32`). Its builder takes no table metadata (`position_delete_writer.rs:194`). No transaction action checks the format version for delete files. `FastAppend` is the only action that adds files, as #2580 also notes. - **Delete content is rejected by content type, not by version (run).** `fast_append` of a `PositionDeletes` file fails with `DataInvalid => Only data content type is allowed for fast append` on both a v2 and a v3 test table. Open #2203 returns `FeatureUnsupported` at commit whenever delete files are added, for any table version (read). - **v3 row lineage and encryption keys accepted on a v2 table (run).** `add_snapshot` with `with_row_range(0, 10)` succeeds on a v2 table. The `add-snapshot` update keeps `first-row-id: 0` and `added-rows: 10`, but the v2 metadata JSON drops both fields (`SnapshotV2` in `crates/iceberg/src/spec/snapshot.rs` has no row range fields). The REST catalog sends the update list as commit `updates` (`crates/catalog/rest/src/catalog.rs:1457`, read). `add_encryption_key` behaves the same way: it emits an `add-encryption-key` update, while the v2 metadata JSON omits `encryption-keys`, which the spec lists for v3 only. The `encryption.key-id` on a v2 table case is filed separately as #3402. Expected: either the builder rejects v3-only updates on v2 tables, or the update list and the persisted metadata agree. A v3 table should not accept new position delete files. Found during an agent-assisted audit of v3 support (see #2411). The reproduction above was run against main at 050ddfda8. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
