Hi Renjie, Thanks for raising this! For context, here's the iceberg-rust PR <https://github.com/apache/iceberg-rust/pull/1087> where we found the inconsistency. I am also in favor of approach 2 for backwards compatibility reasons. Perhaps, we can include this behavior in "Appendix E: Format version changes" <https://iceberg.apache.org/spec/#version-2>, under the "Reading v1 metadata for v2" section.
Best, Kevin Liu On Mon, Mar 17, 2025 at 7:17 PM Renjie Liu <liurenjie2...@gmail.com> wrote: > Hi: > > We found an inconsistency between java implementation and spec about > partition-spec and schema in v1 table. > > In spec, it says in v1 table partition-spec and schema are required but > deprecated: https://iceberg.apache.org/spec/#table-metadata > > While in java implementation, they are both optional. For schema, it > checks for `schemas` first, and looks up the current table schema by > `schema-id`. Otherwise it looks for a `schema` field. Similar things > happen for `partition spec`. > > We found this problem when implementing iceberg-rust. To resolve this > inconsistency, there are two approaches: > > 1. Modify java implementation to match spec, e.g. force checking `schema` > and `partition spec` field. > 2. Update spec to claim that both `schema` and `partition spec` are > optional for v1 table. > > Personally I'm in favor of approach 2 as it keeps backward compatibility > and seems a more reasonable solution to me. > > Looking forward to hearing from you! >
