Hey Dhruv, I think the second approach is a much cleaner solution, though it relies on some nuance for backward compatibility with clients.
A few points to note: 1) The REST spec's required/optional fields are not intended to match exactly what is in the spec. TableMetadata is a great example of this, where most fields are required in the table spec, but are optional in the REST spec. 2) Backward compatibility comes down to whether a client will break when receiving a payload that it doesn't understand. 3) Unfortunately, some of the gates you might expect to be possible (like table version) aren't feasible if you break client-side parsers (clients will fail to parse the JSON before they can determine it's a v4 table). All that said, the only thing I believe we need to do is add a `v2::loadTable` endpoint and require all v4 tables to be loaded through that mechanism. It's important to note that only load table requires a version bump since committing a table still use the normal snapshot update and clients will know from the supported endpoints whether v4 is even supported (via the v2::loadTable endpoint). I love to hear what others think, but I'm strongly in favor of both starting the process of introducing new versioned endpoints and scoping what we need for V4. There are a few open questions, like what other metadata changes will be necessary (e.g. check constraints/default expressions/generated columns/etc)? How do we want to introduce the new endpoint (e.g. allow it to evolve with breaking changes until we vote on v2 endpoint / v4 spec? I think this is a great start though and would love to hear what others think, -Dan On Thu, Aug 13, 2026 at 3:58 PM Dhruv Arya <[email protected]> wrote: > Hi all, > > Kicking off this thread to discuss the initial REST spec changes needed > for Iceberg V4: > 1. root-manifest in Snapshot: We need the returned Snapshot to have a > root-manifest . (oneOf of root-manifest or manifest-list). For V4 tables, > manifest-list will not be set. > 2. Add table-location: LoadTableResult, CommitTableResponse, and > UnregisterTableResult all need to return table-location. When defined, this > will be the absolute path of the table root. (Allows the paths inside the > metadata to be relative making table migration easier). > 3. Make metadata-location in CommitTableResponse optional: This is for > catalog managed tables. > > Some of these changes are not backwards compatible: > `manifest-list` in Snapshot is a required field right now. Some clients > might rely on this field being set in the response. If we allow this to be > empty when `root-manifest` is set, these clients could break. The same > problem applies to `metadata-location`. Furthermore, many endpoints > (unregister, table, etc) transitively depend on Snapshot. If we create a > new SnapshotV2, many endpoints will need a version bump to V2. > > I have a PR that shows what this would roughly look like: (approach 1) > https://github.com/apache/iceberg/pull/17543 . The diff is pretty big and > we end up duplicating many classes and endpoints. > > Another way of solving this would be to make a required response field > optional (manifest-list and metadata-location) without bumping the > model/api versions. This might be a more practical approach assuming that > most clients don't break this way. This PR shows how minimal these changes > are: (approach 2) https://github.com/apache/iceberg/pull/17642 > > I lean toward the in-place approach, assuming few clients actually require > these fields to be present. Does that seem reasonable, or are there > compatibility concerns I'm missing? > > Regards, > Dhruv Arya >
