Hi Yufei, Thanks for starting this thread.
I'd like to correct the problem statement, though. The question is not about the data consistency in different systems (different endpoints), but about cache correctness. Since the storage endpoint identifies a service, Polaris can reasonably expect that service to produce consistent data for the same object URI. Subsequently, Iceberg data and metadata files are supposed to be immutable by design. Therefore, Polaris can assume that the same Iceberg table file loaded through the same endpoint and same URI produces the same data every time. Therefore, it can be cached. If the endpoint is different, if may still resolve to the same storage system, or it may not. There is some uncertainty, which, IMHO, makes caching metadata invalid in that situation. More specifically, a cached response from one endpoint cannot be used to substitute a response from another endpoint even if the location URI (e.g. s3://something) is the same. Cheers, Dmitri. On Tue, Oct 6, 2026 at 1:42 PM Yufei Gu <[email protected]> wrote: > Hi folks, > > During review of PR #5328 <https://github.com/apache/polaris/pull/5328>, > we > encountered a question: is it valid for a storage endpoint change to make > an existing table’s metadata URI return different content? > > For example, two S3-compatible stores contain > s3://warehouse/orders/metadata/xxx.metadata.json with the same table UUID: > > - Endpoint A’s metadata points to table metadata A. > - Endpoint B’s metadata points to table metadata B. > > Changing the catalog’s endpoint from A to B changes the metadata returned > without updating the table’s metadata pointer. Neither underlying object > was modified. > > Does this violate Iceberg’s metadata immutability model, or is immutability > scoped to each underlying storage object? Are there legitimate migration or > recovery workflows that require this behavior? > > My understanding is that an existing metadata location should continue to > identify the same content after storage configuration changes. Clarifying > this contract would help determine whether metadata caching needs to > account for storage endpoints. > Yufei >
