Hi everyone,

I’d like to discuss https://github.com/apache/polaris/pull/6036, which
addresses custom metadata paths for views and their location validation.

Creating or updating a view on S3 could fail with HTTP 403 when using
scoped credentials and write.metadata.path pointed outside the view base
location. Polaris omitted the custom metadata path when scoping the
temporary credentials used to write view metadata, so S3 rejected the write
because those credentials didn’t allow access to it. FILE storage didn’t
expose this problem because it doesn’t scope credentials.

The PR includes the metadata path in the credential scope and applies the
same location rules used for tables. By default, a new or changed metadata
path must be inside the view location. With
ALLOW_UNSTRUCTURED_TABLE_LOCATION enabled, it can be elsewhere within the
catalog allowed locations, subject to the configured overlap checks.

It also records view locations for sibling overlap checks and metadata
purge, and validates view locations during renames. Purge uses the recorded
locations after checking that the metadata agrees with them. Older views
without recorded locations keep a metadata-based fallback so they can still
be dropped.

One point I’d particularly like feedback on is compatibility. Following
earlier review feedback, updates that leave both the base location and
metadata path unchanged preserve the existing layout, while still enforcing
the catalog allowed locations. This avoids breaking existing views after an
upgrade. New views and changed paths get the full validation.

For context, Iceberg also removed the deprecation of this property:
https://github.com/apache/iceberg/pull/15250


Do you have any objections to this approach, particularly the compatibility
behavior for existing views?


Thanks,
Serhii

Reply via email to