Thanks, Alex, for the review and approval.

Ayush, yes, the compatibility behavior is deliberate. The aim is to avoid
breaking ordinary updates to existing views after an upgrade.
My proposal is to keep that behavior for views in this PR and leave the
existing table behavior unchanged. That would be an explicit compatibility
exception for views.
You’re also right that this is ongoing grandfathering, rather than a
one-time migration exemption. As long as both paths remain unchanged,
commits enforce allowed locations but skip structured-location and overlap
checks. Changing either path triggers full validation.
Overlap consequences can be documented: once a view’s paths are recorded,
they can block later creates at overlapping locations, while the existing
layout remains accepted.
Logging would be useful for auditing, but I’d prefer to consider that
separately from this fix.
Regarding the flag, the name is imperfect, but the documentation now
explicitly covers its use for both view base locations and view metadata
paths.

Does keeping this compatibility exception for views, with those
consequences documented, sound reasonable?

Thanks,
Serhii

On Fri, Nov 13, 2026 at 6:06 PM Ayush Saxena <[email protected]> wrote:

> Hi Serhii,
>
> The credential scoping part is clearly a fix — omitting
> write.metadata.path from the scope while writing to it can only fail,
> and FILE storage hiding it explains why it went unnoticed. No
> objection there.
>
> On the compatibility behaviour you asked about: it is more lenient
> than what tables do today, and I would like to understand whether that
> is deliberate. A table last committed before 1.3.0 has all its
> locations validated on its next commit, and that commit fails if they
> do not satisfy the current rules. The proposal for views is that a
> commit leaving the base location and metadata path unchanged keeps the
> existing layout, enforcing allowed locations but skipping the
> structured-location and overlap checks. So after this change the same
> legacy situation gives a hard failure on a table and a pass on a view.
>
> I can see the argument for either, and not breaking existing views on
> upgrade is a good reason. What I would rather avoid is the two
> diverging without that being an explicit decision, since an operator
> upgrading cannot then reason about existing objects without knowing
> which kind each one is. Could we settle it on this thread either way —
> keep the view grandfathering as intended, or plan to align tables with
> it later — so that it is recorded as a choice rather than read as an
> inconsistency by whoever next touches this?
>
> A related question: the exemption appears to be keyed on the paths
> staying unchanged rather than on a single migration commit, so a view
> that never changes its paths is never fully checked. That is
> defensible as grandfathering, but it does mean a legacy view can keep
> a layout that a new view could not be created with.
>
> On overlap specifically, recording the locations means other objects
> can see them from then on. So a legacy view whose location overlaps
> something keeps that overlap, while a table or view created afterwards
> in the same place is rejected because of it. First writer wins, and
> the one that gets rejected is the one nobody has touched yet. I think
> that is unavoidable if we grandfather at all. The storage-layout page
> already describes the exemption itself, but not this consequence of
> it, and it is worth adding there: the rejection will not look related
> to the view.
>
> Optional: logging when a non-conforming legacy layout is recorded
> would make the grandfathered set auditable, so an operator can find
> out what is exempt without inferring it from failures later.
>
> Minor: ALLOW_UNSTRUCTURED_TABLE_LOCATION already covered view base
> locations, and this change also uses it for view write.metadata.path.
> Reusing it avoids a second flag, but the name still does not say what
> it does.
>
> Thanks for picking this up.
>
> -Ayush
>
> On Fri, 13 Nov 2026 at 21:14, Serhii Svynarchuk <[email protected]>
> wrote:
> >
> > Hi everyone,
> >
> > I’d like to discuss https://github.com/apache/polaris/pull/6036, which
> > addresses custom metadata paths for views and their location validation.
> >
> > Creating or updating a view on S3 could fail with HTTP 403 when using
> > scoped credentials and write.metadata.path pointed outside the view base
> > location. Polaris omitted the custom metadata path when scoping the
> > temporary credentials used to write view metadata, so S3 rejected the
> write
> > because those credentials didn’t allow access to it. FILE storage didn’t
> > expose this problem because it doesn’t scope credentials.
> >
> > The PR includes the metadata path in the credential scope and applies the
> > same location rules used for tables. By default, a new or changed
> metadata
> > path must be inside the view location. With
> > ALLOW_UNSTRUCTURED_TABLE_LOCATION enabled, it can be elsewhere within the
> > catalog allowed locations, subject to the configured overlap checks.
> >
> > It also records view locations for sibling overlap checks and metadata
> > purge, and validates view locations during renames. Purge uses the
> recorded
> > locations after checking that the metadata agrees with them. Older views
> > without recorded locations keep a metadata-based fallback so they can
> still
> > be dropped.
> >
> > One point I’d particularly like feedback on is compatibility. Following
> > earlier review feedback, updates that leave both the base location and
> > metadata path unchanged preserve the existing layout, while still
> enforcing
> > the catalog allowed locations. This avoids breaking existing views after
> an
> > upgrade. New views and changed paths get the full validation.
> >
> > For context, Iceberg also removed the deprecation of this property:
> > https://github.com/apache/iceberg/pull/15250
> >
> >
> > Do you have any objections to this approach, particularly the
> compatibility
> > behavior for existing views?
> >
> >
> > Thanks,
> > Serhii
>

Reply via email to