Hi Ayush, Thanks for confirming.
I’ve updated the upgrade notes to explain the deliberate table/view difference and the storage-layout page to describe the overlap consequence. The changes are pushed to https://github.com/apache/polaris/pull/6036 Thanks, Serhii On Fri, Nov 13, 2026 at 6:27 PM Ayush Saxena <[email protected]> wrote: > Hi Serhii, > > Yes, that sounds reasonable. An explicit compatibility exception for > views is what I was after — my concern was the two diverging silently, > not the direction you have chosen. > > One thing to fold into the documentation alongside the overlap > consequence: that tables and views deliberately differ here, so an > operator reading the upgrade notes can see that a legacy table's next > commit is fully validated while a legacy view's is not. Stating the > difference is most of the value of making it a decision. > > Agreed on keeping the logging separate, and the documentation covering > both uses of the flag is enough for me. > > Thanks, > > -Ayush > > On Fri, 13 Nov 2026 at 21:53, Serhii Svynarchuk <[email protected]> > wrote: > > > > Thanks, Alex, for the review and approval. > > > > Ayush, yes, the compatibility behavior is deliberate. The aim is to avoid > > breaking ordinary updates to existing views after an upgrade. > > My proposal is to keep that behavior for views in this PR and leave the > > existing table behavior unchanged. That would be an explicit > compatibility > > exception for views. > > You’re also right that this is ongoing grandfathering, rather than a > > one-time migration exemption. As long as both paths remain unchanged, > > commits enforce allowed locations but skip structured-location and > overlap > > checks. Changing either path triggers full validation. > > Overlap consequences can be documented: once a view’s paths are recorded, > > they can block later creates at overlapping locations, while the existing > > layout remains accepted. > > Logging would be useful for auditing, but I’d prefer to consider that > > separately from this fix. > > Regarding the flag, the name is imperfect, but the documentation now > > explicitly covers its use for both view base locations and view metadata > > paths. > > > > Does keeping this compatibility exception for views, with those > > consequences documented, sound reasonable? > > > > Thanks, > > Serhii > > > > On Fri, Nov 13, 2026 at 6:06 PM Ayush Saxena <[email protected]> wrote: > > > > > Hi Serhii, > > > > > > The credential scoping part is clearly a fix — omitting > > > write.metadata.path from the scope while writing to it can only fail, > > > and FILE storage hiding it explains why it went unnoticed. No > > > objection there. > > > > > > On the compatibility behaviour you asked about: it is more lenient > > > than what tables do today, and I would like to understand whether that > > > is deliberate. A table last committed before 1.3.0 has all its > > > locations validated on its next commit, and that commit fails if they > > > do not satisfy the current rules. The proposal for views is that a > > > commit leaving the base location and metadata path unchanged keeps the > > > existing layout, enforcing allowed locations but skipping the > > > structured-location and overlap checks. So after this change the same > > > legacy situation gives a hard failure on a table and a pass on a view. > > > > > > I can see the argument for either, and not breaking existing views on > > > upgrade is a good reason. What I would rather avoid is the two > > > diverging without that being an explicit decision, since an operator > > > upgrading cannot then reason about existing objects without knowing > > > which kind each one is. Could we settle it on this thread either way — > > > keep the view grandfathering as intended, or plan to align tables with > > > it later — so that it is recorded as a choice rather than read as an > > > inconsistency by whoever next touches this? > > > > > > A related question: the exemption appears to be keyed on the paths > > > staying unchanged rather than on a single migration commit, so a view > > > that never changes its paths is never fully checked. That is > > > defensible as grandfathering, but it does mean a legacy view can keep > > > a layout that a new view could not be created with. > > > > > > On overlap specifically, recording the locations means other objects > > > can see them from then on. So a legacy view whose location overlaps > > > something keeps that overlap, while a table or view created afterwards > > > in the same place is rejected because of it. First writer wins, and > > > the one that gets rejected is the one nobody has touched yet. I think > > > that is unavoidable if we grandfather at all. The storage-layout page > > > already describes the exemption itself, but not this consequence of > > > it, and it is worth adding there: the rejection will not look related > > > to the view. > > > > > > Optional: logging when a non-conforming legacy layout is recorded > > > would make the grandfathered set auditable, so an operator can find > > > out what is exempt without inferring it from failures later. > > > > > > Minor: ALLOW_UNSTRUCTURED_TABLE_LOCATION already covered view base > > > locations, and this change also uses it for view write.metadata.path. > > > Reusing it avoids a second flag, but the name still does not say what > > > it does. > > > > > > Thanks for picking this up. > > > > > > -Ayush > > > > > > On Fri, 13 Nov 2026 at 21:14, Serhii Svynarchuk <[email protected]> > > > wrote: > > > > > > > > Hi everyone, > > > > > > > > I’d like to discuss https://github.com/apache/polaris/pull/6036, > which > > > > addresses custom metadata paths for views and their location > validation. > > > > > > > > Creating or updating a view on S3 could fail with HTTP 403 when using > > > > scoped credentials and write.metadata.path pointed outside the view > base > > > > location. Polaris omitted the custom metadata path when scoping the > > > > temporary credentials used to write view metadata, so S3 rejected the > > > write > > > > because those credentials didn’t allow access to it. FILE storage > didn’t > > > > expose this problem because it doesn’t scope credentials. > > > > > > > > The PR includes the metadata path in the credential scope and > applies the > > > > same location rules used for tables. By default, a new or changed > > > metadata > > > > path must be inside the view location. With > > > > ALLOW_UNSTRUCTURED_TABLE_LOCATION enabled, it can be elsewhere > within the > > > > catalog allowed locations, subject to the configured overlap checks. > > > > > > > > It also records view locations for sibling overlap checks and > metadata > > > > purge, and validates view locations during renames. Purge uses the > > > recorded > > > > locations after checking that the metadata agrees with them. Older > views > > > > without recorded locations keep a metadata-based fallback so they can > > > still > > > > be dropped. > > > > > > > > One point I’d particularly like feedback on is compatibility. > Following > > > > earlier review feedback, updates that leave both the base location > and > > > > metadata path unchanged preserve the existing layout, while still > > > enforcing > > > > the catalog allowed locations. This avoids breaking existing views > after > > > an > > > > upgrade. New views and changed paths get the full validation. > > > > > > > > For context, Iceberg also removed the deprecation of this property: > > > > https://github.com/apache/iceberg/pull/15250 > > > > > > > > > > > > Do you have any objections to this approach, particularly the > > > compatibility > > > > behavior for existing views? > > > > > > > > > > > > Thanks, > > > > Serhii > > > >
