Hi Yufei, When locations are not randomized, clients are able (at their discretion) to stage (long) table creation operations in the same location multiple times.
I do not mean to discuss here whether it is wise to do so. It is largely a concern of the Polaris Administrator, I think. My point is simply that Polaris behaviour will change in a way that will prevent certain use cases that were possible before. As for actual credential refresh mechanism, let's discuss that in the more specific thread [1] [1] https://lists.apache.org/thread/by053ptrxrxqx9fm57q64bl3bv41vy46 Cheers, Dmitri. On Tue, Jun 9, 2026 at 2:00 AM Yufei Gu <[email protected]> wrote: > Hi Dmitri, > > Just to make sure I understand, is the concern that for long running staged > create operations (for example CTAS), clients won't be able to refresh > vended credentials because the final randomized location isn't known ahead > of time? > > If so, how is that different from the broader credential refresh problem > discussed in the staged create thread? Couldn't a refresh mechanism be tied > to the assigned location or a refresh token instead of the table > identifier? > > Yufei > > > On Mon, Jun 8, 2026 at 11:37 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > Hi All, > > > > I support using unique table locations for the reasons stated in > > previous emails. > > > > However, I'd like to highlight the fact that currently, with randomized > new > > table locations, it will not be possible for clients executing > long-running > > Staged Create operations (usually CTAS [1]) to refresh vended > credentials. > > > > I believe we ought to advise users about that, especially after making > > randomized locations the default. > > > > [1] https://lists.apache.org/thread/by053ptrxrxqx9fm57q64bl3bv41vy46 > > > > Cheers, > > Dmitri. > > > > On Mon, Jun 8, 2026 at 12:48 PM Yufei Gu <[email protected]> wrote: > > > > > Hi folks, > > > > > > I've been working on a change [1] and would like to get feedback. The > > > change introduces two related capabilities: > > > > > > 1. > > > > > > Tables and views created without an explicit location will use > unique > > > table locations by default. Please note that this is a behavior > > change. > > > 2. > > > > > > Operators can optionally disable client-specified locations in > create > > > table/view requests via a feature flag > > > ALLOW_CLIENT_SPECIFIED_TABLE_LOCATION. The flag is disabled by > default > > > to > > > preserve the current behavior. > > > > > > Existing tables and views are unaffected. Federated catalogs and > register > > > table/view workflows are also unaffected. > > > > > > Using unique table locations by default has a few benefits: > > > > > > - It avoids accidental overlap between tables sharing the same > > namespace > > > or storage location. > > > - It provides stronger isolation between tables, making table > > lifecycle > > > operations such as drop, purge, and maintenance (e.g., removing > orphan > > > files) safer. > > > - It makes the overlapping check logic easier and faster. We don't > > have > > > to worry about location conflicts under a structured table location > > (the > > > default setting). > > > > > > This direction also aligns with the intent of recent Iceberg > discussions > > > [2] around unique table locations and catalog managed storage. > > > > > > Personally, I think enabling unique table locations by default is the > > right > > > choice. For organizations that want stricter control over storage > > > locations, disabling ALLOW_CLIENT_SPECIFIED_TABLE_LOCATION provides a > > > straightforward way to enforce catalog managed placement. > > > > > > I'd be interested in hearing the community's thoughts on both the > overall > > > direction and the proposed defaults. > > > > > > 1. https://github.com/apache/polaris/pull/4606 > > > > > > 2. https://github.com/apache/iceberg/pull/12892 > > > > > > Thanks, > > > > > > Yufei > > > > > >
