Hi Michael, > ALLOW_SETTING_S3_ENDPOINTS in particular doesn't have a catalog-level configuration key
The reason for that is precisely that such a key is currently not effective at all... > Shouldn't catalog creators be able to specify some of these overrides by setting properties on the catalog itself? Catalog creators can provide all properties in one request. However, Polaris code that deals with feature flags requires a CatalogEntity, but the entity is not available when ALLOW_SETTING_S3_ENDPOINTS is checked during the catalog creation process. We could probably refactor existing code to make a CatalogEntity earlier, then check flags and config. That would solve the immediate problem. Yet, such a CatalogEntity would be ephemeral in the sense of not having a real entity ID. I am wondering if there's interest in doing the catalog creation in two stages, nonetheless. This addresses the immediate feature flag problem and also allows more advanced flows where feature flags and more mainstream catalog properties could be managed by different principals with potentially different permissions. WDYT? Thanks, Dmitri. On Tue, Oct 14, 2025 at 1:28 PM Michael Collado <[email protected]> wrote: > Shouldn't catalog creators be able to specify some of these overrides by > setting properties on the catalog itself? I see that > ALLOW_SETTING_S3_ENDPOINTS in particular doesn't have a catalog-level > configuration key, but many catalogs do. I can imagine a service-level > configuration for ALLOW_SETTING_S3_ENDPOINTS with a catalog-level key that > can be set by the catalog creator at creation time. Does that solve > the problem? > > Mike > > On Tue, Oct 14, 2025 at 8:44 AM Dmitri Bourlatchkov <[email protected]> > wrote: > > > To clarify my earlier email: Certain call paths require the storage > config > > to be provided at catalog creation time, IIRC. At the same time > processing > > storage config requires access to feature flags. > > > > Admins can indeed manage grants without exposing access to end users. > > However, I think the chicken and egg problem still exists with > > storage configuration even for admins. > > > > Cheers, > > Dmitri. > > > > On Tue, Oct 14, 2025 at 1:48 AM Eric Maynard <[email protected]> > > wrote: > > > > > Could an administrator implement this two-step process by first > creating > > > the catalog and granting themself " CATALOG_MANAGE_CONTENT " before > > doing > > > any other grants? > > > > > > --EM > > > > > > On Mon, Oct 13, 2025 at 10:25 AM Jean-Baptiste Onofré <[email protected] > > > > > wrote: > > > > > > > Hi Dmitri > > > > > > > > That's a good point. > > > > Imho, we should have a two step approach for catalog creation: first > > > > create the "abstract" entity, and then all permission, etc. > > > > > > > > Regards > > > > JB > > > > > > > > On Thu, Sep 25, 2025 at 5:36 PM Dmitri Bourlatchkov < > [email protected]> > > > > wrote: > > > > > > > > > > Hi All, > > > > > > > > > > Our feature flags code supports setting flags per catalog [1]. > > However > > > > when > > > > > dealing with catalog creation, it may be necessary to check those > > flags > > > > too. > > > > > > > > > > This creates a chicken and egg problem where certain flags that > apply > > > to > > > > > catalogs (e.g. ALLOW_SETTING_S3_ENDPOINTS) can only be set per > realm. > > > > > > > > > > Would it make sense to allow a two phase approach to creating > > catalogs > > > > > where > > > > > 1) a catalog object is created as an empty shell (ID + name) > > > > > 2) An admin user adjusts feature flags / permissions > > > > > 3) A regular user sets catalog config properties > > > > > > > > > > Any other thoughts / suggestions on this matter? > > > > > > > > > > [1] > > > > > > > > > > > > > > > https://github.com/apache/polaris/blob/453e9fb19aaad48f8c46ef4ffe3d516df62e4706/polaris-core/src/main/java/org/apache/polaris/core/config/PolarisConfiguration.java#L167 > > > > > > > > > > Thanks, > > > > > Dmitri. > > > > > > > > > >
