Hi Dmitri and Srinivas, That is exactly the same context. I wanted to provide some additional perspective by focusing specifically on the use case.
I also prefer managing the storage configurations at the catalog level, which was the intent behind the "identifier" idea. I will review the PRs mentioned and follow up. Regards, JB On Thu, Mar 19, 2026 at 12:12 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi JB, > > This looks related to the per-table storage config discussion [1], right? > > A name for StorageConfigInfo was already introduced in [3409]. I suppose, > we could build on that and the next step would be to adjust the lookup > mechanism to consider storage config names. > > Overall, I think this idea is pretty straight-forward and can likely fully > implement the non-credential aspect of the per-table storage config > proposal [1]. > > I tend to faviour managing named storage configs at the catalog level over > unnamed config at arbitrary places of the catalog tree. I think the former > is simpler from the user's perspective and functionally equivalent to the > latter. > > [1] https://lists.apache.org/thread/boqdzdtmhhk1bncv2xr43sz4nsrhgwro > > [3409] https://github.com/apache/polaris/pull/3409 > > Cheers, > Dmitri. > > On Wed, Mar 18, 2026 at 11:19 AM Jean-Baptiste Onofré <[email protected]> > wrote: > > > Hi folks, > > > > I would like to discuss a use case and potentially an enhancement on > > Polaris. > > > > The use case is as follows: > > 1. I have one Polaris catalog. > > 2. I have a S3 StorageConfigurationInfo (containing my S3 credentials, > > allowed locations, region, endpoint). > > 3. Optionally, at namespace level, I can have base-location, inheriting > the > > catalog's storage credentials. > > 4. A table level, I can specify a location/base-location, again > inheriting > > the catalog's storage credentials. > > It means that a table can have its data in s3://bucket-a/table1/, and > > another table can use s3://bucket-b/table2/, as long as both buckets are > > accessible with the same catalog-level credentials and are in the > catalog's > > allowed locations. > > > > Now, I would like to have (again in a single catalog), one table on a S3, > > another table on another S3. > > The credential vending walks up the entity hierarchy > > (FileIOUtil.findStorageInfoFromHierarchy) and uses the first > > StorageConfigurationInfo it finds, which is always at the catalog level. > > > > The only viable option today is to create separate catalogs, each with > its > > own S3 storage configuration pointing to a different bucket. > > > > I would like to introduce an "identifier" on StorageConfigurationInfo and > > having several StorageConfigurationInfos defined at catalog level. > > Then, we could add an extra table property to define the > > StorageConfigurationInfo identifier to us. > > > > Thoughts ? > > > > I would be happy to work on a proposal about that if there is interest. > > > > Regards > > JB > > >
