Hi JB, This looks related to the per-table storage config discussion [1], right?
A name for StorageConfigInfo was already introduced in [3409]. I suppose, we could build on that and the next step would be to adjust the lookup mechanism to consider storage config names. Overall, I think this idea is pretty straight-forward and can likely fully implement the non-credential aspect of the per-table storage config proposal [1]. I tend to faviour managing named storage configs at the catalog level over unnamed config at arbitrary places of the catalog tree. I think the former is simpler from the user's perspective and functionally equivalent to the latter. [1] https://lists.apache.org/thread/boqdzdtmhhk1bncv2xr43sz4nsrhgwro [3409] https://github.com/apache/polaris/pull/3409 Cheers, Dmitri. On Wed, Mar 18, 2026 at 11:19 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi folks, > > I would like to discuss a use case and potentially an enhancement on > Polaris. > > The use case is as follows: > 1. I have one Polaris catalog. > 2. I have a S3 StorageConfigurationInfo (containing my S3 credentials, > allowed locations, region, endpoint). > 3. Optionally, at namespace level, I can have base-location, inheriting the > catalog's storage credentials. > 4. A table level, I can specify a location/base-location, again inheriting > the catalog's storage credentials. > It means that a table can have its data in s3://bucket-a/table1/, and > another table can use s3://bucket-b/table2/, as long as both buckets are > accessible with the same catalog-level credentials and are in the catalog's > allowed locations. > > Now, I would like to have (again in a single catalog), one table on a S3, > another table on another S3. > The credential vending walks up the entity hierarchy > (FileIOUtil.findStorageInfoFromHierarchy) and uses the first > StorageConfigurationInfo it finds, which is always at the catalog level. > > The only viable option today is to create separate catalogs, each with its > own S3 storage configuration pointing to a different bucket. > > I would like to introduce an "identifier" on StorageConfigurationInfo and > having several StorageConfigurationInfos defined at catalog level. > Then, we could add an extra table property to define the > StorageConfigurationInfo identifier to us. > > Thoughts ? > > I would be happy to work on a proposal about that if there is interest. > > Regards > JB >
