+1 on overall idea Comment however on keeping the same endpoint for STS and S3 - there may be a customer use case (I have seen this before for similar use cases) where the S3 bucket is not in the same region as the STS endpoint that has been exposed to the Polaris server. In that case, you really need two separate endpoints (example below). But I don't think this necessarily blocks this idea from being merged today and then enriched later.
Example: * Polaris is deployed in us-west-1 in a EC2 instance that is within a private subnet, which is given access only to STS' us-west-1 regional endpoint through AWS configurations. * User has S3 data in both us-west-1 and us-east-1. * Under this proposal, user can now access data in us-west-1, but when trying to access data in us-east-1, STS will not respond due to no network routing tables for the us-east-1 regional endpoint. * Important thing to remember is that STS token generated at any regional endpoint can be used at any other regional endpoint. So, in theory, user could still use us-west-1 STS to generate credentials to use for S3 data in us-east-1 and it would've been ok - but this proposal does not allow for this. Best, Adnan Hemani On Tue, Jun 24, 2025 at 9:23 PM Dmitri Bourlatchkov <di...@apache.org> wrote: > Hi All, > > I propose to add an `endpoint` optional parameter to AwsStorageConfigInfo > in PR [1913]. > > The main idea is to support non-AWS S3 implementations for [1530]. > > Existing classes related to supporting S3 Polaris are coded to the AWS SDK, > which supports setting STS endpoints. Therefore, it seems natural to allow > users to optionally define a specific endpoint for their catalogs. > > This change is backward-compatible with existing clients and deployed > catalogs. > > When an endpoint is defined it will be used for both STS and S3 requests > inside Polaris and will be used to populate the "s3.endpoint" properties in > REST Catalog clients. > > Thoughts? > > [1530] https://github.com/apache/polaris/issues/1530 > [1913] https://github.com/apache/polaris/pull/1913 > > Thanks, > Dmitri. >
