Hi Eric, Unfortunately, SKIP_CREDENTIAL_SUBCOPING_INDIRECTION=true does not work for non-AWS S3 storage because that setting effectively causes "endpoint" settings to be ignored.
I'll look into other options for now. Cheers, Dmitri. On Thu, Jul 31, 2025 at 3:21 PM Eric Maynard <eric.w.mayn...@gmail.com> wrote: > The existence of SKIP_CREDENTIAL_SUBCOPING_INDIRECTION may be helpful > context for this thread. With that config set, we already support loading > tables without subscoped vended credentials. If there are certain > storage configurations that only work with this config set, that doesn't > seem like it would affect our security posture. > --EM > > On Fri, Aug 1, 2025 at 12:36 AM Robert Stupp <sn...@snazy.de> wrote: > > > Hi, > > > > not sure whether exposing the object storage credentials given to > > Polaris to all clients isn't going to cause a "false impression of > > security" (aka: "our credentials are vended by Polaris, so we're safe" > > - nope...). > > With my "evil user" hat on, I'd try to figure out the configuration > > option (is it realm-specific?) to tell Polaris to yield its "master" > > object storage credentials for a few seconds, just long enough so I > > can gain access to it and have access to all the data. > > > > No doubt, there are S3 implementations (software and appliances) that > > do not support STS, which is admittedly not great. I can imagine that > > at least some appliance vendors and software projects/products will > > get STS. > > > > For the non-STS use cases, I think S3 signing is the way to go. Sure, > > it requires one more request, but we can make those requests fast (aka > > not require any persistence access) as we did in Nessie. With that we > > could still ensure that clients don't have access to everything, > > respecting the object-storage level read/write/list privileges. > > > > Another option is still to configure the object storage credentials at > > the clients. It's not great, but it's still an option. Admins can give > > each client individual credentials to reduce potential risks, being > > able to revoke access for individual clients, and/or audit those. > > > > On Thu, Jul 31, 2025 at 2:51 AM Yufei Gu <flyrain...@gmail.com> wrote: > > > > > > Thanks for raising this, Dmitri! > > > > > > For non-STS use cases, some users may be more comfortable without > > > credential vending. They could configure the storage credentials at the > > > engines side. Can we first confirm that vending raw credentials are > > really > > > users asking for? > > > > > > If that's the case, raw credential vending should be at least optional, > > > which could be guarded by feature flags. > > > > > > And I didn't see much difference between option 1 and option 2. Both > > > provide raw credentials and need rotation. Either way is fine with me. > > > > > > Yufei > > > > > > > > > On Wed, Jul 30, 2025 at 3:24 PM Dmitri Bourlatchkov <di...@apache.org> > > > wrote: > > > > > > > Hi All, > > > > > > > > Recent conversations [1] [2] about non-AWS S3 storage brought up user > > needs > > > > for operating with S3-compatible storage that does not have STS. > > > > > > > > Remote request signing can be used to support those use cases, but it > > is a > > > > considerable development effort to add to Polaris, plus it has > > different > > > > performance characteristics than vended credentials. > > > > > > > > I propose two short-term options to support users of non-STS S3 > > storage. > > > > > > > > 1) Add a configuration option to vend the same credentials that > > Polaris has > > > > to clients. > > > > > > > > While this may (rightly) be considered suboptimal from the security > > > > perspective, this option does give users a choice to operate clients > > > > without explicitly configuring storage credentials for them. Polaris > > > > Servers still control the rotation of those credentials. > > > > > > > > 2) Add secondary plain credentials for vending to clients. Polaris > > itself > > > > will use one key/secret pair. Clients will be issued another > key/secret > > > > pair. Rotation of the client credentials should be possible to > > implement > > > > too. > > > > > > > > WDYT? > > > > > > > > [1] > > https://github.com/apache/polaris/issues/1530#issuecomment-3137374380 > > > > [2] https://github.com/apache/polaris/issues/2207 > > > > > > > > Thanks, > > > > Dmitri. > > > > > > >
