Hi all,
Seeing such a big engagement is amazing! Having so many people care
about security and user oriented features is absolutely awesome for
the Polaris project!
Truth to be told: security is relative and needs to be considered in a
holistic way. Security can never be 100% perfect (it's also software).
It is totally valid and actually required to uphold strong and strict
security standards, wherever possible and wherever it makes sense.
This is what I fully support!
On the other hand, it doesn't make much sense to lock the door if you
can walk into the house through the open garage.
Full disclosure: I'll deviate into very related aspects and pieces to
cover the "whole picture".
>From the above messages, it feels like there is some confusion about
what Polaris can fully control/enforce and what Polaris can only
"recommend" due to the API and architecture and libraries we and our
users have to use. More on this below.
The term "privilege escalation" in the previous messages caught my attention.
A "privilege escalation" is a situation when a user can gain *more* or
*higher* access privileges (against the same or a different system).
In this AssumeRoleWithWebIdentity case, the user already has the
privileges. Nothing that's been described gives a user *more* or
*higher* access privileges.
Polaris returns (potentially down-scoped) credentials leveraging IAM
policies, but those can already be overridden by users with different
credentials. This allows a (bad?) client, for example, to write data
to arbitrary locations, effectively bypassing the restrictions of
down-scoped credentials to enforce table locations. But even that can
already be worked around using standard configuration options (think:
using write.folder-storage.path or write.object-storage.path).
For a "smoother experience", it's also possible for users to extend
the S3FileIO to automatically retrieve and refresh the object storage
credentials directly via STS/AssumeRoleWithWebIdentity - just override
the file-io configuration to use it. It probably wouldn't even count
as a security issue, because it does not work around the already
granted access privileges.
All these _standard_ configuration options and mechanisms are the root
cause, but it is neither a new issue nor a Polaris issue nor can
Polaris do anything to prevent this from ever happening.
A different previously mentioned aspect that I'd like to highlight:
There is currently no way to gain insights into *who* did *what*
against the object store, because the object storage credentials that
are currently vended/issued by Polaris are _not_ linked to a
user/client identity. AssumeRoleWithWebIdentity seems to enable
getting this very valuable information, which can be necessary for
certain legal/governance requirements.
Looking a bit around in the ecosystem (disclaimer: I haven't looked at
the nitty gritty details), Trino and Hive already support S3 via
AssumeRoleWithWebIdentity.
As already mentioned in the previous messages, "trust" is the key for
the STS/AssumeRoleWithWebIdentity scenario.
The STS service should ideally require a token issued for Polaris on
behalf of the user ("delegation"). That's a legit scenario and a very
valuable feature for Polaris, in addition to the request to support
AssumeRoleWithWebIdentity. (Mentioned in the previous messages.)
The "normal" workflow would be: client -> Polaris -> Polaris gets
down-scoped credentials from STS -> Polaris returns (potentially
down-scoped) credentials to the client.
Again, a client can also call STS directly to get S3 credentials via
AssumeRoleWithWebIdentity, and work with *any* tool directly against
S3.
There is nothing that can prevent this from happening, as long as the
STS service is accessible and STS accepts the JWT ("trust").
Technically and assuming that the user has the necessary access
privileges, all operations, even those that effectively bypass
validations and checks in Polaris, are possible.
This is, again, not a problem with AssumeRoleWithWebIdentity, but a
general and not a Polaris specific problem. That the source of truth
(table/view metadata) is in the object storage contributes to the
situation. Plus: that the client generates the path names and the
contents of all meta information.
Polaris does govern access to the meta information about namespaces
and tables and views. For the latter it is, beside some additional
properties, just the pointer to the table/view metadata. If a user has
the necessary privileges, overwriting table/view metadata is possible.
What I read between the lines in some of the previous messages is an
ask for a "perfect security model". Due to the API and the
architecture we have to work with, it is IMO not possible. It might
help to generally restrict access to the object store to totally
controlled systems in an isolated security enclave, which is a very
different thing.
My proposal is to enable _all_ Polaris users to gain the value add of
the original proposal, and not push our users towards maintaining hard
forks for general-purpose functionality:
1. Continue the AssumeRoleWithWebIdentity proposal, opt-in via a
catalog or realm level configuration, maybe guarded with a "production
ready warning", if people really insist
2. Add OAuth delegation support to Polaris, starting with the
AssumeRoleWithWebIdentity use case
2.a) Polaris tokens: re-issue the token with a specific claim trusted by STS
2.b) Delegation for "proper" IdPs
Yes, 1 without 2 might be a concern.
How does that sound?
Robert
On Sat, Dec 6, 2025 at 1:46 AM Adnan Hemani via dev
<[email protected]> wrote:
>
> Hi all,
>
> I can get behind Yufei's idea - I don't see any security concerns with it.
> In this case, the admin is explicitly electing (at an application level) to
> outsource their AuthZ to an external system and there still remains a
> single source of truth of which principals are allowed to access what. To
> put it in explicit terms, Polaris authorization should be *fully disabled*
> for any catalog that elects this model - and we *must* enforce disabling
> Polaris' own authorization checks to prevent users being misled and/or
> confused by this feature.
>
> But I also agree with Laurent's analysis of where the OIDC/JWT world is
> going today - and I think we should invest in supporting that workflow in a
> similar manner to what Laurent is expressing. That model will allow users
> to still use Polaris authorization *AND* AssumeRoleWithWebIdentity APIs for
> their on-prem (or even cloud) platforms in a secure, non-confusing manner -
> which is the ideal scenario to me.
>
> However for RJ's use case, the question remains, will this on-prem "STS"
> system be able to integrate with the External Policy Decision Point (PDP
> <https://polaris.apache.org/in-dev/unreleased/managing-security/external-pdp/>)
> model?
> I have no background in what system is being used, so it's hard for me to
> tell.
>
> Best,
> Adnan Hemani
>
> On Fri, Dec 5, 2025 at 10:09 AM Yufei Gu <[email protected]> wrote:
>
> > Hi everyone,
> >
> > To add one more perspective here, this is not only a security concern but
> > also a major usability and mental-model concern for Polaris users. Here is
> > an example:
> >
> > - Alice has read + write privileges at the STS/storage layer.
> > - In Polaris, Alice is intentionally granted read-only access to a
> > table.
> >
> > Under the proposed model, because the user JWT is forwarded directly to
> > STS, Alice can simply bypass Polaris and call STS directly. STS then
> > (correctly, from its own viewpoint) grants her write permission, since it
> > has no awareness of Polaris’ table-level authorization.
> >
> > This results in *privilege escalation: *Polaris says “read-only,” but Alice
> > can still perform writes by going around Polaris.
> >
> > Beyond the security implications, this is extremely confusing for admins
> > and users. Polaris presents one authoritative access decision, but the
> > underlying storage enforces another. Users cannot reasonably be expected to
> > understand why they can “write when Polaris says they can’t,”, and
> > why audit logs and enforcement don’t align.
> >
> > *A possible path forward:* one direction we can take, which keeps your use
> > case viable while also preserving Polaris’ governance guarantees, is to
> > introduce an OPA-style authorization module:
> >
> > - Polaris can delegate authZ decisions to a remote system (e.g., OPA,
> > your existing STS-side policy engine, etc).
> > - Credential vending still occurs in Polaris, but only after the
> > delegated AuthZ result has been evaluated.
> >
> > This approach allows deployments like yours to centralize their existing
> > rules in the external system, while also ensuring:
> >
> > - No privilege escalation occurs by bypassing Polaris
> > - Credential vending always honors Polaris’ (or delegated) AuthZ
> > decision
> > - We avoid creating two conflicting interpretations of the same user’s
> > permissions
> >
> > This feels like a direction that could satisfy both RJ’s on-prem scenario
> > and the broader community’s expectations around governance semantics.
> >
> > Happy to continue iterating on how such a module would be wired in, but
> > conceptually it gives us a clean separation: AuthZ may be delegated, but
> > enforcement remains consistent and predictable through Polaris.
> >
> > Yufei
> >
> >
> > On Fri, Dec 5, 2025 at 9:18 AM Dmitri Bourlatchkov <[email protected]>
> > wrote:
> >
> > > Hi All,
> > >
> > > I guess my comment in GH regarding Polaris-owned tokens was not precise
> > > enough. Apologies for the confusion and I'll try to clarify below.
> > >
> > > As far as I understand, in RJ's use case, Polaris is actually expected to
> > > pass the user's auth token from Polaris API to STS. Arguably, it is a
> > very
> > > specific use case and may be applicable only to some very specific
> > on-prem
> > > environments (please correct me if I misinterpret that), but I believe it
> > > is fine to support.
> > >
> > > In that use case Polaris acts as a Resource server from the OAuth2
> > > perspective and no extra auth tokens are necessary.
> > >
> > > Outside of RJ's use case (and PR), I do not really object to supporting
> > > OAuth2 token exchange in Polaris. It could be a valuable feature, but
> > from
> > > my POV it is a totally separate feature (not related to [3170])
> > >
> > > [3170] https://github.com/apache/polaris/pull/3170
> > >
> > > Cheers,
> > > Dmitri.
> > >
> > > On Thu, Dec 4, 2025 at 2:22 PM Arsenault, Reginald P. via dev <
> > > [email protected]> wrote:
> > >
> > > > UNCLASSIFIED / NON CLASSIFIÉ
> > > >
> > > > Hi Laurent, Thanks for your questions!
> > > >
> > > > > I'm a big fan of those token exchange scenarios which avoid to create
> > > > long lived secrets to interact between systems, and rely instead of
> > > setting
> > > > trust mechanisms between them.
> > > >
> > > > We are too, which is why can’t go with the standard
> > > > access-key-id/access-key-secret setup that a regular AWS S3 system
> > uses.
> > > By
> > > > passing along the short lived JWT token of the user to the STS, we can
> > > get
> > > > a new, short-lived, access-key-id/access-key-secret for our on-prem S3
> > > > buckets with every request. Having one generic service account doing
> > all
> > > > the operations on the S3 for all our users is a security risk for us.
> > > >
> > > >
> > > > > My main question is that if the authorization token sent to polaris
> > is
> > > > passed as-is to the STS server, what happens if the client directly
> > sends
> > > > the token to the STS system directly?
> > > >
> > > > For our on-prem S3/STS setup we have no issue with users accessing STS
> > > > System directly. It is normal for users to be able to access the
> > buckets.
> > > > The STS checks the permissions users have on the buckets they’re
> > > > requesting, so there is no way for a user to gain more privileges to
> > > > potentially read/write a bucket they don’t get access to through
> > > polaris,
> > > > by going directly to the STS. If anything if a user going directly to
> > the
> > > > STS they’d get MORE permissions, which is totally fine and completely
> > > > reasonable.
> > > >
> > > >
> > > > > You probably have thought of that scenario already so you may have
> > some
> > > > contingency system in place or maybe it's not a actual issue in your
> > > > deployment situation?
> > > >
> > > > We heavily audit what the users are doing. Polaris will log “this user
> > is
> > > > requesting credentials for this table”, the STS logs “this user is
> > > > requesting credentials for this bucket” and our on-prem S3 logs “this
> > > user
> > > > did <thing> to this bucket.” There is no concern from us over someone
> > > > potentially skirting privileges.
> > > >
> > > >
> > > > > One idea I had to prevent user to bypass Polaris authorization system
> > > > would be for Polaris itself to issue its own JWT tokens, and when
> > > accessing
> > > > storage, Polaris would do an internal exchange where claims from the
> > > > inbound token (received by Polaris) would be copied over to the
> > outbound
> > > > token (sent to STS), after proper validation of course. Is it a
> > scenario
> > > > you have considered by any chance?
> > > >
> > > > From what I gather, Polaris has no intentions on being in the business
> > of
> > > > issuing JWT tokens [1]. Again, for our on-prem S3 system, we have no
> > > issue
> > > > with users going around Polaris to access the STS and S3, this is by
> > > > design. There is no guarantee that this feature will work for AWS/Minio
> > > > [2], it will enable Polaris to support on-prem S3 use cases like [3],
> > > > making it more widely available for everyone.
> > > >
> > > > [1]
> > https://github.com/apache/polaris/pull/3170#issuecomment-3606973902
> > > > [2]
> > https://github.com/apache/polaris/pull/3170#issuecomment-3608346092
> > > > [3] https://github.com/apache/polaris/issues/3038
> > > >
> > > > Thank you for your feedback and please let me know if there’s any more
> > > > questions!
> > > >
> > > > Thanks,
> > > > R.J.
> > > >
> > >
> >
