Thanks Parth! Anyone have any questions or comments? I would like to start voting on Monday.
Best, Peter On Thu, Jul 9, 2026 at 7:04 PM Parth Chandra <[email protected]> wrote: > SPIP is updated. > > On Wed, Jul 8, 2026 at 11:21 AM Parth Chandra <[email protected]> wrote: > >> Thanks Cheng Pan! >> Updating the SPIP with Steve's suggestions. >> >> >> Parth >> >> On Wed, Jul 8, 2026 at 10:42 AM Cheng Pan <[email protected]> wrote: >> >>> Thanks Peter and Parth for summarize the discussion, now I’m fine with >>> this SPIP given you provide several valid cases that OIDC approach does not >>> offer. >>> >>> For HADOOP-19906, on JDK 25, when kerberos is disabled, all threads see >>> login UGI, thus credential can correctly propagation; when kerberos is >>> enabled, issue happens. Since the goal of this SPIP is extending DT >>> framework to non-kerberos cases, it’s not a blocker. But from the user >>> perspective, a functionality, that works without kerberos, gets broken with >>> kerberos looks weird. >>> >>> Anyway, HADOOP-19906 has landed Hadoop branch-3.5 and will be delivered >>> in Hadoop 3.5.1 in a few months. >>> >>> > Steve's suggestion is a real simplification. We can remove the >>> DirectTokenProvider trait and just change HadoopDelegationTokenManager. >>> >>> +1 for this direction. >>> >>> Thanks, >>> Cheng Pan >>> >>> >>> >>> On Jul 9, 2026, at 01:00, Parth Chandra <[email protected]> wrote: >>> >>> Thanks Peter! You are correct CoarseGrainedExecutorBackend does wrap >>> SparkHadoopUtil.runAsSparkUser. However, as you pointed out, the end result >>> does not change. >>> You are also correct that HADOOP-19906 is an orthogonal issue and must >>> be addressed. >>> >>> Also, Steve's suggestion is a real simplification. We can remove the >>> DirectTokenProvider trait and just change HadoopDelegationTokenManager. We >>> sort of lose the explicit signal that the provider does not use Kerberos, >>> though that is now signalled by the provider indirectly, so we do not lose >>> much. >>> >>> Parth >>> >>> On Wed, Jul 8, 2026 at 8:59 AM Peter Toth <[email protected]> wrote: >>> >>>> Thanks all for the great discussion. >>>> >>>> First, a small correction on the JDK 25 / Subject-propagation point. I >>>> went through the code and it doesn't seem quite right that "on executor >>>> task threads there is no active doAs() scope" because >>>> CoarseGrainedExecutorBackend wraps everything in >>>> SparkHadoopUtil.runAsSparkUser, which does createSparkUser().doAs(...). >>>> If I'm not mistaken what happens is that JEP 486 stops that Subject >>>> from propagating to the RPC/task threads, so getCurrentUser() falls back to >>>> the static login user, and the pushed credentials are read back from there, >>>> which is why the outcome Parth described still holds, just via the >>>> login-user fallback rather than the absence of a doAs. (Parth, please >>>> correct me if I'm misreading the code here.) >>>> >>>> IMO, HADOOP-19906 looks orthogonal to this SPIP rather than a >>>> requirement of it. It's the same pre-existing JDK 25 dependency that the >>>> current Kerberos and Kafka delegation-token paths already need. So I'd >>>> suggest we track it separately and not treat it as a blocker here. >>>> >>>> That leaves the main open question: the overlap with the OIDC >>>> Credential Propagation SPIP. >>>> Parth listed three cases that the OIDC approach can't cover and that >>>> fit naturally into the existing delegation-token framework: >>>> 1. Kafka delegation tokens over SCRAM — they target brokers (not URIs) >>>> and need no UserContext, so they can't be expressed as >>>> CredentialProvider.resolve(UserContext, URI). >>>> 2. The existing S3A/ABFS Hadoop delegation-token bindings — these >>>> already work under Kerberos and just need the activation gates removed; the >>>> OIDC manager doesn't run them. >>>> 3. Proprietary, non-JWT IdP tokens — the OIDC path requires an OIDC JWT. >>>> >>>> Cheng Pan (and anyone else who has the redundancy concern), does this >>>> resolve it for you? My read is that the two are complementary rather than >>>> redundant: >>>> - this SPIP unlocks the existing, provider-agnostic DT mechanism for >>>> any non-Kerberos provider; >>>> - while the OIDC SPIP adds per-user/session identity propagation for >>>> OIDC. >>>> >>>> I'd like to make sure we agree on that framing, and that you're >>>> comfortable with the direction of extending the existing DT management, >>>> before we go further and discuss the implementation options that Steve >>>> mentioned earlier. >>>> >>>> Best, >>>> Peter >>>> >>>> On Mon, Jun 29, 2026 at 11:58 PM Parth Chandra <[email protected]> >>>> wrote: >>>> >>>>> Hi Cheng Pan >>>>> >>>>> >> then we decided to keep them independent >>>>> >>>>> > I see that, but if we decide to accept and implement both SPIPs, >>>>> then we are going to provide two approaches for users that enable cloud >>>>> credentials refresh, this is functionality redundant, and as you know, >>>>> this >>>>> part usually involve private data and 3rd party services dependencies, >>>>> when >>>>> user report issues, they are likely limited to share the related part of >>>>> logs and environment information to provide a minimal reproducible cases, >>>>> this makes diagnosis extremely difficult. Offering two distinct cloud >>>>> credential refresh mechanisms undoubtedly increases system complexity. >>>>> > I would lean towards to the OIDC Credential Propagation approach >>>>> unless it does not cover the functionality (user perspective) provided by >>>>> this SPIP. >>>>> >>>>> There are a few cases covered by this not covered by the OIDC >>>>> credential propagation approach. For instance, - >>>>> 1. Kafka delegation tokens over SCRAM — the existing >>>>> KafkaDelegationTokenProvider is currently blocked by the Kerberos gates. >>>>> It >>>>> targets brokers, not URIs, and doesn't need a UserContext. However, the >>>>> OIDC approach depends on a UserContext (an OIDC JWT with >>>>> principal/issuer/rawToken). It cannot be reimplemented as a >>>>> CredentialProvider.resolve(UserContext, URI). >>>>> 2. Existing S3A/ABFS delegation token bindings (Steve's point in this >>>>> thread) — these are already implemented as HadoopDelegationTokenProvider >>>>> and work today in Kerberos environments. They just need the activation >>>>> gates removed to work without Kerberos. The OIDC SPIP's parallel manager >>>>> does not unblock them. >>>>> 3. There can also be proprietary IdP systems which have non JWT tokens >>>>> (which is what prompted this SPIP in the first place) >>>>> >>>>> >> The DirectProviderPath proposed in this SPIP does not go >>>>> through Subject.doAs() or UserGroupInformation.doAs() and will be >>>>> unaffected. The existing Kerberos path will have to be updated. >>>>> > Sorry, I overlook this reply. I think this is also affected. The JDK >>>>> change breaks the Subject propagation between threads, that means you can >>>>> not get the same Subject (UGI) instance from the task thread as >>>>> the updateTokensTask thread, so you can not access any kind of the >>>>> credential you offered from the task thread. >>>>> >>>>> On executor task threads, there is no active doAs()/callAs() scope. >>>>> getCurrentUser() sees null from Subject.current() (or Subject.getSubject() >>>>> on older JDKs) and falls back to getLoginUser() — which is a static field >>>>> and is not based on Subject propagation. The credentials added via >>>>> addCredentials() on the RPC handler thread are added to this same static >>>>> login user instance so task threads reading from getLoginUser() should see >>>>> them >>>>> However, broadly speaking, we do need HADOOP-19906 >>>>> >>>>> I've updated the SPIP document to include these points >>>>> >>>>> Parth >>>>> >>>> >>>
