[
https://issues.apache.org/jira/browse/SPARK-57894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kousuke Saruta reassigned SPARK-57894:
--------------------------------------
Assignee: Kousuke Saruta
> Implement UpdateUserCredentials RPC, backend integration, and task closure
> delivery
> -----------------------------------------------------------------------------------
>
> Key: SPARK-57894
> URL: https://issues.apache.org/jira/browse/SPARK-57894
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 4.3.0
> Reporter: Kousuke Saruta
> Assignee: Kousuke Saruta
> Priority: Major
> Labels: pull-request-available
>
> This sub-task is part of the umbrella
> [SPARK-57703|https://issues.apache.org/jira/browse/SPARK-57703] (OIDC
> Credential Propagation).
> h2. Problem
> There is no RPC to distribute delegated credentials from the driver to
> executors, and no guarantee that credentials are available before task
> execution.
> h2. Goal
> Add credential distribution via RPC broadcast and task closure, ensuring
> executors always have valid credentials when running tasks.
> h2. Scope
> * New message:
> {code:scala}
> case class UpdateUserCredentials(payload: Array[Byte]) extends
> CoarseGrainedClusterMessage
> {code}
> * {{CoarseGrainedSchedulerBackend}}: new method
> {{updateUserCredentials(bytes)}} that stores in {{AtomicReference}} and sends
> to all registered executors. Wired to {{UserCredentialManager}} callback.
> * {{CoarseGrainedExecutorBackend}}: handles {{UpdateUserCredentials}} by
> updating executor credential store.
> * {{SparkEnv}}: add credential store field.
> * {{TaskDescription}}: include current credentials to guarantee availability
> before task execution (avoids race between RPC broadcast and task dispatch).
> * RPC broadcast remains for credential refresh during long-running tasks.
> h2. Acceptance criteria
> * {{UpdateUserCredentials}} is sent to all executors on credential update.
> * Executor credential store is updated on receiving the RPC.
> * {{TaskDescription}} contains current credentials; executor applies them
> before task runs.
> * rawToken is NOT present in any RPC payload or TaskDescription.
> * Unit tests for RPC encoding/decoding, backend handling, and task closure
> credential verification.
> Reference: [SPIP design document Appendix
> B|https://docs.google.com/document/d/1usJKncCPMiyFUg7aIdpZ0HQsklXIHow_sU_6dfFMjN0/edit].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
