sarutak opened a new pull request, #59185:
URL: https://github.com/apache/spark/pull/59185
### What changes were proposed in this pull request?
When OIDC credential propagation is enabled
(`spark.security.oidc.enabled=true`) but Spark's RPC channel encryption is not
configured, Spark now logs a warning at startup.
The check is added to `UserCredentialManager.create()`, which runs once on
the driver when the scheduler backend starts. It reuses the same "RPC
encryption is enabled" criteria as `HadoopDelegationTokenManager`'s
direct-credential-provider path:
- `spark.ssl.rpc.enabled=true`, or
- `spark.authenticate=true` together with one of
`spark.network.crypto.enabled` / `spark.authenticate.enableSaslEncryption`.
Unlike the `directCredentialProviders` path (which fails fast via
`require()`), this only logs a warning, per the SPIP's default behavior.
`docs/security.md` is updated to note that the warning is emitted.
### Why are the changes needed?
OIDC-derived service credentials are transmitted to executors over Spark's
RPC channel (via `UpdateUserCredentials` broadcasts and `TaskDescription`
delivery). If RPC encryption is not configured, those credentials travel in
cleartext.
The SPIP (design document) specifies this behavior:
> Enabling credential propagation without RPC encryption logs a warning by
default. A stricter
> mode (`spark.security.credentials.requireEncryption=true`) that fails
application startup unless
> RPC encryption is enabled may be introduced based on community feedback.
The design and the documentation already assumed this warning, but the
implementation was missing.
This change closes that gap.
### Does this PR introduce _any_ user-facing change?
Yes. A warning is now logged at driver startup when OIDC credential
propagation is enabled without RPC encryption configured. There is no
behavioral change beyond the log line; startup is not blocked.
### How was this patch tested?
New unit tests in `UserCredentialManagerSuite`:
- `isRpcEncryptionEnabled` truth-table coverage (nothing configured / SSL /
auth+crypto /
auth+SASL / auth-only-without-encryption).
- `warnIfRpcEncryptionDisabled` emits / does not emit the warning (verified
with `LogAppender`).
- `create()` emits the warning when OIDC is enabled without encryption, and
does not emit it when OIDC is disabled (guards the wiring).
All 42 tests in `UserCredentialManagerSuite` pass. `core/scalastyle` passes.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Kiro CLI / Claude
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
