This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new 2eaed4940cef [SPARK-57538][YARN] Revise a warning message in
`YarnShuffleService`
2eaed4940cef is described below
commit 2eaed4940cefe3f730e3ff2c2ba614a23e6975b1
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Jun 18 12:50:03 2026 -0700
[SPARK-57538][YARN] Revise a warning message in `YarnShuffleService`
In `YarnShuffleService.initializeApplication`, when the application data
fails to parse,
the warning logged the entire `payload`. This PR logs only the application
ID instead.
With `spark.authenticate` enabled, the default code path sends the raw
shuffle secret as
the `payload`, so the warning logged the secret in plaintext to the
NodeManager logs.
No.
Pass the CIs.
Yes. Generated-by: Claude Code (Claude Opus 4.8)
Closes #56599 from dongjoon-hyun/SPARK-57538.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit a0f9088aba673f160c4d7834876aa9f477cba224)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../main/java/org/apache/spark/network/yarn/YarnShuffleService.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
b/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
index e0af3c5ae246..c1ed9f8c13d0 100644
---
a/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
+++
b/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
@@ -436,7 +436,8 @@ public class YarnShuffleService extends AuxiliaryService {
MDC.of(LogKeys.APP_ID$.MODULE$, appId));
}
} catch (IOException ioe) {
- logger.warn("Unable to parse application data for service: " +
payload);
+ logger.warn("Unable to parse the application data for application {}",
+ MDC.of(LogKeys.APP_ID$.MODULE$, appId));
metaInfo = null;
}
if (isAuthenticationEnabled()) {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
