This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new c31e09dadc45 [SPARK-57538][YARN] Revise a warning message in
`YarnShuffleService`
c31e09dadc45 is described below
commit c31e09dadc45a547af766581ef89eab718ab6b22
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Thu Jun 18 12:50:03 2026 -0700
[SPARK-57538][YARN] Revise a warning message in `YarnShuffleService`
### What changes were proposed in this pull request?
In `YarnShuffleService.initializeApplication`, when the application data
fails to parse,
the warning logged the entire `payload`. This PR logs only the application
ID instead.
### Why are the changes needed?
With `spark.authenticate` enabled, the default code path sends the raw
shuffle secret as
the `payload`, so the warning logged the secret in plaintext to the
NodeManager logs.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code (Claude Opus 4.8)
Closes #56599 from dongjoon-hyun/SPARK-57538.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit a0f9088aba673f160c4d7834876aa9f477cba224)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../main/java/org/apache/spark/network/yarn/YarnShuffleService.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
b/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
index b5718946252e..ab23585170b4 100644
---
a/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
+++
b/common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java
@@ -433,7 +433,8 @@ public class YarnShuffleService extends AuxiliaryService {
MDC.of(LogKeys.APP_ID, appId));
}
} catch (IOException ioe) {
- logger.warn("Unable to parse application data for service: " +
payload);
+ logger.warn("Unable to parse the application data for application {}",
+ MDC.of(LogKeys.APP_ID, appId));
metaInfo = null;
}
if (isAuthenticationEnabled()) {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
