ayushtkn commented on code in PR #512:
URL: https://github.com/apache/tez/pull/512#discussion_r3472556742
##########
tez-dag/src/main/java/org/apache/tez/dag/app/DAGAppMaster.java:
##########
@@ -528,6 +529,12 @@ protected void serviceInit(final Configuration conf)
throws Exception {
jobTokenSecretManager.addTokenForJob(
appAttemptID.getApplicationId().toString(), sessionToken);
+ // Create a JobTokenSecretManager initialized with the session token's
password
+ // for shuffle delete requests. The shuffle handler verifies requests
using the
+ // token password, so the MAC must be initialized with it (as tasks do).
+ shuffleJobTokenSecretManager = new JobTokenSecretManager(
Review Comment:
`JobTokenSecretManager` has two different key paths:
1. `computeHash(byte[] msg)` (instance method) — uses mac.doFinal(msg)
with the MAC master key (random at construction)
2. `retrieveTokenSecret(String jobId)` — returns the per-job key from the
`currentJobTokens` map (populated by `addTokenForJob`)
The client-side `AsyncHttpConnection.computeEncHash()` calls:
`encHash` = `SecureShuffleUtils.hashFromString(msgToEncode,
jobTokenSecretMgr)`;
// → `mgr.computeHash(msg)` ← uses MAC master key, NOT the per-job token
from the map
The server-side verifyRequest calls:
`SecretKey tokenSecret = getSecretManager().retrieveTokenSecret(appid); `
// ← uses the map
`SecureShuffleUtils.verifyReply(urlHashStr, enc_str, tokenSecret); `
So `addTokenForJob` populates the `map` (server path), but the client hash
computation uses the MAC field (instance method). They're completely separate
code paths. The MAC must be initialized
with the token password for the client-side hash to match what the server
verifies.
It was leading to test failure here:
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-512/3/testReport/org.apache.tez.auxservices/TestShuffleHandlerJobs/testOrderedWordCount/
Did I decode it wrong?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
