[
https://issues.apache.org/jira/browse/MAPREDUCE-1432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj Das updated MAPREDUCE-1432:
-----------------------------------
Attachment: 1432.2.patch
All tests and test-passed with this patch. The fixes are checks for null for
the TokenStorage/Token objects in the JobInProgress and TaskTracker. Some unit
tests pass them as null and without the checks those tests fail with NPE.
The relevant portion of the diff w.r.t the earlier patch is:
> @@ -377,16 +378,22 @@
> this.startTime = System.currentTimeMillis();
>
> this.localFs = jobtracker.getLocalFileSystem();
> -
> + this.tokenStorage = ts;
97,98c102,105
< + for (Token<? extends TokenIdentifier> token : ts.getAllTokens()) {
< + ugi.addToken(token);
---
> + if (ts != null) {
> + for (Token<? extends TokenIdentifier> token : ts.getAllTokens()) {
> + ugi.addToken(token);
> + }
108a116,124
> @@ -449,8 +456,6 @@
> JobContext.SPECULATIVECAP,0.1f);
> this.slowNodeThreshold = conf.getFloat(
> JobContext.SPECULATIVE_SLOWNODE_THRESHOLD,1.0f);
> - this.tokenStorage = ts;
> -
> }
>
> /**
247c263
< @@ -993,7 +996,17 @@
---
> @@ -993,7 +996,19 @@
257,258c273,276
< + (Token<JobTokenIdentifier>)TokenCache.getJobToken(ts);
< + getJobTokenSecretManager().addTokenForJob(jobId.toString(), jt);
---
> + (Token<JobTokenIdentifier>)TokenCache.getJobToken(ts);
> + if (jt != null) { //could be null in the case of some unit tests
> + getJobTokenSecretManager().addTokenForJob(jobId.toString(), jt);
> + }
> Add the hooks in JobTracker and TaskTracker to load tokens from the token
> cache into the user's UGI
> ---------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1432
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 0.22.0
> Reporter: Devaraj Das
> Assignee: Devaraj Das
> Fix For: 0.22.0
>
> Attachments: 1432.1.patch, 1432.2.patch, 1432.patch
>
>
> Related to HADOOP-6520. Here it is about putting hooks in the
> JobTracker/TaskTracker for loading tokens in the user's UGI. This is required
> when job files are copied from the HDFS on behalf of the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
