[
https://issues.apache.org/jira/browse/MAPREDUCE-4324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291137#comment-13291137
]
Ashutosh Chauhan commented on MAPREDUCE-4324:
---------------------------------------------
As someone working on higher up the stack, I have seen this {{if}} code block
in all the clients. Ideally, {{jobclient}} should do it, freeing apps from this
unnecessary requirement. Thanks, Harsh for picking this up!
> JobClient can perhaps set mapreduce.job.credentials.binary rather than expect
> its presence?
> -------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4324
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4324
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1, mrv2, security
> Affects Versions: 0.22.0, 2.0.0-alpha
> Reporter: Harsh J
> Assignee: Harsh J
>
> HDFS-1007 added in this requirement property
> "mapreduce.job.credentials.binary", that has lead Oozie to add the following
> duplicate snippet to all its Job-launching main classes such as the Pig,
> Hive, MR and Sqoop actions:
> {code}
> if (System.getenv("HADOOP_TOKEN_FILE_LOCATION") != null) {
> jobConf.set("mapreduce.job.credentials.binary",
> System.getenv("HADOOP_TOKEN_FILE_LOCATION"));
> }
> {code}
> Same is required for any client program that launches a job from within a
> task.
> Why can't this simply be set by the JobClient initialization bits itself? If
> no one imagines it causing issues, I'd like to add this snippet somewhere in
> JobSubmitter before it requests NN/JT, as otherwise we'd get…
> {code}
> org.apache.hadoop.ipc.RemoteException: java.io.IOException: Delegation Token
> can be issued only with kerberos or web authentication
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getDelegationToken(FSNamesystem.java:5509)
>
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.getDelegationToken(NameNode.java:536)
>
> at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1434)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1430)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
>
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1428)
> at org.apache.hadoop.ipc.Client.call(Client.java:1107)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
> at $Proxy6.getDelegationToken(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>
> at $Proxy6.getDelegationToken(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.getDelegationToken(DFSClient.java:331)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getDelegationToken(DistributedFileSystem.java:605)
>
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodesInternal(TokenCache.java:115)
>
> at
> org.apache.hadoop.mapreduce.security.TokenCache.obtainTokensForNamenodes(TokenCache.java:79)
>
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:851)
> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:833)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
>
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:833)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:807)
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1242)
> {code}
> … or similar errors when a user submits a job from a task running in a
> secured cluster.
> Let me know your thoughts on this!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
