[ 
https://issues.apache.org/jira/browse/YARN-2563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136486#comment-14136486
 ] 

Arpit Gupta commented on YARN-2563:
-----------------------------------

Here is the stack trace

{code}
ozie-oozi-W@wc] Launcher ERROR, reason: Main class 
[org.apache.oozie.action.hadoop.MapReduceMain], main() threw exception, 
org.apache.hadoop.security.authentication.client.AuthenticationException: 
GSSException: No valid credentials provided (Mechanism level: Failed to find 
any Kerberos tgt)
2014-09-13 00:00:18,595  WARN MapReduceActionExecutor:546 - USER[hrt_qa] 
GROUP[-] TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Launcher exception: 
org.apache.hadoop.security.authentication.client.AuthenticationException: 
GSSException: No valid credentials provided (Mechanism level: Failed to find 
any Kerberos tgt)
java.io.IOException: 
org.apache.hadoop.security.authentication.client.AuthenticationException: 
GSSException: No valid credentials provided (Mechanism level: Failed to find 
any Kerberos tgt)
        at 
org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.getDelegationToken(TimelineAuthenticator.java:135)
        at 
org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.getDelegationToken(TimelineClientImpl.java:181)
        at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.addTimelineDelegationToken(YarnClientImpl.java:275)
        at 
org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:221)
        at 
org.apache.hadoop.mapred.ResourceMgrDelegate.submitApplication(ResourceMgrDelegate.java:282)
        at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:289)
        at 
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:437)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1294)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1291)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1291)
        at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
        at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at 
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
        at 
org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:108)
        at 
org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:66)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:39)
        at 
org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:226)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: 
org.apache.hadoop.security.authentication.client.AuthenticationException: 
GSSException: No valid credentials provided (Mechanism level: Failed to find 
any Kerberos tgt)
        at 
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:306)
        at 
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:196)
        at 
org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.authenticate(TimelineAuthenticator.java:101)
        at 
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:216)
        at 
org.apache.hadoop.yarn.client.api.impl.TimelineAuthenticator.getDelegationToken(TimelineAuthenticator.java:121)
        ... 36 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed 
to find any Kerberos tgt)
        at 
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
        at 
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
        at 
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
        at 
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
        at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
        at 
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
        at 
org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:285)
        at 
org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:261)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)
        at 
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:261)
        ... 40 more

2014-09-13 00:00:18,617 DEBUG ActionEndXCommand:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Execute command [action.end] 
key [0000001-140912235159138-oozie-oozi-W]
2014-09-13 00:00:18,617 DEBUG ActionEndXCommand:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] STARTED ActionEndXCommand for 
action 0000001-140912235159138-oozie-oozi-W@wc
2014-09-13 00:00:18,623 DEBUG ActionEndXCommand:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] End, name [wc] type 
[map-reduce] status[DONE] external status [FAILED/KILLED] signal value [null]
2014-09-13 00:00:18,654 DEBUG HadoopAccessorService:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Checking if filesystem hdfs is 
supported
2014-09-13 00:00:18,685 DEBUG HadoopAccessorService:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Checking if filesystem hdfs is 
supported
2014-09-13 00:00:18,706  INFO ActionEndXCommand:543 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] ERROR is considered as FAILED 
for SLA
2014-09-13 00:00:18,728 DEBUG SignalXCommand:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Execute command [signal] key 
[0000001-140912235159138-oozie-oozi-W]
2014-09-13 00:00:18,728 DEBUG SignalXCommand:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] STARTED SignalCommand for 
jobid=0000001-140912235159138-oozie-oozi-W, 
actionId=0000001-140912235159138-oozie-oozi-W@wc
2014-09-13 00:00:18,732 DEBUG LiteWorkflowInstance:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Signaling job execution path 
[/] signal value [ERROR]
2014-09-13 00:00:18,732 DEBUG LiteWorkflowInstance:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Exiting node [wc] with 
transition[/#fail]
2014-09-13 00:00:18,732 DEBUG LiteWorkflowInstance:549 - USER[hrt_qa] GROUP[-] 
TOKEN[] APP[wordcount-wf] JOB[0000001-140912235159138-oozie-oozi-W] 
ACTION[0000001-140912235159138-oozie-oozi-W@wc] Signaling job execution path 
[/] signal value [::synch::]
2014-09-13 00:00:18,732 DEBUG LiteWorkflowStore
{code}

> On secure clusters call to timeline server fails with authentication errors 
> when running a job via oozie
> --------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2563
>                 URL: https://issues.apache.org/jira/browse/YARN-2563
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: timelineserver
>    Affects Versions: 2.6.0
>            Reporter: Arpit Gupta
>            Priority: Blocker
>
> During our nightlies on a secure cluster we have seen oozie jobs fail with 
> authentication error to the time line server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to