[
https://issues.apache.org/jira/browse/YARN-7618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16415187#comment-16415187
]
Alexandre Linte commented on YARN-7618:
---------------------------------------
No news about that ?
BR.
> YARN REST APi - can't launch yarn job on Kerberised Cluster
> -----------------------------------------------------------
>
> Key: YARN-7618
> URL: https://issues.apache.org/jira/browse/YARN-7618
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.4
> Environment: Hadoop 2.7.4 - Kerberized cluster
> Reporter: Alexandre Linte
> Priority: Critical
>
> Hello,
> I'm currently trying to launch a yarn job through an Hadoop kerberized
> cluster following documentation (
> https://hadoop.apache.org/docs/r2.7.4/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
> )
> I'm doing these operation with an active kerberos keytab.
> I begin to create my new-application
> {code:title=curl new-app|borderStyle=solid}
> curl --negotiate -u : -XPOST
> http://uabigrm01.node.com:8088/ws/v1/cluster/apps/new-application
> response :
> {"application-id":"application_1507815642943_271826","maximum-resource-capability":{"memory":32768,"vCores":24}}
> {code}
> After that I submit my application :
> {code:title=curl submit|borderStyle=solid}
> curl --negotiate -u : -XPOST -H "Content-Type: application/json" --data
> @"submit.json" http://uabigrm01.node.com:8088/ws/v1/cluster/apps
> {code}
> Content of submit.json file :
> {code:title=submit.json|borderStyle=solid}
> {
> "application-id": "application_1507815642943_271826",
> "application-name": "yarn-api-test-new",
> "queue": "myqueue",
> "am-container-spec": {
> "commands": {
> "command": "{{HADOOP_HOME}}/bin/yarn jar
> /opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.4.jar
> wordcount /user/mwxk0647/WORK/dataset-input
> /user/mwxk0647/WORK/dataset-output-test-yarn"
> },
> "environment": {
> "entry": [{
> "key": "CLASSPATH",
>
> "value":"{{CLASSPATH}}<CPS>./*<CPS>{{HADOOP_CONF_DIR}}<CPS>{{HADOOP_COMMON_HOME}}/share/hadoop/common/*<CPS>{{HADOOP_COMMON_HOME}}/share/hadoop/common/lib/*<CPS>{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/*<CPS>{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/lib/*<CPS>{{HADOOP_YARN_HOME}}/share/hadoop/yarn/*<CPS>{{HADOOP_YARN_HOME}}/share/hadoop/yarn/lib/*<CPS>./log4j.properties"
> }]
> }
> },
> "unmanaged-AM": false,
> "max-app-attempts": 2,
> "resource": {
> "memory": 1024,
> "vCores": 1
> },
> "application-type": "MAPREDUCE",
> "keep-containers-across-application-attempts": false
> }
> {code}
> I can see job on scheduler, he is submit but He failed due to a Kerberos
> authentication error...
> {code:title=tracelogs|borderStyle=solid}
> User: mwxk0647
> Name: yarn-api-test
> Application Type: MAPREDUCE
> Application Tags:
> YarnApplicationState: FAILED
> Queue: myqueue
> FinalStatus Reported by AM: FAILED
> Started: Wed Dec 06 14:45:56 +0100 2017
> Elapsed: 10sec
> Tracking URL: History
> Diagnostics:
> Application application_1507815642943_424552 failed 2 times due to AM
> Container for appattempt_1507815642943_424552_000002 exited with exitCode: 255
> For more detailed output, check application tracking
> page:http://uabigrm01.node.com:8188/applicationhistory/app/application_1507815642943_424552Then,
> click on links to logs of each attempt.
> Diagnostics: Exception from container-launch.
> Container id: container_1507815642943_424552_02_000001
> Exit code: 255
> Exception message: java.io.IOException: Failed on local exception:
> java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
> [Caused by GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)]; Host Details : local host is:
> "uabigdata69.node.com/10.77.64.69"; destination host is:
> "uabigname02.node.com":8020;
> {code}
> So kerberos is OK for submit app, but not for launch job.
> For the moment I make application works making manually my kinit on the
> datanode :
> {code:borderStyle=solid}
> “command”: "echo <MYPASSWORD> | kinit mwxk0647 && {{HADOOP_HOME}}/bin/yarn
> jar...
> {code}
> But It's really ugly... And on scheduler, it display a first job which fail
> as before, but it launch the wordcount job, which is very strange..
> How can i make YARN REST API work properly on a Kerberized Environment ?
> Best Regards.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
