[jira] [Commented] (YARN-7618) YARN REST APi - can't launch yarn job on Kerberised Cluster
[
https://issues.apache.org/jira/browse/YARN-7618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16415187#comment-16415187
]
Alexandre Linte commented on YARN-7618:
---
No news about that ?
BR.
> YARN REST APi - can't launch yarn job on Kerberised Cluster
> ---
>
> Key: YARN-7618
> URL: https://issues.apache.org/jira/browse/YARN-7618
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 2.7.4
> Environment: Hadoop 2.7.4 - Kerberized cluster
>Reporter: Alexandre Linte
>Priority: Critical
>
> Hello,
> I'm currently trying to launch a yarn job through an Hadoop kerberized
> cluster following documentation (
> https://hadoop.apache.org/docs/r2.7.4/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
> )
> I'm doing these operation with an active kerberos keytab.
> I begin to create my new-application
> {code:title=curl new-app|borderStyle=solid}
> curl --negotiate -u : -XPOST
> http://uabigrm01.node.com:8088/ws/v1/cluster/apps/new-application
> response :
> {"application-id":"application_1507815642943_271826","maximum-resource-capability":{"memory":32768,"vCores":24}}
> {code}
> After that I submit my application :
> {code:title=curl submit|borderStyle=solid}
> curl --negotiate -u : -XPOST -H "Content-Type: application/json" --data
> @"submit.json" http://uabigrm01.node.com:8088/ws/v1/cluster/apps
> {code}
> Content of submit.json file :
> {code:title=submit.json|borderStyle=solid}
> {
> "application-id": "application_1507815642943_271826",
> "application-name": "yarn-api-test-new",
> "queue": "myqueue",
> "am-container-spec": {
> "commands": {
> "command": "{{HADOOP_HOME}}/bin/yarn jar
> /opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.4.jar
> wordcount /user/mwxk0647/WORK/dataset-input
> /user/mwxk0647/WORK/dataset-output-test-yarn"
> },
> "environment": {
>"entry": [{
> "key": "CLASSPATH",
>
> "value":"{{CLASSPATH}}./*{{HADOOP_CONF_DIR}}{{HADOOP_COMMON_HOME}}/share/hadoop/common/*{{HADOOP_COMMON_HOME}}/share/hadoop/common/lib/*{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/*{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/lib/*{{HADOOP_YARN_HOME}}/share/hadoop/yarn/*{{HADOOP_YARN_HOME}}/share/hadoop/yarn/lib/*./log4j.properties"
> }]
> }
> },
> "unmanaged-AM": false,
> "max-app-attempts": 2,
> "resource": {
> "memory": 1024,
> "vCores": 1
> },
> "application-type": "MAPREDUCE",
> "keep-containers-across-application-attempts": false
> }
> {code}
> I can see job on scheduler, he is submit but He failed due to a Kerberos
> authentication error...
> {code:title=tracelogs|borderStyle=solid}
> User: mwxk0647
> Name: yarn-api-test
> Application Type: MAPREDUCE
> Application Tags:
> YarnApplicationState: FAILED
> Queue:myqueue
> FinalStatus Reported by AM: FAILED
> Started: Wed Dec 06 14:45:56 +0100 2017
> Elapsed: 10sec
> Tracking URL: History
> Diagnostics:
> Application application_1507815642943_424552 failed 2 times due to AM
> Container for appattempt_1507815642943_424552_02 exited with exitCode: 255
> For more detailed output, check application tracking
> page:http://uabigrm01.node.com:8188/applicationhistory/app/application_1507815642943_424552Then,
> click on links to logs of each attempt.
> Diagnostics: Exception from container-launch.
> Container id: container_1507815642943_424552_02_01
> Exit code: 255
> Exception message: java.io.IOException: Failed on local exception:
> java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
> [Caused by GSSException: No valid credentials provided (Mechanism level:
> Failed to find any Kerberos tgt)]; Host Details : local host is:
> "uabigdata69.node.com/10.77.64.69"; destination host is:
> "uabigname02.node.com":8020;
> {code}
> So kerberos is OK for submit app, but not for launch job.
> For the moment I make application works making manually my kinit on the
> datanode :
> {code:borderStyle=solid}
> “command”: "echo | kinit mwxk0647 && {{HADOOP_HOME}}/bin/yarn
> jar...
> {code}
> But It's really ugly... And on scheduler, it display a first job which fail
> as before, but it launch the wordcount job, which is very strange..
> How can i make YARN REST API work properly on a Kerberized Environment ?
> Best Regards.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-7618) YARN REST APi - can't launch yarn job on Kerberised Cluster
Alexandre Linte created YARN-7618:
-
Summary: YARN REST APi - can't launch yarn job on Kerberised
Cluster
Key: YARN-7618
URL: https://issues.apache.org/jira/browse/YARN-7618
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.7.4
Environment: Hadoop 2.7.4 - Kerberized cluster
Reporter: Alexandre Linte
Priority: Critical
Hello,
I'm currently trying to launch a yarn job through an Hadoop kerberized cluster
following documentation (
https://hadoop.apache.org/docs/r2.7.4/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html
)
I'm doing these operation with an active kerberos keytab.
I begin to create my new-application
{code:title=curl new-app|borderStyle=solid}
curl --negotiate -u : -XPOST
http://uabigrm01.node.com:8088/ws/v1/cluster/apps/new-application
response :
{"application-id":"application_1507815642943_271826","maximum-resource-capability":{"memory":32768,"vCores":24}}
{code}
After that I submit my application :
{code:title=curl submit|borderStyle=solid}
curl --negotiate -u : -XPOST -H "Content-Type: application/json" --data
@"submit.json" http://uabigrm01.node.com:8088/ws/v1/cluster/apps
{code}
Content of submit.json file :
{code:title=submit.json|borderStyle=solid}
{
"application-id": "application_1507815642943_271826",
"application-name": "yarn-api-test-new",
"queue": "myqueue",
"am-container-spec": {
"commands": {
"command": "{{HADOOP_HOME}}/bin/yarn jar
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.4.jar
wordcount /user/mwxk0647/WORK/dataset-input
/user/mwxk0647/WORK/dataset-output-test-yarn"
},
"environment": {
"entry": [{
"key": "CLASSPATH",
"value":"{{CLASSPATH}}./*{{HADOOP_CONF_DIR}}{{HADOOP_COMMON_HOME}}/share/hadoop/common/*{{HADOOP_COMMON_HOME}}/share/hadoop/common/lib/*{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/*{{HADOOP_HDFS_HOME}}/share/hadoop/hdfs/lib/*{{HADOOP_YARN_HOME}}/share/hadoop/yarn/*{{HADOOP_YARN_HOME}}/share/hadoop/yarn/lib/*./log4j.properties"
}]
}
},
"unmanaged-AM": false,
"max-app-attempts": 2,
"resource": {
"memory": 1024,
"vCores": 1
},
"application-type": "MAPREDUCE",
"keep-containers-across-application-attempts": false
}
{code}
I can see job on scheduler, he is submit but He failed due to a Kerberos
authentication error...
{code:title=tracelogs|borderStyle=solid}
User: mwxk0647
Name: yarn-api-test
Application Type: MAPREDUCE
Application Tags:
YarnApplicationState: FAILED
Queue: myqueue
FinalStatus Reported by AM: FAILED
Started:Wed Dec 06 14:45:56 +0100 2017
Elapsed:10sec
Tracking URL: History
Diagnostics:
Application application_1507815642943_424552 failed 2 times due to AM Container
for appattempt_1507815642943_424552_02 exited with exitCode: 255
For more detailed output, check application tracking
page:http://uabigrm01.node.com:8188/applicationhistory/app/application_1507815642943_424552Then,
click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1507815642943_424552_02_01
Exit code: 255
Exception message: java.io.IOException: Failed on local exception:
java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed
[Caused by GSSException: No valid credentials provided (Mechanism level: Failed
to find any Kerberos tgt)]; Host Details : local host is:
"uabigdata69.node.com/10.77.64.69"; destination host is:
"uabigname02.node.com":8020;
{code}
So kerberos is OK for submit app, but not for launch job.
For the moment I make application works making manually my kinit on the
datanode :
{code:borderStyle=solid}
“command”: "echo | kinit mwxk0647 && {{HADOOP_HOME}}/bin/yarn
jar...
{code}
But It's really ugly... And on scheduler, it display a first job which fail as
before, but it launch the wordcount job, which is very strange..
How can i make YARN REST API work properly on a Kerberized Environment ?
Best Regards.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id (with application having id 9999)
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LINTE updated YARN-3840: Summary: Resource Manager web ui issue when sorting application by id (with application having id ) (was: Resource Manager web ui issue when sorting application by id with id highter than ) Resource Manager web ui issue when sorting application by id (with application having id ) Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Attachments: RMApps.png On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3840) Resource Manager web ui issue when sorting application by id with id highter than 9999
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LINTE updated YARN-3840: Summary: Resource Manager web ui issue when sorting application by id with id highter than (was: Resource Manager web ui bug on main view after application number ) Resource Manager web ui issue when sorting application by id with id highter than -- Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Attachments: RMApps.png On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3840) Resource Manager web ui bug on main view after application number 9999
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14598419#comment-14598419 ] LINTE commented on YARN-3840: - Hi, xgong, Yes with 2.7.0 yarn version devarak.j, Yes i confirm this is an asc/desc sortingissue with application id over . Regards, Resource Manager web ui bug on main view after application number -- Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE Attachments: RMApps.png On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-3840) Resource Manager web ui bug on main view after application number 9999
LINTE created YARN-3840: --- Summary: Resource Manager web ui bug on main view after application number Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3840) Resource Manager web ui bug on main view after application number 9999
[ https://issues.apache.org/jira/browse/YARN-3840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14595896#comment-14595896 ] LINTE commented on YARN-3840: - Hi, There is no Java stackstrace for this bug. I think that the property yarn.resourcemanager.max-completed-applications is in cause (default value is 1), but it doesn't work properly. Maybe yarn.resourcemanager.max-completed-applications is only effective on ResourceManager GUI. Regards, Resource Manager web ui bug on main view after application number -- Key: YARN-3840 URL: https://issues.apache.org/jira/browse/YARN-3840 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.7.0 Environment: Centos 6.6 Java 1.7 Reporter: LINTE On the WEBUI, the global main view page : http://resourcemanager:8088/cluster/apps doesn't display applications over . With command line it works (# yarn application -list). Regards, Alexandre -- This message was sent by Atlassian JIRA (v6.3.4#6332)
