[jira] [Commented] (YARN-1334) YARN should give more info on errors when running failed distributed shell command
[
https://issues.apache.org/jira/browse/YARN-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14357802#comment-14357802
]
Hadoop QA commented on YARN-1334:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609555/YARN-1334.1.patch
against trunk revision 7a346bc.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6931//console
This message is automatically generated.
YARN should give more info on errors when running failed distributed shell
command
--
Key: YARN-1334
URL: https://issues.apache.org/jira/browse/YARN-1334
Project: Hadoop YARN
Issue Type: Improvement
Components: applications/distributed-shell
Affects Versions: 2.3.0
Reporter: Tassapol Athiapinya
Assignee: Xuan Gong
Attachments: YARN-1334.1.patch
Run incorrect command such as:
/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
-jar distributedshell jar -shell_command ./test1.sh -shell_script ./
would show shell exit code exception with no useful message. It should print
out sysout/syserr of containers/AM of why it is failing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1334) YARN should give more info on errors when running failed distributed shell command
[
https://issues.apache.org/jira/browse/YARN-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13910790#comment-13910790
]
Hadoop QA commented on YARN-1334:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609555/YARN-1334.1.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3169//console
This message is automatically generated.
YARN should give more info on errors when running failed distributed shell
command
--
Key: YARN-1334
URL: https://issues.apache.org/jira/browse/YARN-1334
Project: Hadoop YARN
Issue Type: Improvement
Components: applications/distributed-shell
Affects Versions: 2.3.0
Reporter: Tassapol Athiapinya
Assignee: Xuan Gong
Fix For: 2.4.0
Attachments: YARN-1334.1.patch
Run incorrect command such as:
/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
-jar distributedshell jar -shell_command ./test1.sh -shell_script ./
would show shell exit code exception with no useful message. It should print
out sysout/syserr of containers/AM of why it is failing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1334) YARN should give more info on errors when running failed distributed shell command
[
https://issues.apache.org/jira/browse/YARN-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801304#comment-13801304
]
Xuan Gong commented on YARN-1334:
-
The reason why we can find any details about the container command itself (such
as command that fails to execute) from the diagnostics or output error message
is because
When we launch the container, we are adding this :
{code}
vargs.add(1 + ApplicationConstants.LOG_DIR_EXPANSION_VAR + /stdout);
vargs.add(2 + ApplicationConstants.LOG_DIR_EXPANSION_VAR + /stderr);
{code}
This will redirect the command line output to the files (stdout and stderr).
Also this will cause that there will be nothing output in the screen. And that
works fine.
But, at Shell.java which is used to execute the command using Java, there are
two BufferedReader which is used to get the correct output and error output.
{code}
final BufferedReader errReader = new BufferedReader(new
InputStreamReader(process.getErrorStream()));
BufferedReader inReader = new BufferedReader(new
InputStreamReader(process.getInputStream()));
{code}
But since we re-direct the output to the files, those two BufferedReader will
get nothing. That is why we can not provide the details on the command itself
(such as command that fails to execute) now.
YARN should give more info on errors when running failed distributed shell
command
--
Key: YARN-1334
URL: https://issues.apache.org/jira/browse/YARN-1334
Project: Hadoop YARN
Issue Type: Improvement
Components: applications/distributed-shell
Affects Versions: 2.2.1
Reporter: Tassapol Athiapinya
Assignee: Xuan Gong
Fix For: 2.2.1
Run incorrect command such as:
/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
-jar distributedshell jar -shell_command ./test1.sh -shell_script ./
would show shell exit code exception with no useful message. It should print
out sysout/syserr of containers/AM of why it is failing.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (YARN-1334) YARN should give more info on errors when running failed distributed shell command
[ https://issues.apache.org/jira/browse/YARN-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801319#comment-13801319 ] Tassapol Athiapinya commented on YARN-1334: --- Can you please check if this part is correct? -nodeAddress = getConf().get(YarnConfiguration.NM_ADDRESS)) Does it dynamically retrieves NM address based on container ID? If not, you can safely remove this part. YARN should give more info on errors when running failed distributed shell command -- Key: YARN-1334 URL: https://issues.apache.org/jira/browse/YARN-1334 Project: Hadoop YARN Issue Type: Improvement Components: applications/distributed-shell Affects Versions: 2.2.1 Reporter: Tassapol Athiapinya Assignee: Xuan Gong Fix For: 2.2.1 Attachments: YARN-1334.1.patch Run incorrect command such as: /usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar distributedshell jar -shell_command ./test1.sh -shell_script ./ would show shell exit code exception with no useful message. It should print out sysout/syserr of containers/AM of why it is failing. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1334) YARN should give more info on errors when running failed distributed shell command
[ https://issues.apache.org/jira/browse/YARN-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801322#comment-13801322 ] Xuan Gong commented on YARN-1334: - bq. Does it dynamically retrieves NM address based on container ID? If not, you can safely remove this part. The containerId does not provide any information about NM address. bq. Can you please check if this part is correct? I think this is correct. The configuration which is return by getConf() function in ContainerExecutor is directly from NodeManager. So, I think that will give us the right NM address and port number. YARN should give more info on errors when running failed distributed shell command -- Key: YARN-1334 URL: https://issues.apache.org/jira/browse/YARN-1334 Project: Hadoop YARN Issue Type: Improvement Components: applications/distributed-shell Affects Versions: 2.2.1 Reporter: Tassapol Athiapinya Assignee: Xuan Gong Fix For: 2.2.1 Attachments: YARN-1334.1.patch Run incorrect command such as: /usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client -jar distributedshell jar -shell_command ./test1.sh -shell_script ./ would show shell exit code exception with no useful message. It should print out sysout/syserr of containers/AM of why it is failing. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (YARN-1334) YARN should give more info on errors when running failed distributed shell command
[
https://issues.apache.org/jira/browse/YARN-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13801325#comment-13801325
]
Hadoop QA commented on YARN-1334:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12609555/YARN-1334.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2248//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2248//console
This message is automatically generated.
YARN should give more info on errors when running failed distributed shell
command
--
Key: YARN-1334
URL: https://issues.apache.org/jira/browse/YARN-1334
Project: Hadoop YARN
Issue Type: Improvement
Components: applications/distributed-shell
Affects Versions: 2.2.1
Reporter: Tassapol Athiapinya
Assignee: Xuan Gong
Fix For: 2.2.1
Attachments: YARN-1334.1.patch
Run incorrect command such as:
/usr/bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
-jar distributedshell jar -shell_command ./test1.sh -shell_script ./
would show shell exit code exception with no useful message. It should print
out sysout/syserr of containers/AM of why it is failing.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
