[jira] [Updated] (YARN-1480) RM web services getApps() accepts many more filters than ApplicationCLI list command
[ https://issues.apache.org/jira/browse/YARN-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kenji Kikushima updated YARN-1480: -- Attachment: YARN-1480-2.patch Documented new options in addtion. Thanks, Akira. RM web services getApps() accepts many more filters than ApplicationCLI list command -- Key: YARN-1480 URL: https://issues.apache.org/jira/browse/YARN-1480 Project: Hadoop YARN Issue Type: Bug Reporter: Zhijie Shen Assignee: Kenji Kikushima Attachments: YARN-1480-2.patch, YARN-1480.patch Nowadays RM web services getApps() accepts many more filters than ApplicationCLI list command, which only accepts state and type. IMHO, ideally, different interfaces should provide consistent functionality. Is it better to allow more filters in ApplicationCLI? -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1480) RM web services getApps() accepts many more filters than ApplicationCLI list command
[
https://issues.apache.org/jira/browse/YARN-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878410#comment-13878410
]
Hadoop QA commented on YARN-1480:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624300/YARN-1480-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2926//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2926//console
This message is automatically generated.
RM web services getApps() accepts many more filters than ApplicationCLI
list command
--
Key: YARN-1480
URL: https://issues.apache.org/jira/browse/YARN-1480
Project: Hadoop YARN
Issue Type: Bug
Reporter: Zhijie Shen
Assignee: Kenji Kikushima
Attachments: YARN-1480-2.patch, YARN-1480.patch
Nowadays RM web services getApps() accepts many more filters than
ApplicationCLI list command, which only accepts state and type. IMHO,
ideally, different interfaces should provide consistent functionality. Is it
better to allow more filters in ApplicationCLI?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (YARN-1625) mvn apache-rat:check outputs warning message in YARN-321 branch
Shinichi Yamashita created YARN-1625:
Summary: mvn apache-rat:check outputs warning message in YARN-321
branch
Key: YARN-1625
URL: https://issues.apache.org/jira/browse/YARN-1625
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Trivial
Attachments: YARN-1625.patch
When I ran dev-support/test-patch.sh, following message output.
{code}
mvn apache-rat:check -DHadoopPatchProcess /tmp/patchReleaseAuditOutput.txt
21
There appear to be 1 release audit warnings after applying the patch.
{code}
{code}
!?
/home/sinchii/git/YARN-321-test/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/applicationhistory/.keep
Lines that start with ? in the release audit report indicate files that do
not have an Apache license header.
{code}
To avoid release audit warning, it should fix pom.xml.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1625) mvn apache-rat:check outputs warning message in YARN-321 branch
[
https://issues.apache.org/jira/browse/YARN-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shinichi Yamashita updated YARN-1625:
-
Attachment: YARN-1625.patch
I attach a patch file to fix pom.xml
mvn apache-rat:check outputs warning message in YARN-321 branch
---
Key: YARN-1625
URL: https://issues.apache.org/jira/browse/YARN-1625
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Trivial
Attachments: YARN-1625.patch
When I ran dev-support/test-patch.sh, following message output.
{code}
mvn apache-rat:check -DHadoopPatchProcess /tmp/patchReleaseAuditOutput.txt
21
There appear to be 1 release audit warnings after applying the patch.
{code}
{code}
!?
/home/sinchii/git/YARN-321-test/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/applicationhistory/.keep
Lines that start with ? in the release audit report indicate files that
do not have an Apache license header.
{code}
To avoid release audit warning, it should fix pom.xml.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1625) mvn apache-rat:check outputs warning message in YARN-321 branch
[
https://issues.apache.org/jira/browse/YARN-1625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13878467#comment-13878467
]
Hadoop QA commented on YARN-1625:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624306/YARN-1625.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2927//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2927//console
This message is automatically generated.
mvn apache-rat:check outputs warning message in YARN-321 branch
---
Key: YARN-1625
URL: https://issues.apache.org/jira/browse/YARN-1625
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Priority: Trivial
Attachments: YARN-1625.patch
When I ran dev-support/test-patch.sh, following message output.
{code}
mvn apache-rat:check -DHadoopPatchProcess /tmp/patchReleaseAuditOutput.txt
21
There appear to be 1 release audit warnings after applying the patch.
{code}
{code}
!?
/home/sinchii/git/YARN-321-test/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/applicationhistory/.keep
Lines that start with ? in the release audit report indicate files that
do not have an Apache license header.
{code}
To avoid release audit warning, it should fix pom.xml.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (YARN-1626) Allow logs to be pulled for a given containerId without specifying nodeAddress
Hitesh Shah created YARN-1626: - Summary: Allow logs to be pulled for a given containerId without specifying nodeAddress Key: YARN-1626 URL: https://issues.apache.org/jira/browse/YARN-1626 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-431) [Umbrella] Complete/Stabilize YARN application log-handling
[ https://issues.apache.org/jira/browse/YARN-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated YARN-431: - Summary: [Umbrella] Complete/Stabilize YARN application log-handling (was: [Umbrella] Complete/Stabilize YARN appliation log-handling) [Umbrella] Complete/Stabilize YARN application log-handling --- Key: YARN-431 URL: https://issues.apache.org/jira/browse/YARN-431 Project: Hadoop YARN Issue Type: Task Reporter: Vinod Kumar Vavilapalli -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1627) Enhance yarn logs command to specify the filename to be retrieved
Hitesh Shah created YARN-1627: - Summary: Enhance yarn logs command to specify the filename to be retrieved Key: YARN-1627 URL: https://issues.apache.org/jira/browse/YARN-1627 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1626) Enhance yarn logs CLI to allow logs to be pulled for a given containerId without specifying nodeAddress
[ https://issues.apache.org/jira/browse/YARN-1626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated YARN-1626: -- Summary: Enhance yarn logs CLI to allow logs to be pulled for a given containerId without specifying nodeAddress (was: Allow logs to be pulled for a given containerId without specifying nodeAddress) Enhance yarn logs CLI to allow logs to be pulled for a given containerId without specifying nodeAddress --- Key: YARN-1626 URL: https://issues.apache.org/jira/browse/YARN-1626 Project: Hadoop YARN Issue Type: Sub-task Reporter: Hitesh Shah -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1618) Applications transition from NEW to FINAL_SAVING, and try to update non-existing entries in the state-store
[ https://issues.apache.org/jira/browse/YARN-1618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879138#comment-13879138 ] Karthik Kambatla commented on YARN-1618: [~jianhe], [~vinodkv] - can you comment on if NEW - FINAL_SAVING was intentionally added? Applications transition from NEW to FINAL_SAVING, and try to update non-existing entries in the state-store --- Key: YARN-1618 URL: https://issues.apache.org/jira/browse/YARN-1618 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Blocker Attachments: yarn-1618-1.patch YARN-891 augments the RMStateStore to store information on completed applications. In the process, it adds transitions from NEW to FINAL_SAVING. This leads to the RM trying to update entries in the state-store that do not exist. On ZKRMStateStore, this leads to the RM crashing. Previous description: ZKRMStateStore fails to handle updates to znodes that don't exist. For instance, this can happen when an app transitions from NEW to FINAL_SAVING. In these cases, the store should create the missing znode and handle the update. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1573) ZK store should pick a private password for root-node-acls
[ https://issues.apache.org/jira/browse/YARN-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879142#comment-13879142 ] Sandy Ryza commented on YARN-1573: -- +1, LGTM ZK store should pick a private password for root-node-acls -- Key: YARN-1573 URL: https://issues.apache.org/jira/browse/YARN-1573 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1573-1.patch, yarn-1573-2.patch Currently, when HA is enabled, ZK store uses cluster-timestamp as the password for root node ACLs to give the Active RM exclusive access to the store. A more private value like a random number might be better. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1623) Include queue name in RegisterApplicationMasterResponse
[ https://issues.apache.org/jira/browse/YARN-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1623: - Attachment: YARN-1623-1.patch Include queue name in RegisterApplicationMasterResponse --- Key: YARN-1623 URL: https://issues.apache.org/jira/browse/YARN-1623 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1623-1.patch, YARN-1623.patch This provides the YARN change necessary to support MAPREDUCE-5732. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1623) Include queue name in RegisterApplicationMasterResponse
[ https://issues.apache.org/jira/browse/YARN-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879152#comment-13879152 ] Sandy Ryza commented on YARN-1623: -- Updated patch includes a test Include queue name in RegisterApplicationMasterResponse --- Key: YARN-1623 URL: https://issues.apache.org/jira/browse/YARN-1623 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1623-1.patch, YARN-1623.patch This provides the YARN change necessary to support MAPREDUCE-5732. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1622) 'bin/yarn' command doesn't behave like 'hadoop' and etc.
[ https://issues.apache.org/jira/browse/YARN-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohammad Kamrul Islam updated YARN-1622: Attachment: YARN-1622.1.patch Simple one line patch in script. 'bin/yarn' command doesn't behave like 'hadoop' and etc. - Key: YARN-1622 URL: https://issues.apache.org/jira/browse/YARN-1622 Project: Hadoop YARN Issue Type: Bug Components: client Affects Versions: 2.2.0 Reporter: Mohammad Kamrul Islam Assignee: Mohammad Kamrul Islam Attachments: YARN-1622.1.patch There are few issues with 'bin/yarn' and 'etc/hadoop/yarn-env.sh'. They are loosely related but fixes are minor and will go in the same files. Therefore I combined them into one JIRA. Issues are: 1, bin/yarn has a dangling 'fi' in the last line. Thanks to shell for so compliant! 2. YARN_ROOT_LOGGER is defined as INFO, DFRA in yarn-env.sh. That's why 'bin/yarn' command doesn't show (by default) the log messages in client window. But when we used 'bin/hadoop', it shows the log correctly (because HADOOP_ROOT_LOOGER is INFO,console by default). Need to address this non-consistent behavior. 3. For each client command, yarn creates a log file in $YARN_LOG_DIR/yarn.log own by the 'end-user'. In a multi-tenant environment, the second user will not be able to create its own yarn.log in the same place causing the exception (pasted at the end). By default, it should write to $YARN_LOG_DIR/$USER/yarn.log instead. Note: I plan to address only #1 and #2 in this JIRA. If we default the YARN_ROOT_LOGGER consistent with 'bin/hadoop', the issue #3 will not happen. The scope of this JIRA is come close to 'bin/hadoop' behavior. Exception: log4j:ERROR setFile(null,true) call failed. java.io.FileNotFoundException: /export/apps/hadoop/logs/yarn.log (Permission denied) at java.io.FileOutputStream.open(Native Method) at java.io.FileOutputStream.init(FileOutputStream.java:221) at java.io.FileOutputStream.init(FileOutputStream.java:142) at org.apache.log4j.FileAppender.setFile(FileAppender.java:294) at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165) at org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526) at org.apache.log4j.LogManager.clinit(LogManager.java:127) at org.apache.log4j.Logger.getLogger(Logger.java:104) at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289) at org.apache.commons.logging.impl.Log4JLogger.init(Log4JLogger.java:109) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116) at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:914) at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336) at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310) at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685) at org.apache.hadoop.conf.Configuration.clinit(Configuration.java:165) at org.apache.hadoop.util.RunJar.main(RunJar.java:158) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1623) Include queue name in RegisterApplicationMasterResponse
[ https://issues.apache.org/jira/browse/YARN-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879160#comment-13879160 ] Alejandro Abdelnur commented on YARN-1623: -- +1 pending jenkins Include queue name in RegisterApplicationMasterResponse --- Key: YARN-1623 URL: https://issues.apache.org/jira/browse/YARN-1623 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1623-1.patch, YARN-1623.patch This provides the YARN change necessary to support MAPREDUCE-5732. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1607) TestRM expects the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1607: - Attachment: YARN-1607-2.patch TestRM expects the capacity scheduler - Key: YARN-1607 URL: https://issues.apache.org/jira/browse/YARN-1607 Project: Hadoop YARN Issue Type: Bug Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1607-1.patch, YARN-1607-2.patch, YARN-1607.patch We should either explicitly set the Capacity Scheduler or make it scheduler-agnostic -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1607) TestRM expects the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879167#comment-13879167 ] Karthik Kambatla commented on YARN-1607: Thanks Sandy. LGTM. +1, pending Jenkins. TestRM expects the capacity scheduler - Key: YARN-1607 URL: https://issues.apache.org/jira/browse/YARN-1607 Project: Hadoop YARN Issue Type: Bug Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1607-1.patch, YARN-1607-2.patch, YARN-1607.patch We should either explicitly set the Capacity Scheduler or make it scheduler-agnostic -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1622) 'bin/yarn' command doesn't behave like 'hadoop' and etc.
[
https://issues.apache.org/jira/browse/YARN-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879185#comment-13879185
]
Hadoop QA commented on YARN-1622:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624423/YARN-1622.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2929//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2929//console
This message is automatically generated.
'bin/yarn' command doesn't behave like 'hadoop' and etc.
-
Key: YARN-1622
URL: https://issues.apache.org/jira/browse/YARN-1622
Project: Hadoop YARN
Issue Type: Bug
Components: client
Affects Versions: 2.2.0
Reporter: Mohammad Kamrul Islam
Assignee: Mohammad Kamrul Islam
Attachments: YARN-1622.1.patch
There are few issues with 'bin/yarn' and 'etc/hadoop/yarn-env.sh'. They are
loosely related but fixes are minor and will go in the same files. Therefore
I combined them into one JIRA.
Issues are:
1, bin/yarn has a dangling 'fi' in the last line. Thanks to shell for so
compliant!
2. YARN_ROOT_LOGGER is defined as INFO, DFRA in yarn-env.sh. That's why
'bin/yarn' command doesn't show (by default) the log messages in client
window. But when we used 'bin/hadoop', it shows the log correctly (because
HADOOP_ROOT_LOOGER is INFO,console by default). Need to address this
non-consistent behavior.
3. For each client command, yarn creates a log file in $YARN_LOG_DIR/yarn.log
own by the 'end-user'. In a multi-tenant environment, the second user will
not be able to create its own yarn.log in the same place causing the
exception (pasted at the end). By default, it should write to
$YARN_LOG_DIR/$USER/yarn.log instead.
Note:
I plan to address only #1 and #2 in this JIRA. If we default the
YARN_ROOT_LOGGER consistent with 'bin/hadoop', the issue #3 will not happen.
The scope of this JIRA is come close to 'bin/hadoop' behavior.
Exception:
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /export/apps/hadoop/logs/yarn.log (Permission
denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.init(FileOutputStream.java:221)
at java.io.FileOutputStream.init(FileOutputStream.java:142)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at
org.apache.log4j.DailyRollingFileAppender.activateOptions(DailyRollingFileAppender.java:223)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
at
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
at
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
at
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
at
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)
at
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
at
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
at org.apache.log4j.LogManager.clinit(LogManager.java:127)
at org.apache.log4j.Logger.getLogger(Logger.java:104)
at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
at org.apache.commons.logging.impl.Log4JLogger.init(Log4JLogger.java:109)
at
[jira] [Commented] (YARN-1607) TestRM expects the capacity scheduler
[
https://issues.apache.org/jira/browse/YARN-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879203#comment-13879203
]
Hadoop QA commented on YARN-1607:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624425/YARN-1607-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2928//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2928//console
This message is automatically generated.
TestRM expects the capacity scheduler
-
Key: YARN-1607
URL: https://issues.apache.org/jira/browse/YARN-1607
Project: Hadoop YARN
Issue Type: Bug
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: YARN-1607-1.patch, YARN-1607-2.patch, YARN-1607.patch
We should either explicitly set the Capacity Scheduler or make it
scheduler-agnostic
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1623) Include queue name in RegisterApplicationMasterResponse
[
https://issues.apache.org/jira/browse/YARN-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879208#comment-13879208
]
Hadoop QA commented on YARN-1623:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624422/YARN-1623-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2930//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2930//console
This message is automatically generated.
Include queue name in RegisterApplicationMasterResponse
---
Key: YARN-1623
URL: https://issues.apache.org/jira/browse/YARN-1623
Project: Hadoop YARN
Issue Type: Improvement
Components: scheduler
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: YARN-1623-1.patch, YARN-1623.patch
This provides the YARN change necessary to support MAPREDUCE-5732.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1623) Include queue name in RegisterApplicationMasterResponse
[ https://issues.apache.org/jira/browse/YARN-1623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879305#comment-13879305 ] Hudson commented on YARN-1623: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5035 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5035/]) YARN-1623. Include queue name in RegisterApplicationMasterResponse (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1560545) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/RegisterApplicationMasterResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/RegisterApplicationMasterResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/applicationmasterservice/TestApplicationMasterService.java Include queue name in RegisterApplicationMasterResponse --- Key: YARN-1623 URL: https://issues.apache.org/jira/browse/YARN-1623 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 2.4.0 Attachments: YARN-1623-1.patch, YARN-1623.patch This provides the YARN change necessary to support MAPREDUCE-5732. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1607) TestRM expects the capacity scheduler
[ https://issues.apache.org/jira/browse/YARN-1607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879302#comment-13879302 ] Hudson commented on YARN-1607: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5035 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5035/]) YARN-1607. TestRM relies on the scheduler assigning multiple containers in a single node update (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1560533) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRM.java TestRM expects the capacity scheduler - Key: YARN-1607 URL: https://issues.apache.org/jira/browse/YARN-1607 Project: Hadoop YARN Issue Type: Bug Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 2.4.0 Attachments: YARN-1607-1.patch, YARN-1607-2.patch, YARN-1607.patch We should either explicitly set the Capacity Scheduler or make it scheduler-agnostic -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1573) ZK store should use a private password for root-node-acls
[ https://issues.apache.org/jira/browse/YARN-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated YARN-1573: --- Summary: ZK store should use a private password for root-node-acls (was: ZK store should pick a private password for root-node-acls) ZK store should use a private password for root-node-acls - Key: YARN-1573 URL: https://issues.apache.org/jira/browse/YARN-1573 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1573-1.patch, yarn-1573-2.patch Currently, when HA is enabled, ZK store uses cluster-timestamp as the password for root node ACLs to give the Active RM exclusive access to the store. A more private value like a random number might be better. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1525) Web UI should redirect to active RM when HA is enabled.
[ https://issues.apache.org/jira/browse/YARN-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13879488#comment-13879488 ] Karthik Kambatla commented on YARN-1525: Just skimmed through the patch. Can we abstract out the code to find the Active RM Id as a helper method in HAUtil? Web UI should redirect to active RM when HA is enabled. --- Key: YARN-1525 URL: https://issues.apache.org/jira/browse/YARN-1525 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Cindy Li Attachments: YARN1525.patch.v1, YARN1525.patch.v2 When failover happens, web UI should redirect to the current active rm. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
