[jira] [Updated] (YARN-1504) RM changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1504: - Attachment: YARN-1504.patch RM changes for moving apps between queues - Key: YARN-1504 URL: https://issues.apache.org/jira/browse/YARN-1504 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1504.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1573) ZK store should pick a private password for root-node-acls
[ https://issues.apache.org/jira/browse/YARN-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874602#comment-13874602 ] Sandy Ryza commented on YARN-1573: -- Do we ever want someone else (e.g. an administrator) to be able to access the store? Also, not that we need to do the same, but how does HDFS handle this? ZK store should pick a private password for root-node-acls -- Key: YARN-1573 URL: https://issues.apache.org/jira/browse/YARN-1573 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1573-1.patch Currently, when HA is enabled, ZK store uses cluster-timestamp as the password for root node ACLs to give the Active RM exclusive access to the store. A more private value like a random number might be better. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1496) Protocol additions to allow moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874604#comment-13874604 ] Hudson commented on YARN-1496: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5016 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5016/]) YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1559050) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java Protocol additions to allow moving apps between queues -- Key: YARN-1496 URL: https://issues.apache.org/jira/browse/YARN-1496 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1496-1.patch, YARN-1496-2.patch, YARN-1496-3.patch, YARN-1496-4.patch, YARN-1496-5.patch, YARN-1496-6.patch, YARN-1496.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1498) Common scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1498: - Attachment: YARN-1498.patch Common scheduler changes for moving apps between queues --- Key: YARN-1498 URL: https://issues.apache.org/jira/browse/YARN-1498 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1498-1.patch, YARN-1498.patch, YARN-1498.patch This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1498) Common scheduler changes for moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated YARN-1498: - Description: This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues. In particular, it makes sure that QueueMetrics are properly updated when an app changes queue. (was: This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues.) Common scheduler changes for moving apps between queues --- Key: YARN-1498 URL: https://issues.apache.org/jira/browse/YARN-1498 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.2.0 Reporter: Sandy Ryza Assignee: Sandy Ryza Attachments: YARN-1498-1.patch, YARN-1498.patch, YARN-1498.patch This JIRA is to track changes that aren't in particular schedulers but that help them support moving apps between queues. In particular, it makes sure that QueueMetrics are properly updated when an app changes queue. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1504) RM changes for moving apps between queues
[
https://issues.apache.org/jira/browse/YARN-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874648#comment-13874648
]
Hadoop QA commented on YARN-1504:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623616/YARN-1504.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-tools/hadoop-sls
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2906//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2906//console
This message is automatically generated.
RM changes for moving apps between queues
-
Key: YARN-1504
URL: https://issues.apache.org/jira/browse/YARN-1504
Project: Hadoop YARN
Issue Type: Sub-task
Components: resourcemanager
Affects Versions: 2.2.0
Reporter: Sandy Ryza
Assignee: Sandy Ryza
Attachments: YARN-1504.patch
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1606) Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml
[ https://issues.apache.org/jira/browse/YARN-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874673#comment-13874673 ] Hudson commented on YARN-1606: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #455 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/455/]) YARN-1606. Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml (kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558882) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml --- Key: YARN-1606 URL: https://issues.apache.org/jira/browse/YARN-1606 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Minor Labels: conf, newbie Attachments: yarn-1606-1.patch Looks like YARN-1029 changed at one location, but not the other. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1608) LinuxContainerExecutor has a few DEBUG messages at INFO level
[ https://issues.apache.org/jira/browse/YARN-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874670#comment-13874670 ] Hudson commented on YARN-1608: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #455 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/455/]) YARN-1608. LinuxContainerExecutor has a few DEBUG messages at INFO level (kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558874) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java LinuxContainerExecutor has a few DEBUG messages at INFO level - Key: YARN-1608 URL: https://issues.apache.org/jira/browse/YARN-1608 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Trivial Labels: log Fix For: 2.4.0 Attachments: yarn-1608-1.patch LCE has a few INFO level log messages meant to be at debug level. In fact, they are logged both at INFO and DEBUG. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1496) Protocol additions to allow moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874676#comment-13874676 ] Hudson commented on YARN-1496: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #455 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/455/]) YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1559050) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java Protocol additions to allow moving apps between queues -- Key: YARN-1496 URL: https://issues.apache.org/jira/browse/YARN-1496 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1496-1.patch, YARN-1496-2.patch, YARN-1496-3.patch, YARN-1496-4.patch, YARN-1496-5.patch, YARN-1496-6.patch, YARN-1496.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1496) Protocol additions to allow moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874773#comment-13874773 ] Hudson commented on YARN-1496: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1647 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1647/]) YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1559050) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java Protocol additions to allow moving apps between queues -- Key: YARN-1496 URL: https://issues.apache.org/jira/browse/YARN-1496 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1496-1.patch, YARN-1496-2.patch, YARN-1496-3.patch, YARN-1496-4.patch, YARN-1496-5.patch, YARN-1496-6.patch, YARN-1496.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1608) LinuxContainerExecutor has a few DEBUG messages at INFO level
[ https://issues.apache.org/jira/browse/YARN-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874767#comment-13874767 ] Hudson commented on YARN-1608: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1647 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1647/]) YARN-1608. LinuxContainerExecutor has a few DEBUG messages at INFO level (kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558874) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java LinuxContainerExecutor has a few DEBUG messages at INFO level - Key: YARN-1608 URL: https://issues.apache.org/jira/browse/YARN-1608 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Trivial Labels: log Fix For: 2.4.0 Attachments: yarn-1608-1.patch LCE has a few INFO level log messages meant to be at debug level. In fact, they are logged both at INFO and DEBUG. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1608) LinuxContainerExecutor has a few DEBUG messages at INFO level
[ https://issues.apache.org/jira/browse/YARN-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874778#comment-13874778 ] Hudson commented on YARN-1608: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1672 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1672/]) YARN-1608. LinuxContainerExecutor has a few DEBUG messages at INFO level (kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558874) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java LinuxContainerExecutor has a few DEBUG messages at INFO level - Key: YARN-1608 URL: https://issues.apache.org/jira/browse/YARN-1608 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.2.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Trivial Labels: log Fix For: 2.4.0 Attachments: yarn-1608-1.patch LCE has a few INFO level log messages meant to be at debug level. In fact, they are logged both at INFO and DEBUG. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1606) Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml
[ https://issues.apache.org/jira/browse/YARN-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874781#comment-13874781 ] Hudson commented on YARN-1606: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1672 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1672/]) YARN-1606. Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml (kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558882) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml --- Key: YARN-1606 URL: https://issues.apache.org/jira/browse/YARN-1606 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Minor Labels: conf, newbie Attachments: yarn-1606-1.patch Looks like YARN-1029 changed at one location, but not the other. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1496) Protocol additions to allow moving apps between queues
[ https://issues.apache.org/jira/browse/YARN-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874784#comment-13874784 ] Hudson commented on YARN-1496: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1672 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1672/]) YARN-1496. Protocol additions to allow moving apps between queues (Sandy Ryza) (sandy: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1559050) * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestClientRedirect.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationClientProtocol.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesRequest.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/MoveApplicationAcrossQueuesResponse.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/applicationclient_protocol.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/proto/yarn_service_protos.proto * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/client/ApplicationClientProtocolPBClientImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/impl/pb/service/ApplicationClientProtocolPBServiceImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesRequestPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/protocolrecords/impl/pb/MoveApplicationAcrossQueuesResponsePBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java Protocol additions to allow moving apps between queues -- Key: YARN-1496 URL: https://issues.apache.org/jira/browse/YARN-1496 Project: Hadoop YARN Issue Type: Sub-task Components: scheduler Reporter: Sandy Ryza Assignee: Sandy Ryza Fix For: 3.0.0 Attachments: YARN-1496-1.patch, YARN-1496-2.patch, YARN-1496-3.patch, YARN-1496-4.patch, YARN-1496-5.patch, YARN-1496-6.patch, YARN-1496.patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1606) Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml
[ https://issues.apache.org/jira/browse/YARN-1606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874770#comment-13874770 ] Hudson commented on YARN-1606: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1647 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1647/]) YARN-1606. Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml (kasha) (kasha: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1558882) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml Fix the default value of yarn.resourcemanager.zk-timeout-ms in yarn-default.xml --- Key: YARN-1606 URL: https://issues.apache.org/jira/browse/YARN-1606 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Priority: Minor Labels: conf, newbie Attachments: yarn-1606-1.patch Looks like YARN-1029 changed at one location, but not the other. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1592) CapacityScheduler tries to reserve more than a node's total memory on branch-0.23
[ https://issues.apache.org/jira/browse/YARN-1592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-1592: - Summary: CapacityScheduler tries to reserve more than a node's total memory on branch-0.23 (was: backport YARN-957 to branch-0.23) +1, will commit this shortly. CapacityScheduler tries to reserve more than a node's total memory on branch-0.23 - Key: YARN-1592 URL: https://issues.apache.org/jira/browse/YARN-1592 Project: Hadoop YARN Issue Type: Bug Components: capacityscheduler Affects Versions: 0.23.10 Reporter: Thomas Graves Assignee: Thomas Graves Attachments: YARN-1592.patch see YARN-957 for description. branch 0.23 has the same bug. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-851) Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently.
[ https://issues.apache.org/jira/browse/YARN-851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13874973#comment-13874973 ] Karthik Kambatla commented on YARN-851: --- Correct me if I am wrong - while grepping for this JIRA in the different branches, looks like this never made it to branch-2.1* or branch-2.2*. Given this is an incompatible change, should we remove this from branch-2 as well? Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently. --- Key: YARN-851 URL: https://issues.apache.org/jira/browse/YARN-851 Project: Hadoop YARN Issue Type: Sub-task Reporter: Omkar Vinit Joshi Assignee: Omkar Vinit Joshi Fix For: 2.1.0-beta Attachments: YARN-851-20130618.patch, YARN-851-20130619.patch, YARN-851-20130619.patch, YARN-851-20130620.1.patch, YARN-851-20130620.2.patch, YARN-851-20130620.patch It is a follow up ticket for YARN-694. Changing the way NMTokens are shared. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1600) RM does not startup when security is enabled without spnego configured
[ https://issues.apache.org/jira/browse/YARN-1600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875005#comment-13875005 ] Kihwal Lee commented on YARN-1600: -- My understanding of the Hadoop web security requirement is as follows. - SPNEGO should be used, if a REST API needs to be secured. - User-facing web pages should be accessible through a custom auth filter without requiring SPNEGO. I assume ResourceManager currently does not need to secure its REST APIs. Unless this assumption is wrong, SPNEGO should not be required in RM. The early patch on YARN-1463 might be the easiest way to stay this way. I am for fixing it ASAP this way. We will however extend the REST API in the future and that will likely require auth. So we will eventually face the SPNEGO required in RM if security is on situation. Namenode is already like this. It is okay or even preferable to make the server to fail to start if SPNEGO is not setup in this case, but still web UI accesses should not require SPNEGO-based auth. RM does not startup when security is enabled without spnego configured -- Key: YARN-1600 URL: https://issues.apache.org/jira/browse/YARN-1600 Project: Hadoop YARN Issue Type: Bug Components: resourcemanager Affects Versions: 2.4.0 Reporter: Jason Lowe Priority: Blocker We have a custom auth filter in front of our various UI pages that handles user authentication. However currently the RM assumes that if security is enabled then the user must have configured spnego as well for the RM web pages which is not true in our case. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-851) Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently.
[ https://issues.apache.org/jira/browse/YARN-851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875040#comment-13875040 ] Karthik Kambatla commented on YARN-851: --- Sorry for the false alarm. Looks like it did make it - commit message was misspelt. Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently. --- Key: YARN-851 URL: https://issues.apache.org/jira/browse/YARN-851 Project: Hadoop YARN Issue Type: Sub-task Reporter: Omkar Vinit Joshi Assignee: Omkar Vinit Joshi Fix For: 2.1.0-beta Attachments: YARN-851-20130618.patch, YARN-851-20130619.patch, YARN-851-20130619.patch, YARN-851-20130620.1.patch, YARN-851-20130620.2.patch, YARN-851-20130620.patch It is a follow up ticket for YARN-694. Changing the way NMTokens are shared. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1612) Change Fair Scheduler to not disable delay scheduling by default
Sandy Ryza created YARN-1612: Summary: Change Fair Scheduler to not disable delay scheduling by default Key: YARN-1612 URL: https://issues.apache.org/jira/browse/YARN-1612 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Reporter: Sandy Ryza -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1573) ZK store should pick a private password for root-node-acls
[ https://issues.apache.org/jira/browse/YARN-1573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875051#comment-13875051 ] Karthik Kambatla commented on YARN-1573: The private password is only for create-delete perms. The admin still has admin access and can reclaim all other kinds of accesses as well. ZK store should pick a private password for root-node-acls -- Key: YARN-1573 URL: https://issues.apache.org/jira/browse/YARN-1573 Project: Hadoop YARN Issue Type: Sub-task Components: resourcemanager Affects Versions: 2.4.0 Reporter: Karthik Kambatla Assignee: Karthik Kambatla Attachments: yarn-1573-1.patch Currently, when HA is enabled, ZK store uses cluster-timestamp as the password for root node ACLs to give the Active RM exclusive access to the store. A more private value like a random number might be better. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1534) TestAHSWebApp failed in YARN-321 branch
[
https://issues.apache.org/jira/browse/YARN-1534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhijie Shen updated YARN-1534:
--
Issue Type: Sub-task (was: Bug)
Parent: YARN-321
TestAHSWebApp failed in YARN-321 branch
---
Key: YARN-1534
URL: https://issues.apache.org/jira/browse/YARN-1534
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Environment: CentOS 6.3, JDK 1.6.0_31
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Fix For: YARN-321
Attachments: YARN-1534.patch
I ran the following commands. And I confirmed failure of TestAHSWebApp.
{code}
[sinchii@hdX YARN-321-test]$ mvn clean test
-Dtest=org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.*
{code}
{code}
Running
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices
Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.492 sec -
in
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebServices
Running
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebApp
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.193 sec
FAILURE! - in
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebApp
initializationError(org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.TestAHSWebApp)
Time elapsed: 0.016 sec ERROR!
java.lang.Exception: Test class should have exactly one public zero-argument
constructor
at
org.junit.runners.BlockJUnit4ClassRunner.validateZeroArgConstructor(BlockJUnit4ClassRunner.java:144)
at
org.junit.runners.BlockJUnit4ClassRunner.validateConstructor(BlockJUnit4ClassRunner.java:121)
at
org.junit.runners.BlockJUnit4ClassRunner.collectInitializationErrors(BlockJUnit4ClassRunner.java:101)
at org.junit.runners.ParentRunner.validate(ParentRunner.java:344)
(*snip*)
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1578) Fix how to handle ApplicationHistory about the container
[
https://issues.apache.org/jira/browse/YARN-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhijie Shen updated YARN-1578:
--
Issue Type: Sub-task (was: Bug)
Parent: YARN-321
Fix how to handle ApplicationHistory about the container
Key: YARN-1578
URL: https://issues.apache.org/jira/browse/YARN-1578
Project: Hadoop YARN
Issue Type: Sub-task
Affects Versions: YARN-321
Reporter: Shinichi Yamashita
Assignee: Shinichi Yamashita
Attachments: YARN-1578.patch, screenshot.png
I carried out PiEstimator job at Hadoop cluster which applied YARN-321.
After the job end and when I accessed Web UI of HistoryServer, it displayed
500. And HistoryServer daemon log was output as follows.
{code}
2014-01-09 13:31:12,227 ERROR org.apache.hadoop.yarn.webapp.Dispatcher: error
handling URI:
/applicationhistory/appattempt/appattempt_1389146249925_0008_01
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
(snip...)
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore.mergeContainerHistoryData(FileSystemApplicationHistoryStore.java:696)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.FileSystemApplicationHistoryStore.getContainers(FileSystemApplicationHistoryStore.java:429)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerImpl.getContainers(ApplicationHistoryManagerImpl.java:201)
at
org.apache.hadoop.yarn.server.webapp.AppAttemptBlock.render(AppAttemptBlock.java:110)
(snip...)
{code}
I confirmed that there was container which was not finished from
ApplicationHistory file.
In ResourceManager daemon log, ResourceManager reserved this container, but
did not allocate it.
Therefore, about a container which is not allocated, it is necessary to
change how to handle in ApplicationHistory.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1197) Support changing resources of an allocated container
[ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875079#comment-13875079 ] Sandy Ryza commented on YARN-1197: -- Created a YARN-1197 branch. Will revert the commits in trunk and branch-2 soon. Support changing resources of an allocated container Key: YARN-1197 URL: https://issues.apache.org/jira/browse/YARN-1197 Project: Hadoop YARN Issue Type: Task Components: api, nodemanager, resourcemanager Affects Versions: 2.1.0-beta Reporter: Wangda Tan Assignee: Wangda Tan Attachments: mapreduce-project.patch.ver.1, tools-project.patch.ver.1, yarn-1197-scheduler-v1.pdf, yarn-1197-v2.pdf, yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1, yarn-server-resourcemanager.patch.ver.1 The current YARN resource management logic assumes resource allocated to a container is fixed during the lifetime of it. When users want to change a resource of an allocated container the only way is releasing it and allocating a new container with expected size. Allowing run-time changing resources of an allocated container will give us better control of resource usage in application side -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1613) Fix config name YARN_HISTORY_SERVICE_ENABLED
Zhijie Shen created YARN-1613: - Summary: Fix config name YARN_HISTORY_SERVICE_ENABLED Key: YARN-1613 URL: https://issues.apache.org/jira/browse/YARN-1613 Project: Hadoop YARN Issue Type: Sub-task Reporter: Zhijie Shen YARN_HISTORY_SERVICE_ENABLED property name is yarn.ahs..enabled, which is wrong. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1587) [YARN-321] Merge Patch for YARN-321
[ https://issues.apache.org/jira/browse/YARN-1587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875117#comment-13875117 ] Zhijie Shen commented on YARN-1587: --- bq. YARN_HISTORY_SERVICE_ENABLED property name is yarn.ahs..enabled. Is this property name correct? Good catch! Filed a ticket for it: YARN-1613. [YARN-321] Merge Patch for YARN-321 --- Key: YARN-1587 URL: https://issues.apache.org/jira/browse/YARN-1587 Project: Hadoop YARN Issue Type: Sub-task Reporter: Mayank Bansal Assignee: Vinod Kumar Vavilapalli Fix For: YARN-321 Attachments: YARN-1587-20140113.txt, YARN-1587-20140114.txt, YARN-1587-20140115.txt, YARN-1587-20140116.1.txt, YARN-1587-20140116.txt, YARN-321-merge-1.patch Merge Patch -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Created] (YARN-1614) JobHistory doesn't support fully-functional search
Siqi Li created YARN-1614: - Summary: JobHistory doesn't support fully-functional search Key: YARN-1614 URL: https://issues.apache.org/jira/browse/YARN-1614 Project: Hadoop YARN Issue Type: Bug Reporter: Siqi Li Assignee: Siqi Li Priority: Critical -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1614) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/YARN-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated YARN-1614: -- Description: job history server will only output the first 50 characters of the job names in webUI. JobHistory doesn't support fully-functional search -- Key: YARN-1614 URL: https://issues.apache.org/jira/browse/YARN-1614 Project: Hadoop YARN Issue Type: Bug Reporter: Siqi Li Assignee: Siqi Li Priority: Critical job history server will only output the first 50 characters of the job names in webUI. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1614) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/YARN-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated YARN-1614: -- Attachment: YARN-1614.v1.patch JobHistory doesn't support fully-functional search -- Key: YARN-1614 URL: https://issues.apache.org/jira/browse/YARN-1614 Project: Hadoop YARN Issue Type: Bug Reporter: Siqi Li Assignee: Siqi Li Priority: Critical Attachments: YARN-1614.v1.patch job history server will only output the first 50 characters of the job names in webUI. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1614) JobHistory doesn't support fully-functional search
[
https://issues.apache.org/jira/browse/YARN-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875257#comment-13875257
]
Hadoop QA commented on YARN-1614:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623724/YARN-1614.v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2907//console
This message is automatically generated.
JobHistory doesn't support fully-functional search
--
Key: YARN-1614
URL: https://issues.apache.org/jira/browse/YARN-1614
Project: Hadoop YARN
Issue Type: Bug
Reporter: Siqi Li
Assignee: Siqi Li
Priority: Critical
Attachments: YARN-1614.v1.patch
job history server will only output the first 50 characters of the job names
in webUI.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (YARN-1614) JobHistory doesn't support fully-functional search
[ https://issues.apache.org/jira/browse/YARN-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siqi Li updated YARN-1614: -- Attachment: YARN-1614.v2.patch JobHistory doesn't support fully-functional search -- Key: YARN-1614 URL: https://issues.apache.org/jira/browse/YARN-1614 Project: Hadoop YARN Issue Type: Bug Reporter: Siqi Li Assignee: Siqi Li Priority: Critical Attachments: YARN-1614.v1.patch, YARN-1614.v2.patch job history server will only output the first 50 characters of the job names in webUI. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1614) JobHistory doesn't support fully-functional search
[
https://issues.apache.org/jira/browse/YARN-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875372#comment-13875372
]
Hadoop QA commented on YARN-1614:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12623739/YARN-1614.v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common:
org.apache.hadoop.mapreduce.v2.app.TestRMContainerAllocator
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/2908//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/2908//console
This message is automatically generated.
JobHistory doesn't support fully-functional search
--
Key: YARN-1614
URL: https://issues.apache.org/jira/browse/YARN-1614
Project: Hadoop YARN
Issue Type: Bug
Reporter: Siqi Li
Assignee: Siqi Li
Priority: Critical
Attachments: YARN-1614.v1.patch, YARN-1614.v2.patch
job history server will only output the first 50 characters of the job names
in webUI.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (YARN-1609) Add Service Container type to NodeManager in YARN
[ https://issues.apache.org/jira/browse/YARN-1609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875418#comment-13875418 ] Hitesh Shah commented on YARN-1609: --- [~lvm] I think I understand what the goals are though I am still unclear on the specifics as to what the real problems were when trying to launch processes directly via the NM as compared to using the service container. - Is the NM flow too slow for launching a new container or killing one? - Is it difficult to monitoring of the status of a container? - Can the NM not handle launching/managing a large no. of containers on a single machine? It would also be helpful to understand what kind of workarounds/hacks you needed to do so as to understand the underlying problems that you faced. Add Service Container type to NodeManager in YARN - Key: YARN-1609 URL: https://issues.apache.org/jira/browse/YARN-1609 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 2.2.0 Reporter: Wangda Tan Assignee: Wangda Tan Attachments: Add Service Container type to NodeManager in YARN-V1.pdf From our work to support running OpenMPI on YARN (MAPREDUCE-2911), we found that it’s important to have framework specific daemon process manage the tasks on each node directly. The daemon process, most likely similar in other frameworks as well, provides critical services to tasks running on that node(for example “wireup”, spawn user process in large numbers at once etc). In YARN, it’s hard, if not possible, to have the those processes to be managed by YARN. We propose to extend the container model on NodeManager side to support “Service Container” to run/manage such framework daemon/services process. We believe this is very useful to other application framework developers as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (YARN-1525) Web UI should redirect to active RM when HA is enabled.
[ https://issues.apache.org/jira/browse/YARN-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cindy Li updated YARN-1525: --- Attachment: YARN1525.patch.v1 This 1st patch is tested on a local VM. It redirects the standby RM's url to the active RM's url in 3seconds. If no active RM at a time, it will stays in the standby's RM. /cluster/cluster, and /ws/v1/cluster/info are not redirected. Web UI should redirect to active RM when HA is enabled. --- Key: YARN-1525 URL: https://issues.apache.org/jira/browse/YARN-1525 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Cindy Li Attachments: YARN1525.patch.v1 When failover happens, web UI should redirect to the current active rm. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1197) Support changing resources of an allocated container
[ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875509#comment-13875509 ] Wangda Tan commented on YARN-1197: -- Thanks Sandy, I'm working on breaking down RM patch and will create sub-jiras for better review. Support changing resources of an allocated container Key: YARN-1197 URL: https://issues.apache.org/jira/browse/YARN-1197 Project: Hadoop YARN Issue Type: Task Components: api, nodemanager, resourcemanager Affects Versions: 2.1.0-beta Reporter: Wangda Tan Assignee: Wangda Tan Attachments: mapreduce-project.patch.ver.1, tools-project.patch.ver.1, yarn-1197-scheduler-v1.pdf, yarn-1197-v2.pdf, yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1, yarn-server-resourcemanager.patch.ver.1 The current YARN resource management logic assumes resource allocated to a container is fixed during the lifetime of it. When users want to change a resource of an allocated container the only way is releasing it and allocating a new container with expected size. Allowing run-time changing resources of an allocated container will give us better control of resource usage in application side -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1525) Web UI should redirect to active RM when HA is enabled.
[ https://issues.apache.org/jira/browse/YARN-1525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13875553#comment-13875553 ] Karthik Kambatla commented on YARN-1525: Thanks [~cindy2012] for the patch. Would it be possible to leave out formatting changes unrelated to the patch? For intstance, the license header in the beginning of each file? That way, it would be easy to review what exact changes went into the commit. Web UI should redirect to active RM when HA is enabled. --- Key: YARN-1525 URL: https://issues.apache.org/jira/browse/YARN-1525 Project: Hadoop YARN Issue Type: Sub-task Reporter: Xuan Gong Assignee: Cindy Li Attachments: YARN1525.patch.v1 When failover happens, web UI should redirect to the current active rm. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
