[jira] [Updated] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command
[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3070: - Hadoop Flags: Reviewed Thanks [~hitliuyi] for review. Committing v2 patch in. TestRMAdminCLI#testHelp fails for transitionToActive command Key: YARN-3070 URL: https://issues.apache.org/jira/browse/YARN-3070 Project: Hadoop YARN Issue Type: Test Reporter: Ted Yu Assignee: Junping Du Priority: Minor Attachments: YARN-3070-v2.patch, YARN-3070.patch {code} testError(new String[] { -help, -transitionToActive }, Usage: yarn rmadmin [-transitionToActive serviceId + [--forceactive]], dataErr, 0); {code} fails with: {code} java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testError(TestRMAdminCLI.java:547) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testHelp(TestRMAdminCLI.java:335) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3037) create HBase cluster backing storage implementation for ATS writes
[ https://issues.apache.org/jira/browse/YARN-3037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282094#comment-14282094 ] Li Lu commented on YARN-3037: - Thanks [~zjshen] for pointing me to this. I agree that the initial steps of this issue is overlapped with YARN-2032. Maybe we would like to relate this issue to the previous one, which [~mayank_bansal] and I did some POC patches? I can certainly provide help on this issue, too. It would be great if we can have some further discussion about how to coordinate on this, since we may avoid some duplicated work on HBase based storage. create HBase cluster backing storage implementation for ATS writes -- Key: YARN-3037 URL: https://issues.apache.org/jira/browse/YARN-3037 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Sangjin Lee Assignee: Vrushali C Per design in YARN-2928, create a backing storage implementation for ATS writes based on a full HBase cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3071) Remove invalid char from sample conf in doc of FairScheduler
[ https://issues.apache.org/jira/browse/YARN-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated YARN-3071: --- Attachment: YARN-3071.001.patch Remove invalid char from sample conf in doc of FairScheduler Key: YARN-3071 URL: https://issues.apache.org/jira/browse/YARN-3071 Project: Hadoop YARN Issue Type: Bug Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Priority: Trivial Attachments: YARN-3071.001.patch coping and pasting conf from doc cause failure on RM startup {code} Caused by: org.xml.sax.SAXParseException; systemId: file:/home/iwasakims/dist/hadoop-2.6.0/etc/hadoop/fair-scheduler.xml; lineNumber: 18; columnNumber: 5; The content of elements must consist of well-formed character data or markup. at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:250) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1275) ... 9 more {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-3071) Remove invalid char from sample conf in doc of FairScheduler
Masatake Iwasaki created YARN-3071: -- Summary: Remove invalid char from sample conf in doc of FairScheduler Key: YARN-3071 URL: https://issues.apache.org/jira/browse/YARN-3071 Project: Hadoop YARN Issue Type: Bug Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Priority: Trivial coping and pasting conf from doc cause failure on RM startup {code} Caused by: org.xml.sax.SAXParseException; systemId: file:/home/iwasakims/dist/hadoop-2.6.0/etc/hadoop/fair-scheduler.xml; lineNumber: 18; columnNumber: 5; The content of elements must consist of well-formed character data or markup. at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:250) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1275) ... 9 more {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2616) Add CLI client to the registry to list/view entries
[ https://issues.apache.org/jira/browse/YARN-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akshay Radia updated YARN-2616: --- Attachment: yarn-2616-v7.patch Steve, addressed your feedback. Will add the cli documentation after YARN-2683 is checked in Add CLI client to the registry to list/view entries --- Key: YARN-2616 URL: https://issues.apache.org/jira/browse/YARN-2616 Project: Hadoop YARN Issue Type: Sub-task Components: client Reporter: Steve Loughran Assignee: Akshay Radia Attachments: YARN-2616-003.patch, yarn-2616-v1.patch, yarn-2616-v2.patch, yarn-2616-v4.patch, yarn-2616-v5.patch, yarn-2616-v6.patch, yarn-2616-v7.patch registry needs a CLI interface -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2616) Add CLI client to the registry to list/view entries
[ https://issues.apache.org/jira/browse/YARN-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282230#comment-14282230 ] Hadoop QA commented on YARN-2616: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12693027/yarn-2616-v7.patch against trunk revision 19cbce3. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6357//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6357//console This message is automatically generated. Add CLI client to the registry to list/view entries --- Key: YARN-2616 URL: https://issues.apache.org/jira/browse/YARN-2616 Project: Hadoop YARN Issue Type: Sub-task Components: client Reporter: Steve Loughran Assignee: Akshay Radia Attachments: YARN-2616-003.patch, yarn-2616-v1.patch, yarn-2616-v2.patch, yarn-2616-v4.patch, yarn-2616-v5.patch, yarn-2616-v6.patch, yarn-2616-v7.patch registry needs a CLI interface -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3071) Remove invalid char from sample conf in doc of FairScheduler
[ https://issues.apache.org/jira/browse/YARN-3071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated YARN-3071: --- Description: copying and pasting conf causes failure on RM startup {code} Caused by: org.xml.sax.SAXParseException; systemId: file:/home/iwasakims/dist/hadoop-2.6.0/etc/hadoop/fair-scheduler.xml; lineNumber: 18; columnNumber: 5; The content of elements must consist of well-formed character data or markup. at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:250) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1275) ... 9 more {code} was: coping and pasting conf from doc cause failure on RM startup {code} Caused by: org.xml.sax.SAXParseException; systemId: file:/home/iwasakims/dist/hadoop-2.6.0/etc/hadoop/fair-scheduler.xml; lineNumber: 18; columnNumber: 5; The content of elements must consist of well-formed character data or markup. at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:250) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1275) ... 9 more {code} Remove invalid char from sample conf in doc of FairScheduler Key: YARN-3071 URL: https://issues.apache.org/jira/browse/YARN-3071 Project: Hadoop YARN Issue Type: Bug Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki Priority: Trivial Attachments: YARN-3071.001.patch copying and pasting conf causes failure on RM startup {code} Caused by: org.xml.sax.SAXParseException; systemId: file:/home/iwasakims/dist/hadoop-2.6.0/etc/hadoop/fair-scheduler.xml; lineNumber: 18; columnNumber: 5; The content of elements must consist of well-formed character data or markup. at org.apache.xerces.parsers.DOMParser.parse(Unknown Source) at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source) at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:205) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadAllocations(AllocationFileLoaderService.java:250) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.initScheduler(FairScheduler.java:1275) ... 9 more {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command
[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282211#comment-14282211 ] Hudson commented on YARN-3070: -- FAILURE: Integrated in Hadoop-trunk-Commit #6885 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6885/]) YARN-3070. TestRMAdminCLI#testHelp fails for transitionToActive command. (Contributed by Junping Du) (junping_du: rev 19cbce3898aeee7dd3b46e2c2ffeae25ff6ba88f) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java TestRMAdminCLI#testHelp fails for transitionToActive command Key: YARN-3070 URL: https://issues.apache.org/jira/browse/YARN-3070 Project: Hadoop YARN Issue Type: Test Reporter: Ted Yu Assignee: Junping Du Priority: Minor Attachments: YARN-3070-v2.patch, YARN-3070.patch {code} testError(new String[] { -help, -transitionToActive }, Usage: yarn rmadmin [-transitionToActive serviceId + [--forceactive]], dataErr, 0); {code} fails with: {code} java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testError(TestRMAdminCLI.java:547) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testHelp(TestRMAdminCLI.java:335) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command
[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282046#comment-14282046 ] Yi Liu commented on YARN-3070: -- +1, other test failures are not related. TestRMAdminCLI#testHelp fails for transitionToActive command Key: YARN-3070 URL: https://issues.apache.org/jira/browse/YARN-3070 Project: Hadoop YARN Issue Type: Test Reporter: Ted Yu Assignee: Junping Du Priority: Minor Attachments: YARN-3070-v2.patch, YARN-3070.patch {code} testError(new String[] { -help, -transitionToActive }, Usage: yarn rmadmin [-transitionToActive serviceId + [--forceactive]], dataErr, 0); {code} fails with: {code} java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testError(TestRMAdminCLI.java:547) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testHelp(TestRMAdminCLI.java:335) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command
[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281723#comment-14281723 ] Hadoop QA commented on YARN-3070: - {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12692966/YARN-3070-v2.patch against trunk revision 24315e7. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: org.apache.hadoop.yarn.client.TestResourceTrackerOnHA org.apache.hadoop.yarn.client.api.impl.TestAMRMClient org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA Test results: https://builds.apache.org/job/PreCommit-YARN-Build/6356//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6356//console This message is automatically generated. TestRMAdminCLI#testHelp fails for transitionToActive command Key: YARN-3070 URL: https://issues.apache.org/jira/browse/YARN-3070 Project: Hadoop YARN Issue Type: Test Reporter: Ted Yu Assignee: Junping Du Priority: Minor Attachments: YARN-3070-v2.patch, YARN-3070.patch {code} testError(new String[] { -help, -transitionToActive }, Usage: yarn rmadmin [-transitionToActive serviceId + [--forceactive]], dataErr, 0); {code} fails with: {code} java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testError(TestRMAdminCLI.java:547) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testHelp(TestRMAdminCLI.java:335) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3070) TestRMAdminCLI#testHelp fails for transitionToActive command
[ https://issues.apache.org/jira/browse/YARN-3070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3070: - Attachment: YARN-3070-v2.patch Fix a typo of String.format() which cause the test failure. TestRMAdminCLI#testHelp fails for transitionToActive command Key: YARN-3070 URL: https://issues.apache.org/jira/browse/YARN-3070 Project: Hadoop YARN Issue Type: Test Reporter: Ted Yu Assignee: Junping Du Priority: Minor Attachments: YARN-3070-v2.patch, YARN-3070.patch {code} testError(new String[] { -help, -transitionToActive }, Usage: yarn rmadmin [-transitionToActive serviceId + [--forceactive]], dataErr, 0); {code} fails with: {code} java.lang.AssertionError: null at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testError(TestRMAdminCLI.java:547) at org.apache.hadoop.yarn.client.cli.TestRMAdminCLI.testHelp(TestRMAdminCLI.java:335) {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2683) registry config options: document and move to core-default
[ https://issues.apache.org/jira/browse/YARN-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281932#comment-14281932 ] Sanjay Radia commented on YARN-2683: Feedback on v003: * In the description for the enabled key, you should not mention who starts the registry and who creates which directories. That may change over time. That information should in the main documentation. * Gour saha's comment above * Nit: in some of the descriptions like Zookeeper session timeout you should say Registry Zookeeper session timeout (This is obvious from the key name .. but ..) * Security - I think this explanation could be improved. ** Briefly state that it is based ZK security and briefly describe it 2-3 sentences and how it will integrate with the Hadoop authentication. ** I am not completely clear on the usage model for system accounts. If one desires oozie's name to be registered then we need to add oozie's realm? This will be hard to manage over time. I am also confused by the sasl:ad...@example.com example; is this to allow an admin to manually add/remove entries for some services? Can we please chat offline and we can help clarify this part of the document. registry config options: document and move to core-default -- Key: YARN-2683 URL: https://issues.apache.org/jira/browse/YARN-2683 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.6.0 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-10530-005.patch, YARN-2683-001.patch, YARN-2683-002.patch, YARN-2683-003.patch Original Estimate: 1h Remaining Estimate: 1h Add to {{yarn-site}} a page on registry configuration parameters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2683) registry config options: document and move to core-default
[ https://issues.apache.org/jira/browse/YARN-2683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14281928#comment-14281928 ] Sanjay Radia commented on YARN-2683: Steve you attached HADOOP-10530-005 patch to this jira by mistake. Is there are newer patch for this jira or is version 003 good enough? registry config options: document and move to core-default -- Key: YARN-2683 URL: https://issues.apache.org/jira/browse/YARN-2683 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.6.0 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-10530-005.patch, YARN-2683-001.patch, YARN-2683-002.patch, YARN-2683-003.patch Original Estimate: 1h Remaining Estimate: 1h Add to {{yarn-site}} a page on registry configuration parameters -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3025) Provide API for retrieving blacklisted nodes
[ https://issues.apache.org/jira/browse/YARN-3025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14282006#comment-14282006 ] Ted Yu commented on YARN-3025: -- [~bikassaha]: Can you provide your opinion ? Thanks Provide API for retrieving blacklisted nodes Key: YARN-3025 URL: https://issues.apache.org/jira/browse/YARN-3025 Project: Hadoop YARN Issue Type: Improvement Reporter: Ted Yu We have the following method which updates blacklist: {code} public synchronized void updateBlacklist(ListString blacklistAdditions, ListString blacklistRemovals) { {code} Upon AM failover, there should be an API which returns the blacklisted nodes so that the new AM can make consistent decisions. The new API can be: {code} public synchronized ListString getBlacklistedNodes() {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)