[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369731#comment-15369731 ] Hudson commented on YARN-4058: -- SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10074/]) YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G (sjlee: rev 8d9476ec5f64795e2bdd60bea4d643eaafed2c50) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TimelineServiceV2Publisher.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/TestSystemMetricsPublisherForV2.java > Miscellaneous issues in NodeManager project > --- > > Key: YARN-4058 > URL: https://issues.apache.org/jira/browse/YARN-4058 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R >Priority: Minor > Fix For: YARN-2928 > > Attachments: YARN-4058.YARN-2928.001.patch, > YARN-4058.YARN-2928.002.patch > > > # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing > # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is > created and then checked whether it exists in context.getApplications(). > everytime ApplicationImpl is created state machine is intialized and > TimelineClient is created which is required only if added to the context. > # Remove unused imports in TimelineServiceV2Publisher & > TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720017#comment-14720017 ] Junping Du commented on YARN-4058: -- [~sjlee0], the JIRA number is not correct in your commits. Please update it to a correct one. Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Fix For: YARN-2928 Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720212#comment-14720212 ] Junping Du commented on YARN-4058: -- Yes. Adding a new commit to correct hadoop-yarn/CHANGES.txt is the right way. Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Fix For: YARN-2928 Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720204#comment-14720204 ] Sangjin Lee commented on YARN-4058: --- Thanks for finding that [~djp]. I'd love to edit that commit, but then that would disrupt you all again because I need to force push. How about adding a new commit that fixes hadoop-yarn/CHANGES.txt? Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Fix For: YARN-2928 Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14720242#comment-14720242 ] Sangjin Lee commented on YARN-4058: --- Done. Thanks. Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Fix For: YARN-2928 Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709753#comment-14709753 ] Naganarasimha G R commented on YARN-4058: - moving 2nd point to another jira Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # Unused ApplicationACLsManager in ContainerManagerImpl # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709817#comment-14709817 ] Sangjin Lee commented on YARN-4058: --- Sounds good. I'll review it promptly when you post the updated patch. Thanks! Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[
https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709843#comment-14709843
]
Sangjin Lee commented on YARN-4058:
---
Thanks for the update. How about flipping the order of {{null}} and
{{context.getApplications().putIfAbsent(applicationID, application)}}? :)
Miscellaneous issues in NodeManager project
---
Key: YARN-4058
URL: https://issues.apache.org/jira/browse/YARN-4058
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Naganarasimha G R
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-4058.YARN-2928.001.patch,
YARN-4058.YARN-2928.002.patch
# TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing
# In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is
created and then checked whether it exists in context.getApplications().
everytime ApplicationImpl is created state machine is intialized and
TimelineClient is created which is required only if added to the context.
# Remove unused imports in TimelineServiceV2Publisher
TestSystemMetricsPublisherForV2.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709860#comment-14709860 ] Naganarasimha G R commented on YARN-4058: - Oops, my mistake deleting and uploading new one Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709868#comment-14709868 ] Sangjin Lee commented on YARN-4058: --- LGTM. Once the jenkins comes back and unless I hear objections, I'll commit the patch. Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[
https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710040#comment-14710040
]
Hadoop QA commented on YARN-4058:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 0s | Findbugs (version ) appears to
be broken on YARN-2928. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 8m 20s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 10m 3s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 33s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 28s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 41s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 48s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 6m 47s | Tests passed in
hadoop-yarn-server-nodemanager. |
| {color:red}-1{color} | yarn tests | 23m 55s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 71m 1s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests |
hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler |
| | hadoop.yarn.server.resourcemanager.TestApplicationACLs |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
|
| | hadoop.yarn.server.resourcemanager.TestApplicationCleanup |
| | hadoop.yarn.server.resourcemanager.TestRM |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps |
| | hadoop.yarn.server.resourcemanager.TestResourceTrackerService |
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification |
| | hadoop.yarn.server.resourcemanager.security.TestAMRMTokens |
| | hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
|
| | hadoop.yarn.server.resourcemanager.TestMoveApplication |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings |
| | hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
| | hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler |
| | hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterService |
| | hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs |
| | hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth |
| | hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils |
| | hadoop.yarn.server.resourcemanager.webapp.TestRMWebappAuthentication |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
|
| |
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesDelegationTokenAuthentication
|
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
|
| | hadoop.yarn.server.resourcemanager.TestRMAdminService |
| | hadoop.yarn.server.resourcemanager.resourcetracker.TestRMNMRPCResponseId |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
| | hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| |
hadoop.yarn.server.resourcemanager.metrics.TestSystemMetricsPublisherForV2 |
| Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestRMHA |
| |
org.apache.hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12752068/YARN-4058.YARN-2928.002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / 9d14947 |
| hadoop-yarn-server-nodemanager
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710344#comment-14710344 ] Naganarasimha G R commented on YARN-4058: - thanks for review and commit [~sjlee0] Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Fix For: YARN-2928 Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[
https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710283#comment-14710283
]
Hadoop QA commented on YARN-4058:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | pre-patch | 16m 19s | Findbugs (version ) appears to
be broken on YARN-2928. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 59s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 58s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 0m 45s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 29s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 40s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 45s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 6m 20s | Tests passed in
hadoop-yarn-server-nodemanager. |
| {color:red}-1{color} | yarn tests | 43m 19s | Tests failed in
hadoop-yarn-server-resourcemanager. |
| | | 90m 2s | |
\\
\\
|| Reason || Tests ||
| Timed out tests | org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
|
| | org.apache.hadoop.yarn.server.resourcemanager.TestRM |
| |
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
| | org.apache.hadoop.yarn.server.resourcemanager.TestRMProxyUsersConf |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12752068/YARN-4058.YARN-2928.002.patch
|
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | YARN-2928 / 9d14947 |
| hadoop-yarn-server-nodemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8902/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
|
| hadoop-yarn-server-resourcemanager test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8902/artifact/patchprocess/testrun_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8902/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8902/console |
This message was automatically generated.
Miscellaneous issues in NodeManager project
---
Key: YARN-4058
URL: https://issues.apache.org/jira/browse/YARN-4058
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Naganarasimha G R
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-4058.YARN-2928.001.patch,
YARN-4058.YARN-2928.002.patch
# TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing
# In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is
created and then checked whether it exists in context.getApplications().
everytime ApplicationImpl is created state machine is intialized and
TimelineClient is created which is required only if added to the context.
# Remove unused imports in TimelineServiceV2Publisher
TestSystemMetricsPublisherForV2.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710288#comment-14710288 ] Sangjin Lee commented on YARN-4058: --- The unit test result seems fine to me: https://builds.apache.org/job/PreCommit-YARN-Build/8902/testReport/ If there is no objection, I'll commit this patch soon. Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch, YARN-4058.YARN-2928.002.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. # Remove unused imports in TimelineServiceV2Publisher TestSystemMetricsPublisherForV2.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[
https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708467#comment-14708467
]
Naganarasimha G R commented on YARN-4058:
-
Thanks [~sjlee0] for the review comments,
Yes it makes sense to move ApplicationACLsManager related modifications in
trunk itself as t might involve more changes if we touch constructor.
bq. This style (having values like null first and then the expression) is a
carry-over from the C-style defensive coding, and is no longer needed in java.
Ok, Will take care of it in the next patch.
bq. Then the timeline client must be stopped properly as part of lifecycle
management.
True, we need to handle this too, will take care of it in YARN-3367. For where
to do it; As part of initial thoughts i was planning to do it when
ApplicationImpl(NM class) reaches {{ApplicationState.FINISH_APPLICATION}}.
Miscellaneous issues in NodeManager project
---
Key: YARN-4058
URL: https://issues.apache.org/jira/browse/YARN-4058
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Naganarasimha G R
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-4058.YARN-2928.001.patch
# TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing
# Unused ApplicationACLsManager in ContainerManagerImpl
# In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is
created and then checked whether it exists in context.getApplications().
everytime ApplicationImpl is created state machine is intialized and
TimelineClient is created which is required only if added to the context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[
https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707818#comment-14707818
]
Sangjin Lee commented on YARN-4058:
---
I see. We might then want to address it on our branch. I would still be
reluctant to touch {{ApplicationACLsManager}} or even the constructor. We would
end up touching quite a few files just to do that clean-up. For that issue, I'd
raise it for the trunk and have it fixed there.
As for the application creation issue, I have one small nit, and one larger
question to discuss. As for the nit,
{code}
901 if (null ==
context.getApplications().putIfAbsent(applicationID,
902 application)) {
{code}
This style (having values like null first and then the expression) is a
carry-over from the C-style defensive coding, and is no longer needed in java.
I would just do the normal style:
{code}
901 if (context.getApplications().putIfAbsent(applicationID,
902 application) == null) {
{code}
I know it's existing code, but it'd be good to change that.
The larger question is this. The timeline client is being created *and started*
as part of the {{ApplicationImpl}} constructor call. However, it seems like
we're not stopping the timeline client anywhere. Right now, it might not be a
big issue as the timeline client isn't doing anything special during the
service start, but what if we create things like thread pools as part of the
timeline client as you suggested? Then the timeline client must be stopped
properly as part of lifecycle management. It's not clear how we can do that if
they are created inside {{ApplicationImpl}}. Any thoughts on that?
Miscellaneous issues in NodeManager project
---
Key: YARN-4058
URL: https://issues.apache.org/jira/browse/YARN-4058
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Naganarasimha G R
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-4058.YARN-2928.001.patch
# TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing
# Unused ApplicationACLsManager in ContainerManagerImpl
# In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is
created and then checked whether it exists in context.getApplications().
everytime ApplicationImpl is created state machine is intialized and
TimelineClient is created which is required only if added to the context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[
https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707144#comment-14707144
]
Sangjin Lee commented on YARN-4058:
---
If I'm not mistaken, the issues with {{ContainerManagerImpl}} exist in the
trunk (the unused {{ApplicationACLsManager}} issue and the application
creation). I think we should have a JIRA against the trunk and have it fixed
there. Thoughts?
As for the other changes, it looks good.
Miscellaneous issues in NodeManager project
---
Key: YARN-4058
URL: https://issues.apache.org/jira/browse/YARN-4058
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Naganarasimha G R
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-4058.YARN-2928.001.patch
# TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing
# Unused ApplicationACLsManager in ContainerManagerImpl
# In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is
created and then checked whether it exists in context.getApplications().
everytime ApplicationImpl is created state machine is intialized and
TimelineClient is created which is required only if added to the context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14707703#comment-14707703 ] Naganarasimha G R commented on YARN-4058: - thanks [~sjlee0] for the review comments, The reason for raising the application creation issue here was, the impact is more with our case as we are creating TimelineClient as part of application creation and further we might introduce some thread pools and queues inside TimelineClient for putAsync and other was a small issue with unused variable hence clubbed with other issues in this jira, if required i can segregate it. Thoughts? Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # Unused ApplicationACLsManager in ContainerManagerImpl # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706294#comment-14706294 ] Naganarasimha G R commented on YARN-4058: - Seems like ApplicationACLsManager was present before the Context was created. As context has the ACLManager I think we can remove from the constructor too. Thoughts? Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # Unused ApplicationACLsManager in ContainerManagerImpl # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706278#comment-14706278 ] Naganarasimha G R commented on YARN-4058: - Hi [~djp] [~sjlee0], I have fixed the above mentioned issues can you please take a look at it ? Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-4058.YARN-2928.001.patch # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # Unused ApplicationACLsManager in ContainerManagerImpl # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4058) Miscellaneous issues in NodeManager project
[ https://issues.apache.org/jira/browse/YARN-4058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700679#comment-14700679 ] Naganarasimha G R commented on YARN-4058: - Hi [~djp] [~vinodkv], Have created this jira for the issues identified during YARN-2928 reset and other minor issues. Please add other issues (if any) which were found during the reset which needs to be taken care. Miscellaneous issues in NodeManager project --- Key: YARN-4058 URL: https://issues.apache.org/jira/browse/YARN-4058 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Naganarasimha G R Assignee: Naganarasimha G R Priority: Minor # TestSystemMetricsPublisherForV2.testPublishApplicationMetrics is failing # Unused ApplicationACLsManager in ContainerManagerImpl # In ContainerManagerImpl.startContainerInternal ApplicationImpl instance is created and then checked whether it exists in context.getApplications(). everytime ApplicationImpl is created state machine is intialized and TimelineClient is created which is required only if added to the context. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
