[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162027#comment-17162027 ] Masatake Iwasaki commented on YARN-6515: cherry-picked this to branch-2.10. > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R >Priority: Major > Fix For: 3.0.0-beta1, 2.10.1 > > Attachments: YARN-6515.001.patch, YARN-6515.002.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121046#comment-16121046 ] Naganarasimha G R commented on YARN-6515: - Thanks for the review and commit [~ajisakaa] and reviews from [~cheersyang], [~miklos.szeg...@cloudera.com], [~shaneku...@gmail.com] > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Fix For: 3.0.0-beta1 > > Attachments: YARN-6515.001.patch, YARN-6515.002.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119889#comment-16119889 ] Hudson commented on YARN-6515: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12152 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12152/]) YARN-6515. Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager. (aajisaka: rev 1a18d5e514d13aa3a88e9b6089394a27296d6bc3) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainerMetrics.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Fix For: 3.0.0-beta1 > > Attachments: YARN-6515.001.patch, YARN-6515.002.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118688#comment-16118688 ] Miklos Szegedi commented on YARN-6515: -- Thanks [~Naganarasimha], I created YARN-6968 for the Docker issue. > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6515.001.patch, YARN-6515.002.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118280#comment-16118280 ] Shane Kumpf commented on YARN-6515: --- Thanks [~Naganarasimha] for the updated patch. With regard to the hard coded cgroup path in DLCR, I am fine with leaving it off this patch. The hard coded mount (and path) will be removed after YARN-5534 is in. The rest looks good to me, +1. > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6515.001.patch, YARN-6515.002.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16118201#comment-16118201 ] Akira Ajisaka commented on YARN-6515: - Thanks [~Naganarasimha] for the comment and the patch. +1. > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6515.001.patch, YARN-6515.002.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115856#comment-16115856
]
Hadoop QA commented on YARN-6515:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
57s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
generated 0 new + 1 unchanged - 4 fixed = 1 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
6s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12880569/YARN-6515.002.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d490047816fa 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 024c3ec |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/16727/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16727/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16727/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110462#comment-16110462
]
Hadoop QA commented on YARN-6515:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} YARN-6515 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-6515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12864692/YARN-6515.001.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/16665/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999026#comment-15999026 ] Miklos Szegedi commented on YARN-6515: -- Thank you for the response [~shaneku...@gmail.com]. In this case I think I would also add this as a comment to the mount code, if we keep it for now. > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6515.001.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997340#comment-15997340
]
Shane Kumpf commented on YARN-6515:
---
{quote}
what is the point overriding the cgroups directory mounted by Docker in the
worker containers?
{quote}
Thanks for bringing that up [~miklos.szeg...@cloudera.com]. At the time, I
believe it was required to allow systemd to run inside the containers. That
overlap is confusing and likely not the behavior wanted/anticipated.
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997197#comment-15997197
]
Miklos Szegedi commented on YARN-6515:
--
Thank you, [~shaneku...@gmail.com] for the explanation.
bq. Also, the location inside the container might still need to be hard coded
or configured by a separate entry to support isolation better.
[~naganarasimha...@apache.org], yes I am talking about the Docker container
here.
A little bit off topic but I am now curious, what is the point overriding the
cgroups directory mounted by Docker in the worker containers?
If I create a centos6.7 Docker container on centos7, I see overlapping IDs in
the two cgroups folders, that does no good I think and it will cause confusion
in the future:
host:
{code}
[root@miklos-dev2 ~]# cat /sys/fs/cgroup/cpu/tasks
1
2
3
5
7
8
9
...
26938
root 1 0 0 09:55 ?00:00:03 /usr/lib/systemd/systemd
--system --deserialize 22
{code}
container:
{code}
/ # cat /sys/fs/cgroup/cpu/tasks
1
9
1 root 0:00 sh
{code}
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997127#comment-15997127
]
Shane Kumpf commented on YARN-6515:
---
Thank you [~Naganarasimha] and [~ajisakaa] for bringing up the cgroup mount
path hardcode issue.
I agree with [~miklos.szeg...@cloudera.com] that using
YarnConfiguration.NM_LINUX_CONTAINER_CGROUPS_MOUNT_PATH will require a
documentation update due to the explicit wording of when it used. However, the
setting does seem to be appropriate for this use and I think introducing a new
configuration would be superfluous.
It would be good to remove the CGROUPS_ROOT_DIRECTORY variable all together and
add a default value to YarnConfiguration ("/sys/fs/cgroup" to maintain
backward compatibility). This will require an update to
TestDockerContainerRuntime as well.
{quote} Also, the location inside the container might still need to be hard
coded or configured by a separate entry to support isolation better.{quote}
IMO, I don't think we should change behavior here and allow the mount point in
the container to differ from the source (i.e. /sys/fs/cgroup should be bind
mounted at /sys/fs/cgroup in the container, as we do today). I believe we can
remove this automatic cgroup mount and address the need for different src/dest
paths once YARN-5534 is in.
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdater
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991907#comment-15991907
]
Naganarasimha G R commented on YARN-6515:
-
Thanks [~miklos.szeg...@cloudera.com] for pitching in,
Agree may be my fix is not correct and requires update in multiple places or
even cause change in behavior. IIUC it is not correct to use hard coded path in
*DockerLinuxContainerRuntime*, but not sure will it cause any impact.
bq. Also, the location inside the container might still need to be hard coded
or configured by a separate entry to support isolation better.
you mean docker container here right ? May be i am not completely aware of how
this {{"CGROUPS_ROOT_DIRECTORY"}} is used by the docker container. So please
suggest should we need to handle as a separate configuration entry ?
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991161#comment-15991161 ] Miklos Szegedi commented on YARN-6515: -- Thank you, [~ajisakaa] for raising this issue. [~Naganarasimha], if we do this cgroups change, I think we need to update the documentation in NodemanagerCgroups.md. So far this configuration was used just in the context of yarn.nodemanager.linux-container-executor.cgroups.mount=true. yarn-default.xml is another candidate. Also, the location inside the container might still need to be hard coded or configured by a separate entry to support isolation better. > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6515.001.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990919#comment-15990919
]
Naganarasimha G R commented on YARN-6515:
-
Thanks [~ajisakaa],
but the point which is made by the find bugs is valid and we should not go by
fixed *cgroups* mount folder which is currently being used. Wanted some
clarification from the owners of {{DockerLinuxContainerRuntime}}. cc/
[~sidharta-s], hope to get some clarification on this
One alternative is to open separate ticket for the same, but anyways would like
to wait till he responds.
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15990671#comment-15990671
]
Akira Ajisaka commented on YARN-6515:
-
Mostly looks good to me.
{code}
+cgroupsRootDir =
+conf.get(YarnConfiguration.NM_LINUX_CONTAINER_CGROUPS_MOUNT_PATH,
+CGROUPS_ROOT_DIRECTORY);
{code}
{code}
-.addMountLocation(CGROUPS_ROOT_DIRECTORY,
-CGROUPS_ROOT_DIRECTORY + ":ro", false);
+.addMountLocation(cgroupsRootDir,
+cgroupsRootDir + ":ro", false);
{code}
I'm thinking this patch changes the existing behavior. What do you think?
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
>
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Reporter: Naganarasimha G R
>Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15985282#comment-15985282
]
Hadoop QA commented on YARN-6515:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
43s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
generated 0 new + 0 unchanged - 5 fixed = 0 total (was 5) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
40s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6515 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12864692/YARN-6515.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 84b0eef219f9 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 93fa48f |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15743/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15743/testReport/ |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/15743/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.or
[jira] [Commented] (YARN-6515) Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
[ https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15980870#comment-15980870 ] Weiwei Yang commented on YARN-6515: --- Revised the title a bit for consistency :) > Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager > > > Key: YARN-6515 > URL: https://issues.apache.org/jira/browse/YARN-6515 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Reporter: Naganarasimha G R >Assignee: Naganarasimha G R > Attachments: YARN-6515.001.patch > > > 5 find bugs issue was reported NM project as part of the YARN-4166 [build| > https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html] > Issue 1: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > is a mutable collection which should be package protected > Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics > At ContainerMetrics.java:\[line 134\] > Issue 2: > > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus() > Field > org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources > At ContainerLocalizer.java:\[line 334\] > Issue 3: > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > makes inefficient use of keySet iterator instead of entrySet iterator > Bug type WMI_WRONG_MAP_ITERATOR (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache() > Field > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers > At NodeStatusUpdaterImpl.java:\[line 721\] > Issue 4: > Hard coded reference to an absolute pathname in > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details) > In class > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime > In method > org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext) > File name /sys/fs/cgroup > At DockerLinuxContainerRuntime.java:\[line 455\] > Useless object stored in variable removedNullContainers of method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Bug type UC_USELESS_OBJECT (click for details) > In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl > In method > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List) > Value removedNullContainers > Type java.util.HashSet > At NodeStatusUpdaterImpl.java:\[line 644\] -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
