[
https://issues.apache.org/jira/browse/YARN-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15997127#comment-15997127
]
Shane Kumpf commented on YARN-6515:
-----------------------------------
Thank you [~Naganarasimha] and [~ajisakaa] for bringing up the cgroup mount
path hardcode issue.
I agree with [[email protected]] that using
YarnConfiguration.NM_LINUX_CONTAINER_CGROUPS_MOUNT_PATH will require a
documentation update due to the explicit wording of when it used. However, the
setting does seem to be appropriate for this use and I think introducing a new
configuration would be superfluous.
It would be good to remove the CGROUPS_ROOT_DIRECTORY variable all together and
add a default value to YarnConfiguration ("/sys/fs/cgroup" to maintain
backward compatibility). This will require an update to
TestDockerContainerRuntime as well.
{quote} Also, the location inside the container might still need to be hard
coded or configured by a separate entry to support isolation better.{quote}
IMO, I don't think we should change behavior here and allow the mount point in
the container to differ from the source (i.e. /sys/fs/cgroup should be bind
mounted at /sys/fs/cgroup in the container, as we do today). I believe we can
remove this automatic cgroup mount and address the need for different src/dest
paths once YARN-5534 is in.
> Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager
> ------------------------------------------------------------
>
> Key: YARN-6515
> URL: https://issues.apache.org/jira/browse/YARN-6515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Reporter: Naganarasimha G R
> Assignee: Naganarasimha G R
> Attachments: YARN-6515.001.patch
>
>
> 5 find bugs issue was reported NM project as part of the YARN-4166 [build|
> https://builds.apache.org/job/PreCommit-YARN-Build/15694/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html]
> Issue 1:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> is a mutable collection which should be package protected
> Bug type MS_MUTABLE_COLLECTION_PKGPROTECT (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainerMetrics.usageMetrics
> At ContainerMetrics.java:\[line 134\]
> Issue 2:
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.createStatus()
> Field
> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.pendingResources
> At ContainerLocalizer.java:\[line 334\]
> Issue 3:
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> makes inefficient use of keySet iterator instead of entrySet iterator
> Bug type WMI_WRONG_MAP_ITERATOR (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeVeryOldStoppedContainersFromCache()
> Field
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.recentlyStoppedContainers
> At NodeStatusUpdaterImpl.java:\[line 721\]
> Issue 4:
> Hard coded reference to an absolute pathname in
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> Bug type DMI_HARDCODED_ABSOLUTE_FILENAME (click for details)
> In class
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime
> In method
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime.launchContainer(ContainerRuntimeContext)
> File name /sys/fs/cgroup
> At DockerLinuxContainerRuntime.java:\[line 455\]
> Useless object stored in variable removedNullContainers of method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Bug type UC_USELESS_OBJECT (click for details)
> In class org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl
> In method
> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.removeOrTrackCompletedContainersFromContext(List)
> Value removedNullContainers
> Type java.util.HashSet
> At NodeStatusUpdaterImpl.java:\[line 644\]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
