[
https://issues.apache.org/jira/browse/YARN-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kinga Marton reassigned YARN-9371:
----------------------------------
Assignee: (was: Kinga Marton)
> Better logging for initialization of cgroups in CGroupsHandlerImpl
> ------------------------------------------------------------------
>
> Key: YARN-9371
> URL: https://issues.apache.org/jira/browse/YARN-9371
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Szilard Nemeth
> Priority: Major
>
> We already have YARN-9098 that dealt with (not yet merged, though) code
> duplication in cgroups setup code.
> However, there's still room for improvement in the area of cgroups
> initialization, especially in CGroupsHandlerImpl.
> 1.
> org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsHandlerImpl#initializeControllerPaths:
> There's a decision in this method: Either cgroups are parsed from the
> configured mount path or from the mounted cgroups descriptor file
> (/proc/mounts). This decisions needs to be logged, along with some details,
> e.g. cgroups mount path.
> 2. In ResourceHandlerModule#parseConfiguredCGroupPath: Every candidate file
> could be logged to debug. Moreover, the returned pathSubsystemMappings could
> be also logged to info.
> 3. In CGroupsHandlerImpl#parseMtab: This method iterates over lines of the
> mtab file, tries to match the line and parses data of cgroups.
> We need to better have logs for:
> - If the line does not match: Print the line (I think debug level is fine for
> that)
> - We could log the entire result map to info log.
> 4. In CGroupsHandlerImpl#initializeControllerPaths,
> initializeControllerPathsFromMtab is called. This method returns mapping of
> cgroups controllers (cpu, memory, devices, etc.) to file system paths. This
> is an important piece of information for troubleshooting, so the map needs to
> be logged to info.
> 5. In CGroupsHandlerImpl#initializeCGroupController: There's a condition for
> enableCGroupMount, I think it's worth to log if we are dealing with
> YARN-driven cgroups mount or pre-mounted controllers.
> 6. In CGroupsHandlerImpl#mountCGroupController:
> - existingMountPath needs to be logged for better troubleshooting
> - Other debug logs could also be added, this is up to the owner of this jira.
> 7. In CGroupsHandlerImpl#initializePreMountedCGroupController:
> - controllerPath needs to be logged for better troubleshooting.
> - Other debug logs could also be added, this is up to the owner of this jira.
>
> A side-note: Implementing this Jira could be way easier after YARN-9098 is
> merged as that Jira eliminated lots of code duplication and made the
> initialization of cgroups more clear and concise.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
