Szilard Nemeth created YARN-9371:
------------------------------------
Summary: Better logging for initialization of cgroups in
CGroupsHandlerImpl
Key: YARN-9371
URL: https://issues.apache.org/jira/browse/YARN-9371
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Szilard Nemeth
We already have YARN-9098 that dealt with (not yet merged, though) code
duplication in cgroups setup code.
However, there's still room for improvement in the area of cgroups
initialization, especially in CGroupsHandlerImpl.
1.
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.resources.CGroupsHandlerImpl#initializeControllerPaths:
There's a decision in this method: Either cgroups are parsed from the
configured mount path or from the mounted cgroups descriptor file
(/proc/mounts). This decisions needs to be logged, along with some details,
e.g. cgroups mount path.
2. In ResourceHandlerModule#parseConfiguredCGroupPath: Every candidate file
could be logged to debug. Moreover, the returned pathSubsystemMappings could be
also logged to info.
3. In CGroupsHandlerImpl#parseMtab: This method iterates over lines of the mtab
file, tries to match the line and parses data of cgroups.
We need to better have logs for:
- If the line does not match: Print the line (I think debug level is fine for
that)
- We could log the entire result map to info log.
4. In CGroupsHandlerImpl#initializeControllerPaths,
initializeControllerPathsFromMtab is called. This method returns mapping of
cgroups controllers (cpu, memory, devices, etc.) to file system paths. This is
an important piece of information for troubleshooting, so the map needs to be
logged to info.
5. In CGroupsHandlerImpl#initializeCGroupController: There's a condition for
enableCGroupMount, I think it's worth to log if we are dealing with YARN-driven
cgroups mount or pre-mounted controllers.
6. In CGroupsHandlerImpl#mountCGroupController:
- existingMountPath needs to be logged for better troubleshooting
- Other debug logs could also be added, this is up to the owner of this jira.
7. In CGroupsHandlerImpl#initializePreMountedCGroupController:
- controllerPath needs to be logged for better troubleshooting.
- Other debug logs could also be added, this is up to the owner of this jira.
A side-note: Implementing this Jira could be way easier after YARN-9098 is
merged as that Jira eliminated lots of code duplication and made the
initialization of cgroups more clear and concise.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
