Adam Antal created YARN-10406:
---------------------------------
Summary: YARN log processor
Key: YARN-10406
URL: https://issues.apache.org/jira/browse/YARN-10406
Project: Hadoop YARN
Issue Type: New Feature
Components: yarn
Reporter: Adam Antal
Assignee: Hudáky Márton Gyula
YARN currently does not have any utility that would enable cluster
administrators to re-play actions in a Hadoop YARN cluster in an offline
fashion.
HDFS has the OIV/OEV which does not require a running cluster to look and
modify the filesystem. A corresponding tool would be very helpful in the
context of YARN.
Since ATS is not widespread (is not available for older clusters) and there
isn't a single file or entity that would collect all the application/container
etc. related information, we thought our best option to parse and process the
output of the YARN daemon log files and reconstruct the history of the cluster
from that. We designed and implemented a CLI based solution that after parsing
the log file enables users to query app/container related information (listing,
filtering by certain properties) and search for common errors like CE
failures/error codes, AM preemption or stack traces. The tool can be integrated
into the YARN project as a sub-project.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
