Varun Vasudev commented on YARN-2901:

Some more information on the patch - it adds a new log appender which stores a 
count of the errors and warnings logged with associated time period 
information. It also stores the last 500 unique error and the last 500 unique 
warning messages as well as the number of times and the time when they 
occurred. Messages more than 24 hours old are purged automatically from the 
store. Right now the appender is added in code since log4j.properties is shared 
across hadoop but if it's useful, we could add it to log4j.properties itself.

> Add errors and warning stats to RM, NM web UI
> ---------------------------------------------
>                 Key: YARN-2901
>                 URL: https://issues.apache.org/jira/browse/YARN-2901
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: Screen Shot 2015-03-19 at 7.40.02 PM.png, 
> apache-yarn-2901.0.patch
> It would be really useful to have statistics on the number of errors and 
> warnings in the RM and NM web UI. I'm thinking about -
> 1. The number of errors and warnings in the past 5 min/1 hour/12 hours/day
> 2. The top 'n'(20?) most common exceptions in the past 5 min/1 hour/12 
> hours/day
> By errors and warnings I'm referring to the log level.
> I suspect we can probably achieve this by writing a custom appender?(I'm open 
> to suggestions on alternate mechanisms for implementing this).

This message was sent by Atlassian JIRA

Reply via email to