Varun Vasudev updated YARN-3294:
    Attachment: apache-yarn-3294.2.patch

Thanks for the review Jian!

1. will it dump the logs for package 
“org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity” or 
“org.apache.hadoop.yarn.server.resourcemanager.scheduler”. I think it should be 
dumping latter as some scheduler logs sit in common

Log log = LogFactory.getLog(CapacityScheduler.class.getPackage().getName());

2. the newly added code can be in a separate class, maybe call it 

3. Overall, do you think it's useful to make this as a generic log dumper ? 
i.e. parameterize the class name of the log to be dumped.


Addressed all three. I made it a standalone class called AdHocLogDumper that 
takes the log name and dumps the log. It now dumps the log for 

    private Logger packageLogger; does not need to be a global value


    dumping logs is useful for large amount of logs. Do you think it's also 
useful to surface the logs on the web page.

I think we should skip it for now. On a single node cluster with one pi job 
running, it dumped about 200K of logs in 1 min. I'm worried with larger 
clusters, the log size will be too large. The log will still be accessible via 
the /logs link.

> Allow dumping of Capacity Scheduler debug logs via web UI for a fixed time 
> period
> ---------------------------------------------------------------------------------
>                 Key: YARN-3294
>                 URL: https://issues.apache.org/jira/browse/YARN-3294
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: Screen Shot 2015-03-12 at 8.51.25 PM.png, 
> apache-yarn-3294.0.patch, apache-yarn-3294.1.patch, apache-yarn-3294.2.patch
> It would be nice to have a button on the web UI that would allow dumping of 
> debug logs for just the capacity scheduler for a fixed period of time(1 min, 
> 5 min or so) in a separate log file. It would be useful when debugging 
> scheduler behavior without affecting the rest of the resourcemanager.

This message was sent by Atlassian JIRA

Reply via email to