[
https://issues.apache.org/jira/browse/MAPREDUCE-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
vitthal (Suhas) Gogate updated MAPREDUCE-3202:
----------------------------------------------
Description: Hadoop Vaidya provides a detailed analysis of the M/R job in
terms of various execution inefficiencies and the associated remedies that user
can easily understand and fix. This Jira patch integrates it with Job History
UI under Hadoop 2.0 branch. (was: At present jobdetailshistory page served by
JobHistory Server provides elementary job analysis through link "Analyze This
job". Hadoop Vaidya provides a detailed analysis of the M/R job in terms of
various execution inefficiencies and the associated remedies that user can
easily understand and fix. Integrating Hadoop Vaidya with JobHistory server
would really improve the usability of this tool and also benefit many naive
users understanding various performance problems and/or best practices
violations associated with their job.
Integration would also aim at providing users a convenient interface where they
can manage the existing rules as well as write their own new rules.
During my tenure at Yahoo, Vaidya tool was successfully deployed in production
analyzing tens of thousands of jobs every day with lot more useful rules than
the sample ones present in the contrib project. Many of these rules are open
sourced already (big thanks to Yahoo! MAPREDUCE-1530) but yet to integrate with
the tool.
I will add more design details for this feature in near future as work towards
getting prototype running.. Any thoughts/comments are welcome. )
> Integrating Hadoop Vaidya with Job History UI in Hadoop 2.0
> ------------------------------------------------------------
>
> Key: MAPREDUCE-3202
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3202
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: jobhistoryserver
> Affects Versions: 0.20.205.0, 1.0.0
> Reporter: vitthal (Suhas) Gogate
>
> Hadoop Vaidya provides a detailed analysis of the M/R job in terms of various
> execution inefficiencies and the associated remedies that user can easily
> understand and fix. This Jira patch integrates it with Job History UI under
> Hadoop 2.0 branch.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
