[
https://issues.apache.org/jira/browse/MAPREDUCE-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Kanter updated MAPREDUCE-6415:
-------------------------------------
Attachment: MAPREDUCE-6415_branch-2.001.patch
MAPREDUCE-6415.001.patch
MAPREDUCE-6415.001.patch and MAPREDUCE-6415_branch-2.001.patch contain the
MapReduce changes, though most of it's actually under hadoop-tools. This
includes all of the code to find and process the aggregated log files into HAR
files. It's mostly the same as the prelim patch, with some minor changes and
unit tests. I've uploaded the YARN changes to YARN-4086. The patches for this
and YARN-4086 can be applied independently.
> Create a tool to combine aggregated logs into HAR files
> -------------------------------------------------------
>
> Key: MAPREDUCE-6415
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6415
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.8.0
> Reporter: Robert Kanter
> Assignee: Robert Kanter
> Attachments: HAR-ableAggregatedLogs_v1.pdf, MAPREDUCE-6415.001.patch,
> MAPREDUCE-6415_branch-2.001.patch, MAPREDUCE-6415_branch-2_prelim_001.patch,
> MAPREDUCE-6415_branch-2_prelim_002.patch, MAPREDUCE-6415_prelim_001.patch,
> MAPREDUCE-6415_prelim_002.patch
>
>
> While we wait for YARN-2942 to become viable, it would still be great to
> improve the aggregated logs problem. We can write a tool that combines
> aggregated log files into a single HAR file per application, which should
> solve the too many files and too many blocks problems. See the design
> document for details.
> See YARN-2942 for more context.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
