[Rumen] Parallelize TraceBuilder
--------------------------------
Key: MAPREDUCE-2010
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2010
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: tools/rumen
Affects Versions: 0.22.0
Reporter: Amar Kamat
Assignee: Amar Kamat
Fix For: 0.22.0
Currently, Rumen's {{TraceBuilder}} processes jobs in sequential manner and
emits them in sorted order (based on job-id). Following are the steps :
# Read data from input files
# Parse and analyze the JobHistory data
# Write the data to the output file
Steps #1 and #2 can be done in parallel. Step #3 can be made sequential (if
user needs it) else can also be done in parallel.
I could achieve ~50% speedup by simply parallelizing step#1 and step#2 (i.e
output was sorted based on job-id).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
