[
https://issues.apache.org/jira/browse/MAPREDUCE-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12734542#action_12734542
]
Sharad Agarwal commented on MAPREDUCE-786:
------------------------------------------
I think we should cleanup the way history is written. Providing cleaner API to
plugin the async. Perhaps a separate JIRA for it. Something like:
{code}
public interface Event {
public Map<String, String> getData();
}
public interface EventReader {
public Iterator<Event> read();
public void close();
}
public interface EventWriter {
public void write(Event event);
public void flush();
public void close();
}
{code}
Async writer should be able to wrap to an EventWriter object:
{code}
class AsyncEventWriter implements EventWriter {
AsyncEventWriter(EventWriter writer){
}
}
{code}
> Jobtracker history should be written aysnchronously to the filesystem
> ---------------------------------------------------------------------
>
> Key: MAPREDUCE-786
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-786
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobtracker
> Reporter: Sharad Agarwal
> Assignee: Sharad Agarwal
>
> Jobtracker lock is held while writing the history events. This makes the
> jobtracker slow on flushes, especially when history is written to HDFS.
> History events should be written asynchronously to avoid this problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
