[
https://issues.apache.org/jira/browse/HADOOP-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615158#action_12615158
]
dhruba borthakur commented on HADOOP-3245:
------------------------------------------
+1 to Owen's comment.
A pressing need of our cluster is to not interrupt running jobs if the
jobtracker has to be restarted. This means that job states have to be
persisted in the form of a transaction log. This requirement is all the more
beneficial to sites that have long-running job trackers (instead of HOD).
However, isn't it better to be able to store state in HDFS? It is true that
HDFS stores its transaction log in local files, but with the current focus on
improving HDFS read/write latencies, HDFS itself is considering whether to
store one copy of the transaction log in HDFS blocks (instead of NFS). In fact,
if the JobTracker stores information in a org.apache.hadoop.fs.FileSystem, then
a typical customer install could plug in various forms of storage to support
the JobTracker transaction log.
> Provide ability to persist running jobs (extend HADOOP-1876)
> ------------------------------------------------------------
>
> Key: HADOOP-3245
> URL: https://issues.apache.org/jira/browse/HADOOP-3245
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Amar Kamat
> Attachments: HADOOP-3245-v2.5.patch, HADOOP-3245-v2.6.5.patch,
> HADOOP-3245-v2.6.9.patch, HADOOP-3245-v4.1.patch
>
>
> This could probably extend the work done in HADOOP-1876. This feature can be
> applied for things like jobs being able to survive jobtracker restarts.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
