[
https://issues.apache.org/jira/browse/HADOOP-6121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726573#action_12726573
]
Scott Carey commented on HADOOP-6121:
-------------------------------------
{quote}Do you know who is implementing ZK for the NameNode?{quote}
I don't even know if such a thing is happening, but given that Google's
MapReduce paper and BigTable paper indicate that Chubby is leveraged for HA on
various hadoop equivalents I figured that was part of the discussion.
I am in the process of using ZK for an internal project that has some
similarities with the JT. Previously it was using ping-response for both state
communication and triggering (distributed) events. Switching to ZK proved to
be a bit outside the bounds of its typical use case but is very fast, reliable,
and light-weight. State management and persistence is much easier, and
communication with watches very reliable and fast.
Anyhow, its an option. The reliability, availability, and time sensitivity
issues it addresses. Other special aspects of the JT use case would probably
drive some ZK features -- like an api to return only 'new' children of a node
after a specific transaction id, easier out-of-the-box embedding, and some
higher level APIs for read-only/watch-only use cases.
> High Availability support for Hadoop
> ------------------------------------
>
> Key: HADOOP-6121
> URL: https://issues.apache.org/jira/browse/HADOOP-6121
> Project: Hadoop Common
> Issue Type: New Feature
> Components: dfs, mapred
> Reporter: Jie Qiu
>
> Currently, We look at the HA of Hadoop cluster. We need to consider the
> NameNode HA as well as Jobtracker HA. For NameNode, we want to build
> primary/standy or master-slaves pattern to provide NameNode HA. Therefore, we
> need to consider how to ship log between primary/standby/slaves and how
> commit "write" operation to NameNode after the agreement among
> primary/standby/slaves on log. Whether will we use Linux HA package or
> NameNode-built-in HA package without the help of outter Linux HA package.
> After NameNode become high availability, is it necessary to provide HA for
> Jobtracker? Can Jobtracker persist the states of Jobs and tasks into HA
> NameNode? Or Jobtracker also needs the same approach from NameNode for HA
> support.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.