[
https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233680#comment-14233680
]
Gaurav Menghani commented on HBASE-12476:
-----------------------------------------
Stack, sorry about this. Didn't see your comment during the holidays.
{quote}
Purge versions from sub-poms if the dependency is common with other modules.
Inherit from parent where you can. I found this in the consensus pom:
<version>2.15</version>
and
<version>4.1</version>
... for what I believe are common dependencies (Its fine to have dependencies
in modules with versions if the dependency only used by the submodule)
{quote}
Sure, will do.
{quote}
In the consensus module, there is an hbase-default.xml also? How does it relate
to the one over in hbase-common?
You can purge hadoop-1 references (unless that is how you refer to your hdfs.)
We don't support it in hbase 1.0.
Others have already remarked on undoing the local copies of HColumnDescriptor,
etc. You had them in here so this module could 'work' without dependency? (Kill
HServerAddress!) Do we need to move stuff around in hbase so you can have
minimal dependencies? All the classes you have copied local here, should we
move them back up and into hbase-common module if not there already?
{quote}
I didn't find hbase-default.xml in the consensus package. Yes, we are cleaning
up all that dependency cruft in HBASE-12510.
{quote}
Run the rat tool on your feature branch. Some of these files are still missing
licenses.
{quote}
Yes, we didn't add licenses to mostly any of the code. Will do this.
{quote}
Nice. You have already ported over ConfigurationObserver. Good.
{quote}
Hehe, I wrote the original ConfigurationObserver in 0.89-fb :D
{quote}
We'll have to redo FetchTask, etc., as protobuf? (the swifty annotations look
very nice)
In FetchTask I see in the run that it has this:
// TODO in part 2: fetch log files from the peer!
Does that mean this facility of raft – catchup I presume – is yet to be
implemented or is it rather that this class gets subclassed later?
If the former, is there more to do to have full implementation?
{quote}
As per Rishit, the FetchTask is for advanced offline log recovery. We aren't
using that part yet. We can add more comments to highlight that.
{quote}
Each regionserver stands up an instance of a QuorumClient per region?
{quote}
Yes, a QuorumClient is used to replicate edits. It is the most clear usage of
the protocol inside RS.
{quote}
A quorum name is different from region name?
{quote}
It need not be. Because the consensus module is intended to be independent of
WAL, and even HBase (which it is not right now, with all the dependencies). So
today the quorum is being done on the WAL, so the quorum name is the same as
region name. Tomorrow, we can do a quorum on something else, in which case
quorum name would be something else. It is just a unique identifier for the
quorum.
{quote}
We'd have to move all of the quorum communication up on to hbase rpc and pb?
What you lot think? Keep swift?
{quote}
I would prefer swift, because we don't have to write IDL specs for the service
and data-structs. That said, if it doesn't play well with the bigger project,
we should discuss this in detail.
{quote}
The FSM stuff could do w/ a bit of package doc on how it works. That said, it
is easy to read. (Maybe we could use these primitives elsewhere in hbase)
{quote}
What do you mean by a package doc? Is it synonymous to Javadocs?
{quote}
Region splits handled?
{quote}
Hydrabase cannot handle region splits as of now.
{quote}
We'd have to figure how to do the RMap stuff in master? And bootstrapping the
cluster?
{quote}
Regarding Bootstrapping, we can share some ideas. There are some basic tools to
bootstrap the RMap. I am moving the RMap part to hbase-server in HBASE-12510,
since this is part of the integration of HBase + Protocol. Regarding the master
part, there is a diff coming out soon. We worked on something known as
HydrabaseMaster which is a thin HMaster which alters the RMap as nodes die and
join the quorum.
{quote}
A regionserver would have to do consensuservice?
{quote}
That's correct. Coming up soon.
{quote}
WHen you say 'favored_nodes' in the rmap, does that imply we need 'favored
nodes' working out in apache hbase or is this just how you describe quorum
members for a region?
{quote}
Actually favored_nodes is used for hinting the DFS where to replicate the
blocks. I believe we don't have that ability in open source HDFS right? In the
next diff, we are just ignoring the favored nodes part.
{quote}
Is the read-side, reading from the quorum, a patch as big as this one (smile)?
Nice one.
{quote}
Not quite, I think the patch sizes would go down exponentially from here. This
probably was the largest chunk of code.
> HydraBase Consensus Protocol
> ----------------------------
>
> Key: HBASE-12476
> URL: https://issues.apache.org/jira/browse/HBASE-12476
> Project: HBase
> Issue Type: Sub-task
> Components: Consensus, wal
> Reporter: Gaurav Menghani
> Assignee: Gaurav Menghani
> Fix For: HBASE-12259
>
> Attachments: 0001-HydraBase-consensus-protocol.patch,
> 0002-HydraBase-consensus-protocol.patch
>
>
> This is the first patch for the HydraBase consensus protocol implemented
> according to the Raft consensus protocol
> (https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
