[
https://issues.apache.org/jira/browse/HBASE-2195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098616#comment-13098616
]
[email protected] commented on HBASE-2195:
------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1730/#review1780
-----------------------------------------------------------
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
<https://reviews.apache.org/r/1730/#comment4085>
This is good.
One thought is if these new constants -- the key name and the default
cluster id -- belong in the general HConstants. Are they used across packages?
If so, then this is the place for them. Otherwise, I'd say put them in
replication and then they'll have the replication class prefix when referred
too which will give them some added context. No biggie. I'm just reacting
against this fat HConstants.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java
<https://reviews.apache.org/r/1730/#comment4086>
You can't do much better than this if its UUIDs we are carrying.
Is this code repeated in the Put? If so, might want to factor it out?
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java
<https://reviews.apache.org/r/1730/#comment4087>
Ditto on if repeating code.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Put.java
<https://reviews.apache.org/r/1730/#comment4088>
Hmm.. there is an fb patch that gives Get/Delete, etc. a common heritage.
Let me dig it up and commit so you can put this common code there.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
<https://reviews.apache.org/r/1730/#comment4089>
Is this needed?
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
<https://reviews.apache.org/r/1730/#comment4090>
What is happening here Lars? Are we passing the clusterid from the
HRegionServer down to the WAL by writing it into the Delete or Put? Do we have
to do that? Can we not pass the clusterid to the WAL directly when
HRegionServer creates one?
On above, nvm, I think I figure out why you do it this way later.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
<https://reviews.apache.org/r/1730/#comment4091>
Yeah, can't we pass the cluserid in on the HLog constructor rather than per
append or do we need to allow for different clusterids coming in via append?
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
<https://reviews.apache.org/r/1730/#comment4092>
Yeah, if the clusterids are different, i'd think the edits are different?
That looks like oversight in original code.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
<https://reviews.apache.org/r/1730/#comment4093>
We could have read garbage that happened to be <0? But yeah, this is going
to go out with new major version and we'll just pronounce that it will not be
able to read old WALs (this we've said in the past)
Why not VersionedWritable since it does this version stuff for you?
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
<https://reviews.apache.org/r/1730/#comment4094>
Why double read of vint? Is this how you do vints?
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java
<https://reviews.apache.org/r/1730/#comment4095>
How is this going to be done now? Where do we get cluserid?
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
<https://reviews.apache.org/r/1730/#comment4096>
Hmm... oh, I see. The edit could have come in from other than
HRegionServer receiving from client on local cluster. OK.
- Michael
On 2011-09-06 23:16:48, Lars Hofhansl wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1730/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-06 23:16:48)
bq.
bq.
bq. Review request for hbase, Ted Yu, Michael Stack, and Jean-Daniel Cryans.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. For Master <-> Master replication (as well as cycles > 2) a replication
sink needs to keep track who is was the originator of a change and
bq. (1) do not replicate this data back to the originator and
bq. (2) pass that information on to the next link in a cycle > 2
bq.
bq. In order to do that, HlogKeys read from the WAL are tagged with the
Cluster UUID at the ReplicationSource before the logs are shipped to the Sink.
The sink writes the Cluster UUID into the WAL log (in HlogKey, so only once per
edit). If the sink is also source for yet another sink the ClusterUUID is
retained in the HLogKeys read from the local WAL and passed on to the sink.
bq.
bq.
bq. This addresses bug HBASE-2195.
bq. https://issues.apache.org/jira/browse/HBASE-2195
bq.
bq.
bq. Diffs
bq. -----
bq.
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSink.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/HConstants.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Put.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogKey.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/replication/ReplicationZookeeper.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java
1165864
bq.
http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/replication/TestMasterReplication.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/1730/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. All tests pass.
bq. New test: TestMasterReplication (which tests Master <-> Master, but no
cycles > 2, yet), also passes.
bq. Manually tested a cycle between 3 clusters.
bq.
bq.
bq. Thanks,
bq.
bq. Lars
bq.
bq.
> Support cyclic replication
> --------------------------
>
> Key: HBASE-2195
> URL: https://issues.apache.org/jira/browse/HBASE-2195
> Project: HBase
> Issue Type: Sub-task
> Components: replication
> Reporter: Jean-Daniel Cryans
> Assignee: Lars Hofhansl
> Attachments: 2195-v5.txt, 2195-v6.txt, 2195.txt
>
>
> We need to support cyclic replication by using the cluster id of each HlogKey
> and stop replicating when it goes back to the original cluster.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
