[
https://issues.apache.org/jira/browse/HBASE-10504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13998272#comment-13998272
]
Enis Soztutar commented on HBASE-10504:
---------------------------------------
bq. the number of replication endpoints becomes directly linked to the number
of regionservers – as far as I see, it would no longer be possible to have more
or fewer replication endpoints than regionservers
As you said, if you want to still have a proxy layer to have a different
configuration then you can implement your own server + client layer and plug
only the client in. Run the servers in whatever conf / jvm you want.
bq. the replication (ReplicationConsumer) runs within the same JVM as the
regionserver, which brings some important changes: all of the dependencies then
need to be present in the classpath of the regionserver (and compatible with
the dependencies of the regionserver), it has an effect on the heap, is no
longer possible to tune the JVM settings for the replication process on its own
Same as above, you can have a thin client layer and tick server layer on your
own if you prefer. Otherwise, this will simplify deployments a lot.
bq. from what I see of the current patch (although I'm sure this could be
changed) is that it looks like there's only a single ReplicationConsumer,
meaning that it wouldn't be possible to have "real" replication running next to
a custom ReplicationSource
It is a single ReplicationConsumer per peer. HBaseClusterReplicator implements
this interface. You can have one peer definition replicating to a different
cluster and a peer replicating to a third party.
bq. The tighter-coupling issues that I outlined above could be worked around by
having a lightweight ReplicationConsumer that just passes data to an external
process via RPC, but that's basically exactly what the replication code in
HBase is doing right now, so it would be a shame to re-implement something like
that.
Even in that case, the server-client implementation will be much simpler than
what you have in SEP I think, which does hbase RPC + zookeeper + zk mockup.
Only a server discovery + simple one method thrift / PB RPC is what is needed
from my understanding. It would be great to have a skeleton for this living in
HBase proper as well.
> Define Replication Interface
> ----------------------------
>
> Key: HBASE-10504
> URL: https://issues.apache.org/jira/browse/HBASE-10504
> Project: HBase
> Issue Type: Task
> Reporter: stack
> Assignee: stack
> Priority: Blocker
> Fix For: 0.99.0
>
> Attachments: hbase-10504_wip1.patch
>
>
> HBase has replication. Fellas have been hijacking the replication apis to do
> all kinds of perverse stuff like indexing hbase content (hbase-indexer
> https://github.com/NGDATA/hbase-indexer) and our [~toffer] just showed up w/
> overrides that replicate via an alternate channel (over a secure thrift
> channel between dcs over on HBASE-9360). This issue is about surfacing these
> APIs as public with guarantees to downstreamers similar to those we have on
> our public client-facing APIs (and so we don't break them for downstreamers).
> Any input [~phunt] or [~gabriel.reid] or [~toffer]?
> Thanks.
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)
