[jira] [Commented] (PHOENIX-5520) Phoenix-level HBase ReplicationEndpoint
[ https://issues.apache.org/jira/browse/PHOENIX-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021626#comment-17021626 ] Geoffrey Jacoby commented on PHOENIX-5520: -- [~bharathv] - it's OK if this just goes into the 4.x branches initially, and gets "forward-ported" to master later. There are already a few features like that in Phoenix because of compatibility issues like this one. > Phoenix-level HBase ReplicationEndpoint > --- > > Key: PHOENIX-5520 > URL: https://issues.apache.org/jira/browse/PHOENIX-5520 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Assignee: Bharath Vissapragada >Priority: Major > > A Phoenix implementation of HBase's ReplicationEndpoint that tails the WAL > like a normal replication endpoint. However, rather than writing to HBase's > replication sink APIs (which create HBase RPCs to a remote cluster), they > should write to a new Phoenix Endpoint coprocessor (created in a separate > sub-task). > This assumes that the WAL entries have been annotated with Phoenix metadata > (tenant, logical table/view name, timestamp) using the mechanism in > PHOENIX-5435. > While many custom ReplicationEndpoints inherit from > HBaseInterClusterReplicationEndpoint and just override the filtering logic, > this will need to avoid HBaseInterClusterReplicationEndpoint (which uses > HBase RPCs and the HBase sink manager) and instead inherit from > BaseReplicationEndpoint, or even implement the ReplicationEndpoint interface > + extend AbstractService directly. This is because it has to manage its own > transport mechanism to the remote cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (PHOENIX-5520) Phoenix-level HBase ReplicationEndpoint
[ https://issues.apache.org/jira/browse/PHOENIX-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021624#comment-17021624 ] Andrew Kyle Purtell commented on PHOENIX-5520: -- bq. I think one solution is to prototype on branch 4.15-HBase-1.5 Make a feature branch based on 4.15-HBase-1.5, do your dev there, then merge from feature branch when ready - done! > Phoenix-level HBase ReplicationEndpoint > --- > > Key: PHOENIX-5520 > URL: https://issues.apache.org/jira/browse/PHOENIX-5520 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Assignee: Bharath Vissapragada >Priority: Major > > A Phoenix implementation of HBase's ReplicationEndpoint that tails the WAL > like a normal replication endpoint. However, rather than writing to HBase's > replication sink APIs (which create HBase RPCs to a remote cluster), they > should write to a new Phoenix Endpoint coprocessor (created in a separate > sub-task). > This assumes that the WAL entries have been annotated with Phoenix metadata > (tenant, logical table/view name, timestamp) using the mechanism in > PHOENIX-5435. > While many custom ReplicationEndpoints inherit from > HBaseInterClusterReplicationEndpoint and just override the filtering logic, > this will need to avoid HBaseInterClusterReplicationEndpoint (which uses > HBase RPCs and the HBase sink manager) and instead inherit from > BaseReplicationEndpoint, or even implement the ReplicationEndpoint interface > + extend AbstractService directly. This is because it has to manage its own > transport mechanism to the remote cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (PHOENIX-5520) Phoenix-level HBase ReplicationEndpoint
[ https://issues.apache.org/jira/browse/PHOENIX-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015647#comment-17015647 ] Bharath Vissapragada commented on PHOENIX-5520: --- I think one solution is to prototype on branch 4.15-HBase-1.5 (since HBase-1.5 has all the dependencies) and then port it to master. Any better way? > Phoenix-level HBase ReplicationEndpoint > --- > > Key: PHOENIX-5520 > URL: https://issues.apache.org/jira/browse/PHOENIX-5520 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Priority: Major > > A Phoenix implementation of HBase's ReplicationEndpoint that tails the WAL > like a normal replication endpoint. However, rather than writing to HBase's > replication sink APIs (which create HBase RPCs to a remote cluster), they > should write to a new Phoenix Endpoint coprocessor (created in a separate > sub-task). > This assumes that the WAL entries have been annotated with Phoenix metadata > (tenant, logical table/view name, timestamp) using the mechanism in > PHOENIX-5435. > While many custom ReplicationEndpoints inherit from > HBaseInterClusterReplicationEndpoint and just override the filtering logic, > this will need to avoid HBaseInterClusterReplicationEndpoint (which uses > HBase RPCs and the HBase sink manager) and instead inherit from > BaseReplicationEndpoint, or even implement the ReplicationEndpoint interface > + extend AbstractService directly. This is because it has to manage its own > transport mechanism to the remote cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (PHOENIX-5520) Phoenix-level HBase ReplicationEndpoint
[ https://issues.apache.org/jira/browse/PHOENIX-5520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015634#comment-17015634 ] Bharath Vissapragada commented on PHOENIX-5520: --- I've started prototyping this and was attempting to annotate the entries and then realized that HBASE-22622 didn't make it to any HBase release in branch-2 line yet. There has been some recent discussion [1] about cutting branch-2.3 but I highly doubt if that is going to happen anytime soon. Current HBase version dependency of Phoenix core is 2.0.1. What is the ideal way to unblock ourselves here? Can the Phoenix master dev branch pull in SNAPSHOT dependencies of HBase rather than release versions? How does it work usually in cases like these? cc: [~apurtell] [1] https://lists.apache.org/thread.html/2a67c4515338d40c2b81006636d1be0596a40115a6149674bec6ebdc%40%3Cdev.hbase.apache.org%3E > Phoenix-level HBase ReplicationEndpoint > --- > > Key: PHOENIX-5520 > URL: https://issues.apache.org/jira/browse/PHOENIX-5520 > Project: Phoenix > Issue Type: Sub-task >Reporter: Geoffrey Jacoby >Priority: Major > > A Phoenix implementation of HBase's ReplicationEndpoint that tails the WAL > like a normal replication endpoint. However, rather than writing to HBase's > replication sink APIs (which create HBase RPCs to a remote cluster), they > should write to a new Phoenix Endpoint coprocessor (created in a separate > sub-task). > This assumes that the WAL entries have been annotated with Phoenix metadata > (tenant, logical table/view name, timestamp) using the mechanism in > PHOENIX-5435. > While many custom ReplicationEndpoints inherit from > HBaseInterClusterReplicationEndpoint and just override the filtering logic, > this will need to avoid HBaseInterClusterReplicationEndpoint (which uses > HBase RPCs and the HBase sink manager) and instead inherit from > BaseReplicationEndpoint, or even implement the ReplicationEndpoint interface > + extend AbstractService directly. This is because it has to manage its own > transport mechanism to the remote cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005)
