[
https://issues.apache.org/jira/browse/HBASE-19494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297160#comment-16297160
]
wolfgang hoschek commented on HBASE-19494:
------------------------------------------
bq. First cut. I don't think it enough. I think hbase-indexer is going to want
some context passed
Yes, indeed. I don't think your proposed patch works. Instead, my idea (as I've
already outlined above in
https://issues.apache.org/jira/browse/HBASE-19494?focusedCommentId=16292858&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16292858)
is for ReplicationSink. replicateEntries() to delegate the filter call to the
Connection (if it implements that new filter interface method), which can
maintain state on a per table basis, like so:
{code}
Connection conn = getConnection();
if (conn instanceof WALEntrySinkFilter) { // this is a new Interface
if (((WALEntrySinkFilter)conn).filter(table,
entry.getKey().getWriteTime())) {
continue; // ignore unwanted entry
}
{code}
The connection can maintain state on a per table basis, and connection can look
up the timestamp thresholds via the table name (again as already outlined above
in
https://issues.apache.org/jira/browse/HBASE-19494?focusedCommentId=16292858&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16292858)
like so:
{code}
public class IndexerConnection implements Connection, WALEntryFilter {
private Map<String, Long> timestampThresholds = ...
public boolean filter(TableName table, long writeTime) {
long timestampThreshold =
timestampThresholds.get(table.getNameAsString());
return writeTime < timestampThreshold;
}
... and so on
}
{code}
> Create simple WALKey filter that can be plugged in on the Replication Sink
> --------------------------------------------------------------------------
>
> Key: HBASE-19494
> URL: https://issues.apache.org/jira/browse/HBASE-19494
> Project: HBase
> Issue Type: Sub-task
> Components: Replication
> Reporter: stack
> Assignee: stack
> Fix For: 2.0.0-beta-1
>
> Attachments:
> 0003-HBASE-19494-Create-simple-WALKey-filter-that-can-be-.patch
>
>
> hbase-indexer used to look at WALKeys on the sink to see if their time of
> creation was before the time at which the replication stream was enabled.
> In the parent redo, there is no means for doing this anymore (because WALKey
> used to be Private and because to get at the WALKey in the Sink, you had to
> override all of the Replication which meant importing a million Private
> objects...).
> This issue is about adding a simple filter to Replication on the sink-side
> that just takes a WALKey (now InterfaceAudience LimitedPrivate and recently
> made read-only).
> Assigned myself. Need to do this so hbase-indexer can move to hbase2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
