[
https://issues.apache.org/jira/browse/HBASE-19423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16281401#comment-16281401
]
Anoop Sam John commented on HBASE-19423:
----------------------------------------
My concern was on the general approach.. What the test is doing in the CP hook.
{quote}
I have a scenario where in a user table some column families are system
generated and these families data need not to be replicated.
Because these are system generated column families it would not be appropriate
to expect user to disable replication for these families.
{quote}
Well the replication enable is at CF level. One can set on HCD. By default
the replication is disabled for any CF. ie REPLICATION_SCOPE_LOCAL
One can enable this by setting REPLICATION_SCOPE_GLOBAL on those CFs which has
to be replicated. So in ur case when the table is having some of the system
generated CFs the replication is OFF on them by default.. And this is what u
need. The other CFs on the user table, which the user created, the use can opt
to set replication scope on those only which he wants.
So in general I dont see any need NOT to set the replication global scope
enabled on any of the CFs of the table and doing the selective replication of
some of the CF data using the CP hook. That is very very strange usage for me.
Hope am making it clear. Ya seems in master branch, the place where the scope
been set on the WALKey seems different so there is no reset happening. But
setting the scope on the walkey, I really doubt whether that can be a work of
the CPs. This is some thing the core code has to do. Even on master, the
WALKey is still marked @InterfaceAudience.Private. Means we really dont expect
CPs to use all methods in that. This is an issue that we over expose things to
CPs. I agree.. So pls consider ur solution 1st I would say.
> Replication entries are not filtered correctly when replication scope is set
> through WAL Co-processor
> -----------------------------------------------------------------------------------------------------
>
> Key: HBASE-19423
> URL: https://issues.apache.org/jira/browse/HBASE-19423
> Project: HBase
> Issue Type: Bug
> Reporter: Mohammad Arshad
> Labels: Replication, WAL
> Fix For: 1.4.0, 1.3.2
>
> Attachments: HBASE-19423-branch-1.3-001.patch,
> HBASE-19423-branch-1.4-001.patch, HBASE-19423-master-001-test.patch
>
>
> Replicaion scope set in WALObserver is getting reset in
> Replication.scopeWALEdits().
> Because of this problem custom implementation of WALObserver can not be used
> as a replication filter.
> Suppose WALObserver implementation has logic to filter all entries from
> family f2
> {code}
> // Filter all family f2 rows
> public static class ReplicationFilterWALCoprocessor extends BaseWALObserver
> {
> @Override
> public boolean preWALWrite(ObserverContext<? extends
> WALCoprocessorEnvironment> ctx,
> HRegionInfo info, WALKey logKey, WALEdit logEdit) throws IOException {
> ArrayList<Cell> cells = logEdit.getCells();
> for (Cell cell : cells) {
> byte[] fam = CellUtil.cloneFamily(cell);
> if ("f2".equals(Bytes.toString(fam))) {
> NavigableMap<byte[], Integer> scopes = logKey.getScopes();
> if (scopes == null) {
> logKey.setScopes(new TreeMap<byte[],
> Integer>(Bytes.BYTES_COMPARATOR));
> }
> logKey.getScopes().put(fam, HConstants.REPLICATION_SCOPE_LOCAL);
> }
> }
> return false;
> }
> }
> {code}
> This logic can not work as
> {{org.apache.hadoop.hbase.replication.regionserver.Replication.scopeWALEdits()}}
> recreates and populates scopes.
> *SOLUTION:*
> In Replication.scopeWALEdits(), create scopes map only if WALKey does not
> have it.
> {code}
> NavigableMap<byte[], Integer> scopes = logKey.getScopes();
> if (scopes == null) {
> scopes = new TreeMap<byte[], Integer>(Bytes.BYTES_COMPARATOR);
> }
> {code}
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
