[
https://issues.apache.org/jira/browse/HBASE-22622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872614#comment-16872614
]
Andrew Purtell edited comment on HBASE-22622 at 6/25/19 6:48 PM:
-----------------------------------------------------------------
In order to process a WALentry generated by Phoenix we need have a unique
schema key that precisely identifies the Phoenix schema used to generate the
edits, a tuple of [ tenantId, schemaName, schemaTimestamp ] . This isn't
something that HBase should care about, and we are not asking for this. We are
only asking for the ability to attach arbitrary attributes to the WALkey so we
can take care of these details in the Phoenix layer.
We want to use this information in the replication context, in a custom
replication endpoint.
Bear in mind this should work at both source and sink side in the local
regionserver process (where custom replication endpoint is installed at source)
and a remote sink process (where custom regionserver-like process received the
replication stream, this is the Lily hbase-sep "fake regionserver" model). So
it's not sufficient to have an in-memory only decorator for WALKey, the
attributes should be serialized. Fortunately we can do this in a wire
compatible way because WALKey is already protobuf serialized.
was (Author: apurtell):
In order to process a WALentry generated by Phoenix we need have a unique
schema key that precisely identifies the Phoenix schema used to generate the
edits, a tuple of [ tenantId, schemaName, schemaTimestamp ] . This isn't
something that HBase should care about, and we are not asking for this. We are
only asking for the ability to attach arbitrary attributes to the WALkey so we
can take care of these details in the Phoenix layer.
Bear in mind this should work at both source and sink side in the local
regionserver process (where custom replication endpoint is installed at source)
and a remote sink process (where custom regionserver-like process received the
replication stream, this is the Lily hbase-sep "fake regionserver" model). So
it's not sufficient to have an in-memory only decorator for WALKey, the
attributes should be serialized. Fortunately we can do this in a wire
compatible way because WALKey is already protobuf serialized.
> WALKey Extended Attributes
> --------------------------
>
> Key: HBASE-22622
> URL: https://issues.apache.org/jira/browse/HBASE-22622
> Project: HBase
> Issue Type: New Feature
> Components: wal
> Reporter: Geoffrey Jacoby
> Assignee: Geoffrey Jacoby
> Priority: Major
>
> It would be useful if the WAL protobuf and WALKey class included an optional
> map of extended key/value attributes that downstream coprocessors could use
> to annotate WAL Entries. While standard HBase replication would not use them,
> custom replication endpoints could use the data to make filtering decisions
> or take actions.
> An example use case would be allowing a tool like Phoenix to annotate
> WAL.Entries to indicate that a given Entry is associated with a particular
> Phoenix view rather than the base Phoenix table. (Multiple logical views in
> Phoenix can map to the same physical HBase table.) A custom replication
> endpoint might choose to replicate some views but not others.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
