[
https://issues.apache.org/jira/browse/HBASE-14511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977553#comment-14977553
]
Enis Soztutar commented on HBASE-14511:
---------------------------------------
This is very good.
bq. I'd like to use this for Phoenix to store min/max for some column
qualifiers in the HFile itself. At scan time we can then efficiently rule out
entire HFiles based on those (similar to HBase does it with key ranges, and
timestamps) - that would be a cheap local secondary index. James Taylor, FYI.
Yes, this should be one of the goals.
bq. Can we make this accessible through coprocessor hooks somehow (I'd need to
think about this side, though).
We can do the co-processor way or the Plugin way. Adding these methods and
functionality to RegionObserver is definitely one option.
If we are doing the plugin way as in the patch, we should allow plugins to be
defined per table.
If we are doing the Plugin-way (rather than coprocessor), can we please pass
Context objects instead of passing the parameters directly (Configuration,
Writer, etc). This becomes important since we do not want to break phoenix and
other third party plugins if we decide to pass new parameters in the future. An
example of this pattern is:
{code}
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.REPLICATION)
public interface ReplicationEndpoint extends Service {
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.REPLICATION)
class Context {
...
}
void init(Context context) throws IOException;
@InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.REPLICATION)
static class ReplicateContext {
..
}
boolean replicate(ReplicateContext replicateContext);
}
{code}
Why {{MetaWriter}} and {{Plugin}} are different classes? We can just have a
single Plugin class, no?
It is a nit, but HBase convention has the opening curly bracket "{" in the same
line for class and method definitions.
> StoreFile.Writer Meta Plugin
> ----------------------------
>
> Key: HBASE-14511
> URL: https://issues.apache.org/jira/browse/HBASE-14511
> Project: HBase
> Issue Type: New Feature
> Reporter: Vladimir Rodionov
> Assignee: Vladimir Rodionov
> Attachments: HBASE-14511-v3.patch, HBASE-14511.v1.patch,
> HBASE-14511.v2.patch
>
>
> During my work on a new compaction policies (HBASE-14468, HBASE-14477) I had
> to modify the existing code of a StoreFile.Writer to add additional meta-info
> required by these new policies. I think that it should be done by means of a
> new Plugin framework, because this seems to be a general capability/feature.
> As a future enhancement this can become a part of a more general
> StoreFileWriter/Reader plugin architecture. But I need only Meta section of a
> store file.
> This could be used, for example, to collect rowkeys distribution information
> during hfile creation. This info can be used later to find the optimal region
> split key or to create optimal set of sub-regions for M/R jobs or other jobs
> which can operate on a sub-region level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
