[
https://issues.apache.org/jira/browse/HBASE-13159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351410#comment-14351410
]
stack commented on HBASE-13159:
-------------------------------
Tell us more about use cases. Give us example transforms (other than identity
and 'drop'... or is that it?) Regards transforms, ain't that just filter at
HFile level?
How would you get the 'view' up to high level when transform is at the HFile
level?
bq. ...and can be referred from multiple regions
Always shied from supporting this. Was afraid of the complexity and dangling
pointers.
I suppose if we had RangeReferenceFiles, could help at log split time. Same
count of files so same NN traffic... we'd just be reading from one file. With
current small files, you can remove them when done but with
RangeReferenceFiles, you'd need to do refcounting... in meta?
RangeReferenceFiles would not save NN traffic since we have to drop them in the
FS but could save i/o not replicating Cells.
@enis The index would be persisted in HFile or to a separate
RangeReferenceFile? The benefit is smaller indices? Transform would add prefix
back? Will that work? (maybe I'm not following)
> Consider RangeReferenceFiles with transformations
> -------------------------------------------------
>
> Key: HBASE-13159
> URL: https://issues.apache.org/jira/browse/HBASE-13159
> Project: HBase
> Issue Type: Brainstorming
> Reporter: Lars Hofhansl
>
> Currently we have References used by HalfStoreReaders and HFileLinks.
> For various use cases we have here we have need for a RangeReferences with
> simple transformation of the keys.
> That would allow us to map HFiles between regions or even tables without
> copying any data.
> We can probably combine HalfStores, HFileLinks, and RangeReferences into a
> single concept:
> * RangeReference = arbitrary start and stop row, arbitrary key transformation
> * HFileLink = start and stop keys set to the linked file's start/stop key,
> transformation = identity
> * (HalfStore) References = start/stop key set according to top or bottom
> reference, transformation = identity
> Note this is a *brainstorming* issue. :)
> (Could start with just references with arbitrary start/stop keys, and do
> transformations later)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
