[
https://issues.apache.org/jira/browse/HBASE-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15088075#comment-15088075
]
stack commented on HBASE-15016:
-------------------------------
Thanks for taking the time to bring me along[~eshcar]
bq. Agree?
Agree on #1 and #2.
bq. Now, the region is responsible for decisions #3 and #4.
Thinking on it, being more specific (enabled by our helpful listing out of what
is needed in your comment above), it is not the Region that is responsible but
the FlushPolicy implementation that should be responsible. It is here where we
should encapsulate all decision making about flushing (If not possible, then
there is something wrong with the FlushPolicy Interface and we should fix it).
FlushPolicies are pluggable or we could just change the FlushLargeStorePolicy
to accommodate the compacting Store.
* If all Stores in a Region are of the compacting type, just up the general
region flush threshold for the Table so the compacting Stores have more elbow
room in which to do their machinations, AND/OR
* If a Region is a mix of Store types -- compacting and old-school
non-compacting simple Store types -- then the flush policy should give the
Compacting Stores "...more memory space..." and it should be more explicit that
is doing this asking the Store its Type and acting on this attribute directly
rather than rely on a fuzzy 'active' memory used; e.g. flush a compacting Store
only if its 4 x this.flushSizeLowerBound AND we've already lushed all
old-school Stores > 1 x this.flushSizeLowerBound. You have all the context you
would need in the selectStoresToFlush method I believe (you might want to
expose more about Store internals but maybe you don't need to if compacting
Store is a configuration in HTableDescriptor or in HColumnDescriptor -- these
are available to you in the selectStoresToFlush context). If this would work
for you, then no need to change Region? Maybe I am missing something still.
Responding to your suggestion on how to bridge our approaches:
bq. ....A Region defines two thresholds and maintains a bit (flag) indicating
whether or not it has compacted memstores.
bq. Can we agree on this solution stack?
I put off agreeing to the above until there is finding that the above suggested
FlushPolicy approach won't work (FlushPolicy can ask if any of its Stores are
of the compacting type on initialization. This would be your "bit (flag)".
Regards the two thresholds up in Region, on review, this seems an indirect
means of you obtaining your needs; be explicit about what you are at inside in
the FlushPolicy selection of Stores to compact.)
Thanks [~eshcar]
> StoreServices facility in Region
> --------------------------------
>
> Key: HBASE-15016
> URL: https://issues.apache.org/jira/browse/HBASE-15016
> Project: HBase
> Issue Type: Sub-task
> Reporter: Eshcar Hillel
> Assignee: Eshcar Hillel
> Attachments: HBASE-15016-V01.patch, HBASE-15016-V02.patch,
> HBASE-15016-V03.patch, Regioncounters.pdf, suggestion.patch
>
>
> The default implementation of a memstore ensures that between two flushes the
> memstore size increases monotonically. Supporting new memstores that store
> data in different formats (specifically, compressed), or that allows to
> eliminate data redundancies in memory (e.g., via compaction), means that the
> size of the data stored in memory can decrease even between two flushes. This
> requires memstores to have access to facilities that manipulate region
> counters and synchronization.
> This subtasks introduces a new region interface -- StoreServices, through
> which store components can access these facilities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
