Thanks for the info. Looks like I'll avoid using that feature then. We
have a fairly complicated and well thought out compaction management
running but since we're moving into needing near 100% uptime we don't
have the window for tables to be tied up compacting. We're working on a
solution to chunk our data up over multiple tables so it's more
manageable. However, we'll be stuck with the large table until it done
aging off a year from now.
I'll look into the tiered compactions but we have a spatio-temporal key
(GeoMesa) and we may need to write an extension to the
DateTieredStoreEngine.
Thanks,
Austin
On 09/17/2018 07:51 PM, sudhir patil wrote:
Have you considered below options to reduce number/time of compactions?
- Reduce memstore flushes to create fewer hfiles there by reducing number
of compactions. By setting hbase.hregion.memstore.flush.size and other
parameters.
- hbase.offpeak.start & end , so that compactions triggered during off peak
time
- disabling major compaction and running manually on days with low load OR
delay major compactions using *hbase.hregion.majorcompaction*
- Checkout date tiered compaction, which doesn't compact older hfiles
https://issues.apache.org/jira/browse/HBASE-15337
On Tue, Sep 18, 2018 at 6:53 AM Ted Yu <yuzhih...@gmail.com> wrote:
Hi,
To my knowledge, stripe compaction has not seen patches for a few years.
Have you looked at :
http://hbase.apache.org/book.html#ops.date.tiered
If the above doesn't suit your needs, can you tell us more about your use
case ?
Thanks
On Mon, Sep 17, 2018 at 11:39 AM Austin Heyne <ahe...@ccri.com> wrote:
The HBase cluster we're running has well over 100TB of data spread
across two tables and as you'd guess we're suffering from compaction
times taking way to long (and we need to double or triple that volume).
I've found information on Stripe Compactions [1] which it seems like we
could benefit a lot from, however, 'experimental' is a scary word for
productions systems so I just wanted to get a general sentiment around
the stability of the feature. Any experience or input would be very
helpful.
Thanks again,
Austin
[1] https://hbase.apache.org/book.html#ops.stripe
--
Austin L. Heyne
