[
https://issues.apache.org/jira/browse/HBASE-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086587#comment-14086587
]
stack commented on HBASE-11208:
-------------------------------
Other problems with this config as it currently works:
+ Stops the write dead in its tracks. Write throughput goes to zero. Looks
like a bug to the innocent user.
+ It makes the current 'live' client suffer so a background process does not
get 'overwhelmed'. Its like walking into a store, getting up ot the counter,
and then the salesperson answers the phone and takes care of the caller's query
ahead of yours.
Removing this config ameliorates rather than aggravates memory pressure since
we are letting the flush proceed.
On other hand hundreds of hfiles under a column family (we've seen thousands
IIRC) is a bit of a perf killer (smile) and difficult to recover from.
Making writes second class to reads is a legit use case but think we need a
less coarse means of blocking than count of hfiles (if a multiput, we block the
whole multiput succeeding rather than just the puts to this overwhelmed cf?)
> Remove the hbase.hstore.blockingStoreFiles setting
> --------------------------------------------------
>
> Key: HBASE-11208
> URL: https://issues.apache.org/jira/browse/HBASE-11208
> Project: HBase
> Issue Type: Brainstorming
> Components: Compaction, regionserver
> Affects Versions: 0.99.0
> Reporter: Nicolas Liochon
> Assignee: Nicolas Liochon
> Fix For: 0.99.0
>
>
> It's a little bit of a provocation, but the rational is:
> - there are some bugs around the delayed flush. For example, if the periodic
> scheduler has asked for a delayed flush, and that we need to flush, we will
> have to wait
> - if the number of WAL files increases, we won't flush immediately if the
> blockingFile number has been reached. This impacts the MTTR.
> - We don't write to limit the compaction impact, but they are many cases
> where we would want to flush anyway, as the writes cannot wait.
> - this obviously leads to huge write latency peaks.
> So I'm questioning this setting, it leads to multiple intricate cases,
> unpredictable write latency, and looks like a workaround for compaction
> performances. With all the work done on compaction, I think we can get rid of
> it. A solution in the middle would be to deprecate it and to set it to a
> large value...
> Any opinion before I shoot :-) ?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
