[
https://issues.apache.org/jira/browse/HBASE-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191008#comment-14191008
]
stack commented on HBASE-10201:
-------------------------------
I wrote the list to get another opinion on the patch pre-commit.
Has this patch been deployed somewhere in production (smile?). If so, would be
good to know. In production, it helps?
On rereview:
" This value should be less than half of the total memstore
+ threshold (hbase.hregion.memstore.flush.size)."
Do we ensure this in code? If not, should we?
bq. I think it is better to open another issue to handle the duplication.
Can you do this for the accounting fixup so by-Store in HLog.
Should log when we do this:
+ long columnfamilyFlushSize = this.htableDescriptor
+ .getMemStoreColumnFamilyFlushSize();
+ if (columnfamilyFlushSize <= 0) {
+ columnfamilyFlushSize = conf.getLong(
+ HConstants.HREGION_MEMSTORE_COLUMNFAMILY_FLUSH_SIZE_LOWER_BOUND,
+
HTableDescriptor.DEFAULT_MEMSTORE_COLUMNFAMILY_FLUSH_SIZE_LOWER_BOUND);
I can add on commit unless we are doing a new version.
This does not have to be public since it is used from same package:
+ public long getEarliestFlushTimeForAllStores() {
ditto this
getLatestFlushTimeForAllStores
And this ... isPerColumnFamilyFlushEnabled
nit: Guard debug logging with an if LOG.isDebugEnabled... +
LOG.debug("Since none of the CFs were above the size, flushing all.");
When we flush, we write the sequenceid flush to WAL. This patch should have no
effect on it.
Sequenceids are region scoped. If we flush by Store, will there be holes in
our accounting?
For example, given 3 column families, A, B, and C.
I write sequenceid 1 to A, sequenceid 2 to B, and sequenceid 3 to C. I then
write sequence 4 to A. The edit at sequenceid 4 is big and pushes us over and
brings on a flush. We flush A and edits 1 and 4. Is the fact that edits 2 and
3 are still up in memory going to mess us up.... Say the server crashes, at
replay time we see we flushed up to edit 4, will we think that we edits 2 and 3
persisted? If you don't have an answer, I can work on the answer.
> Port 'Make flush decisions per column family' to trunk
> ------------------------------------------------------
>
> Key: HBASE-10201
> URL: https://issues.apache.org/jira/browse/HBASE-10201
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: Ted Yu
> Assignee: zhangduo
> Priority: Critical
> Fix For: 2.0.0, 0.99.2
>
> Attachments: 3149-trunk-v1.txt, HBASE-10201-0.98.patch,
> HBASE-10201-0.98_1.patch, HBASE-10201-0.98_2.patch, HBASE-10201-0.99.patch,
> HBASE-10201.patch, HBASE-10201_1.patch, HBASE-10201_2.patch,
> HBASE-10201_3.patch
>
>
> Currently the flush decision is made using the aggregate size of all column
> families. When large and small column families co-exist, this causes many
> small flushes of the smaller CF. We need to make per-CF flush decisions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
