[
https://issues.apache.org/jira/browse/HBASE-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584201#action_12584201
]
Jim Kellerman commented on HBASE-469:
-------------------------------------
> stack - 31/Mar/08 02:26 PM
> Before reviewing, does this new patch do as previous versions did - IIRC -
> removing the memory cap
> for a region rather moving the memory cap down to the store such that if ten
> stores, we could use ten
> times the memory we used to use if all stores are undergoing updates?
No the flushing policy has not changed. That is it still uses checkResources to
block updates when the total memcache size for a region reaches 64MB.
The patch for HBASE-69 is the one you're thinking of.
This patch makes the HStore smarter. For example loadHStoreFiles also gets the
sequence number and file size while it is loading the stores so a separate scan
is no longer necessary. cache flushing updates the store size. The only time
the store size is recomputed is after a compaction: in that case compact knows
the size of the file it just created so it only has to look at any new store
files that might have been created by a cache flush that occurred while the
compaction was running.
compact also computes the mid key if the store needs to be (and can be) split
even if it decides no compaction is necessary. Thus the compactor thread knows
if the region needs to be split after calling compact and does not have to
query the region.
QueueEntry's were removed because neither the flusher nor the compactor use
delay queues and at that point, the only payload of the QueueEntry was the
region reference. HRegion now implements equals and hashCode so it can be
inserted into a blocking queue and a hash set. The hash set is a set of all the
regions in the queue and can determine if a region is in the queue much faster
than BlockingQueue.contains() which does a linear scan of the queue.
> Streamline HStore startup and compactions
> -----------------------------------------
>
> Key: HBASE-469
> URL: https://issues.apache.org/jira/browse/HBASE-469
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: regionserver
> Affects Versions: 0.2.0
> Reporter: Jim Kellerman
> Assignee: Jim Kellerman
> Fix For: 0.2.0
>
> Attachments: patch.txt, patch.txt
>
>
> Several useful patches that streamline HStore startup and compactions that
> were a part of the abandoned changes in HBASE-69 should be incorporated.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
