[
https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-15314:
-------------------------------------------
Resolution: Fixed
Assignee: chunhui shen (was: Aaron Tokhy)
Hadoop Flags: Reviewed
Fix Version/s: 2.0
Release Note: (was: The following patch adds a couple of features
without making any big changes to the existing BucketCache allocator. This
approach introduces the notion of 'segmentation' (or in other words, the
underlying IOEngine can be made up of non-contiguous segments. Two methods are
added to expose this information to the BucketCache allocator.
boolean IOEngine#isSegmented()
boolean IOEngine#doesAllocationCrossSegments(long offset, long len)
BucketCache calls these methods to determine if a 'contiguous' allocation of a
particular block size can occur. It does this by checking if
doesAllocationCrossSegments(offset, len) is true. If an allocation crosses a
segment, another call to allocate is made for the same block. The first block
is wasted (it is marked allocated). The worst case is 1 'largest block' per
file. If an allocation fails for any reason, all/any allocated blocks
(including wasted ones) are freed again for subsequent allocation requests.
This is very similar to a 'JBOD' configuration (there is no striping of any
kind).
To configure HBase to use multiple files when configuring the HBase bucket
cache, set the following configurations in hbase-site.xml:
...
<property>
<name>hbase.bucketcache.ioengine</name>
<value>files:/mnt/disk1/bucketcache,/mnt/disk2/bucketcache,/mnt/disk3/bucketcache,/mnt/disk4/bucketcache</value>
</property>
<property>
<name>hbase.bucketcache.size</name>
<value>1048756</value>
</property>
...
/mnt/disk1, /mnt/disk2, /mnt/disk3 and /mnt/disk4 are expected to be separate
filesystems (one mountpoint per volume). HBase would need to have read/write
permissions to these directories. In this configuration, 4 sparsely allocated
256GB files (each named 'bucketcache') are created in each directory on HBase
RegionServer startup. Ideally such a configuration is useful when the
underlying HBase filesystem is non-local.
)
Status: Resolved (was: Patch Available)
Pushed to master. Thanks for all the your efforts [~zyork] and @aaron
Thanks for the patch [~zjushch].
Thanks for the reviews.
[~zjushch]
Can you update the release note with the latest change? If not let me know I
will try to phrase it.
> Allow more than one backing file in bucketcache
> -----------------------------------------------
>
> Key: HBASE-15314
> URL: https://issues.apache.org/jira/browse/HBASE-15314
> Project: HBase
> Issue Type: Sub-task
> Components: BucketCache
> Reporter: stack
> Assignee: chunhui shen
> Fix For: 2.0
>
> Attachments: FileIOEngine.java, HBASE-15314.master.001.patch,
> HBASE-15314.master.001.patch, HBASE-15314.patch, HBASE-15314-v2.patch,
> HBASE-15314-v3.patch, HBASE-15314-v4.patch, HBASE-15314-v5.patch,
> HBASE-15314-v6.patch, HBASE-15314-v7.patch, HBASE-15314-v8.patch
>
>
> Allow bucketcache use more than just one backing file: e.g. chassis has more
> than one SSD in it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
