[
https://issues.apache.org/jira/browse/HBASE-14757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988808#comment-14988808
]
Andrew Purtell edited comment on HBASE-14757 at 11/4/15 6:04 PM:
-----------------------------------------------------------------
For HFileBlock read buffers, these allocations are expected. This is where
block data lands. These allocations are carried through into the block cache
when using the LRUBlockCache and are retained potentially for a very long time.
When using BucketCache, the on-heap block data isn't needed after the IO engine
serializes it off-heap or to SSD. We could reuse allocations for the latter
case but not the former. Needs investigation.
Meanwhile, for block encoding / seeker state, I think these are all short lived
allocations. So, a strawman: Round up byte[] allocation requests here to the
nearest power of 2. Keep object caches for powers of 2 over a common range.
Serve requests from the appropriate free list where available. Prune free lists
using LRU.
was (Author: apurtell):
For HFileBlock read buffers: These allocations are expected. This is where
block data lands. These allocations are carried through into the block cache
when using the LRUBlockCache and are retained potentially for a very long time.
When using BucketCache, the on-disk block data isn't needed after the IO engine
serializes it off-heap or to SSD. We could reuse allocations for the latter
case but not the former. Needs investigation.
Meanwhile, for block encoding / seeker state, I think these are all short lived
allocations. So, a strawman: Round up byte[] allocation requests here to the
nearest power of 2. Keep object caches for powers of 2 over a common range.
Serve requests from the appropriate free list where available. Prune free lists
using LRU.
> Reduce allocation pressure imposed by HFile block processing
> ------------------------------------------------------------
>
> Key: HBASE-14757
> URL: https://issues.apache.org/jira/browse/HBASE-14757
> Project: HBase
> Issue Type: Improvement
> Reporter: Andrew Purtell
> Assignee: Andrew Purtell
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
>
> Using Flight Recorder to look at the object allocation profiles of
> regionservers processing the various YCSB workloads when block encoding is
> enabled (specifically, FAST_DIFF, but this applies to any), we can see:
> - Allocations of byte[] for block encoding contribute 40-70% of all
> allocation pressure in TLABs.
> - Of that subset of allocation pressure, ~50-70% is byte[] for SeekerState
> - Greater than 99% of allocation of byte[] outside of TLABs are for read
> buffers for HFileBlock#readBlockDataInternal.
> This issue is for investigation of strategy for and impact of reducing that
> allocation pressure. Reducing allocation pressure reduces demand for GC,
> which reduces GC activity overall, which reduces a source of system latency.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
