[
https://issues.apache.org/jira/browse/HBASE-30391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huginn updated HBASE-30391:
---------------------------
Description:
ByteArrayOutputStream copies the entire backing array when it expands, even
though only the prefix containing written data is valid.
When a write exceeds the current capacity, checkSizeAndGrow() allocates a
larger array and copies buf.length bytes. The unused portion of the old buffer
is copied unnecessarily, increasing CPU work and temporary heap memory traffic.
The fix is to copy only pos bytes, which preserves the stream contents while
avoiding copies of unused capacity.
was:
## What happens
The custom ByteArrayOutputStream copies the entire backing array when it
expands, including unused capacity.
## When it happens
This occurs when a write exceeds the current buffer capacity and the stream
allocates a larger backing array.
## Impact
The unnecessary copy increases CPU work and temporary heap memory traffic,
especially for buffers whose logical content is much smaller than their
capacity.
## Root cause
In ByteArrayOutputStream.checkSizeAndGrow, the expansion path copies buf.length
bytes even though only pos bytes contain valid stream data.
## Proposed fix
Copy only the bytes already written to the stream (pos) when expanding the
backing array. This preserves the stream contents while avoiding copies of
unused capacity.
## Reproduction
The behavior can be observed by writing data in increments that trigger buffer
expansion and profiling the resulting array-copy work. Testing evidence will be
added by the reporter.
> Reduce array copying when expanding ByteArrayOutputStream
> ---------------------------------------------------------
>
> Key: HBASE-30391
> URL: https://issues.apache.org/jira/browse/HBASE-30391
> Project: HBase
> Issue Type: Improvement
> Components: HFile
> Affects Versions: 2.4.11
> Reporter: huginn
> Priority: Minor
> Labels: pull-request-available
>
> ByteArrayOutputStream copies the entire backing array when it expands, even
> though only the prefix containing written data is valid.
> When a write exceeds the current capacity, checkSizeAndGrow() allocates a
> larger array and copies buf.length bytes. The unused portion of the old
> buffer is copied unnecessarily, increasing CPU work and temporary heap memory
> traffic.
> The fix is to copy only pos bytes, which preserves the stream contents while
> avoiding copies of unused capacity.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
