[
https://issues.apache.org/jira/browse/HBASE-30391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huginn updated HBASE-30391:
---------------------------
Description:
org.apache.hadoop.hbase.io.ByteArrayOutputStream copies the entire backing
array when it expands, even though only the prefix containing written data is
valid.
When a write exceeds the current capacity, checkSizeAndGrow() allocates a
larger array and copies buf.length bytes. The unused portion of the old buffer
is copied unnecessarily, increasing CPU work and temporary heap memory traffic.
The fix is to copy only pos bytes, which preserves the stream contents while
avoiding copies of unused capacity.
Reproduction and validation:
* Trigger buffer growth by writing data in increments that exceed the initial
capacity.
* Source review against Apache master identifies the expansion copy using
buf.length.
* The change was compiled with JDK 17 and passed targeted Checkstyle; no unit
test is included because this is a one-line copy-length correction.
was:
## What happens
The custom ByteArrayOutputStream copies the entire backing array when it
expands, including unused capacity.
## When it happens
This occurs when a write exceeds the current buffer capacity and the stream
allocates a larger backing array.
## Impact
The unnecessary copy increases CPU work and temporary heap memory traffic,
especially for buffers whose logical content is much smaller than their
capacity.
## Root cause
In ByteArrayOutputStream.checkSizeAndGrow, the expansion path copies buf.length
bytes even though only pos bytes contain valid stream data.
## Proposed fix
Copy only the bytes already written to the stream (pos) when expanding the
backing array. This preserves the stream contents while avoiding copies of
unused capacity.
## Reproduction
The behavior can be observed by writing data in increments that trigger buffer
expansion and profiling the resulting array-copy work. Testing evidence will be
added by the reporter.{color:red}着色文本{color}
> Reduce array copying when expanding ByteArrayOutputStream
> ---------------------------------------------------------
>
> Key: HBASE-30391
> URL: https://issues.apache.org/jira/browse/HBASE-30391
> Project: HBase
> Issue Type: Improvement
> Components: HFile
> Affects Versions: 2.4.11
> Reporter: huginn
> Priority: Minor
> Labels: pull-request-available
>
> org.apache.hadoop.hbase.io.ByteArrayOutputStream copies the entire backing
> array when it expands, even though only the prefix containing written data is
> valid.
> When a write exceeds the current capacity, checkSizeAndGrow() allocates a
> larger array and copies buf.length bytes. The unused portion of the old
> buffer is copied unnecessarily, increasing CPU work and temporary heap memory
> traffic.
> The fix is to copy only pos bytes, which preserves the stream contents while
> avoiding copies of unused capacity.
> Reproduction and validation:
> * Trigger buffer growth by writing data in increments that exceed the initial
> capacity.
> * Source review against Apache master identifies the expansion copy using
> buf.length.
> * The change was compiled with JDK 17 and passed targeted Checkstyle; no unit
> test is included because this is a one-line copy-length correction.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
