[
https://issues.apache.org/jira/browse/HBASE-30391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huginn updated HBASE-30391:
---------------------------
Description:
## What happens
The custom ByteArrayOutputStream copies the entire backing array when it
expands, including unused capacity.
## When it happens
This occurs when a write exceeds the current buffer capacity and the stream
allocates a larger backing array.
## Impact
The unnecessary copy increases CPU work and temporary heap memory traffic,
especially for buffers whose logical content is much smaller than their
capacity.
## Root cause
In ByteArrayOutputStream.checkSizeAndGrow, the expansion path copies buf.length
bytes even though only pos bytes contain valid stream data.
## Proposed fix
Copy only the bytes already written to the stream (pos) when expanding the
backing array. This preserves the stream contents while avoiding copies of
unused capacity.
## Reproduction
The behavior can be observed by writing data in increments that trigger buffer
expansion and profiling the resulting array-copy work. Testing evidence will be
added by the reporter.
was:
org.apache.hadoop.hbase.io.ByteArrayOutputStream copies the entire backing
array when it expands, even though only the prefix containing written data is
valid.
When a write exceeds the current capacity, checkSizeAndGrow() allocates a
larger array and copies buf.length bytes. The unused portion of the old buffer
is copied unnecessarily, increasing CPU work and temporary heap memory traffic.
The fix is to copy only pos bytes, which preserves the stream contents while
avoiding copies of unused capacity.
Reproduction and validation:
* Trigger buffer growth by writing data in increments that exceed the initial
capacity.
* Source review against Apache master identifies the expansion copy using
buf.length.
* The change was compiled with JDK 17 and passed targeted Checkstyle; no unit
test is included because this is a one-line copy-length correction.
Summary: Reduce array copying when expanding ByteArrayOutputStream
(was: https://issues.apache.org/jira/secure/CreateIssue!default.jspa)
> Reduce array copying when expanding ByteArrayOutputStream
> ---------------------------------------------------------
>
> Key: HBASE-30391
> URL: https://issues.apache.org/jira/browse/HBASE-30391
> Project: HBase
> Issue Type: Improvement
> Components: HFile
> Affects Versions: 2.4.11
> Reporter: huginn
> Priority: Minor
> Labels: pull-request-available
>
> ## What happens
> The custom ByteArrayOutputStream copies the entire backing array when it
> expands, including unused capacity.
> ## When it happens
> This occurs when a write exceeds the current buffer capacity and the stream
> allocates a larger backing array.
> ## Impact
> The unnecessary copy increases CPU work and temporary heap memory traffic,
> especially for buffers whose logical content is much smaller than their
> capacity.
> ## Root cause
> In ByteArrayOutputStream.checkSizeAndGrow, the expansion path copies
> buf.length bytes even though only pos bytes contain valid stream data.
> ## Proposed fix
> Copy only the bytes already written to the stream (pos) when expanding the
> backing array. This preserves the stream contents while avoiding copies of
> unused capacity.
> ## Reproduction
> The behavior can be observed by writing data in increments that trigger
> buffer expansion and profiling the resulting array-copy work. Testing
> evidence will be added by the reporter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
