yugan95 commented on PR #8159:
URL: https://github.com/apache/paimon/pull/8159#issuecomment-4665726875
@JingsongLi Thanks for the suggestion! Updated to use a combined fixed-cap
and ratio check:
```
private static final int MAX_RETAINED_REUSE_BUFFER_SIZE = 4 * 1024 * 1024;
// 4MB
private static final int SHRINK_RATIO = 4;
public void resetIfTooLarge(BinaryRow currentRow) {
if (currentRow == reuseRow
&& reuseWriter != null
&& reuseWriter.getSegments() != null) {
int bufferCapacity = reuseWriter.getSegments().size();
if (bufferCapacity > MAX_RETAINED_REUSE_BUFFER_SIZE
&& bufferCapacity > (long) currentRow.getSizeInBytes() *
SHRINK_RATIO) {
reuseRow = null;
reuseWriter = null;
}
}
}
```
- 100MB buffer + 5MB row → 20x > 4x → release
- 10MB buffer + 5MB row → 2x < 4x → retain
- 3MB buffer + tiny row → below 4MB cap → retain
SHRINK_RATIO = 4 is well above the 1.5x grow factor
(AbstractBinaryWriter.grow()), so normal buffer growth will never trigger a
false release.
Tests updated to cover the ratio-based logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
