Yukang-Lian opened a new pull request, #64140:
URL: https://github.com/apache/doris/pull/64140
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Page compression builds an OwnedSlice directly from a
faststring. For large ZSTD pages, the faststring capacity can remain at the max
compressed length even after its logical size is reduced to the actual
compressed size. OwnedSlice preserves that capacity, so buffered compressed
pages can retain much more heap than their visible slice size. This change
shrinks the compressed faststring before building the OwnedSlice and adds a
unit test covering the retained capacity for a large ZSTD-compressed page.
### Release note
Fix a BE memory amplification issue when retaining compressed page buffers.
### Check List (For Author)
- Test: Unit Test and code style check
- Unit Test: ./run-be-ut.sh --run --filter=PageIOTest.* -j 80
- Code Style: PATH=/data/data14/lianyukang/ldb_toolchain_v16/bin:$PATH
build-support/check-format.sh
- Behavior changed: Yes. Compressed page buffers shrink retained capacity
before being stored.
- Does this need documentation: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
