xiangfu0 commented on code in PR #19828:
URL: https://github.com/apache/pinot/pull/19828#discussion_r4265374152
##########
pinot-common/pom.xml:
##########
@@ -201,7 +201,7 @@
</dependency>
<dependency>
<groupId>org.apache.arrow</groupId>
- <artifactId>arrow-memory-netty</artifactId>
+ <artifactId>arrow-memory-unsafe</artifactId>
Review Comment:
Fixed in 259d50e9089. All four Pinot-owned OSS roots now use one
allocation-manager factory with a shared finite Arrow reservation derived from
the existing Netty direct-memory ceiling. Int-sized buffers use public pooled
ByteBuf APIs and participate in the ordinary Netty counter, while caller
limits, children and final-reference release remain intact. The paired StarTree
commit 090d07c293 routes its five production sites through the same provided
common helper.
The legacy larger-than-int raw path is now bounded by the shared Arrow
reservation, but remains separate from Netty accounting; external
factories/native Arrow allocations are outside this budget. The encoder also
closes partially constructed vectors when allocation/filling fails, preventing
stranded reservations.
The committed ArrowAllocatorsTest rejects an oversized response under a 1
MiB limit, verifies shared roots/children, existing Netty usage, rollback and
retained references, then succeeds on a small IPC round-trip. An immutable old
unsafe-encoder control accepted 2 MiB, and a factory-only control without
cleanup leaked 65,536 bytes. Fresh wrapper/JDK 25 OSS coverage passes 45 cases
with two existing skips and no failures/errors; paired StarTree passes 176 Java
and 612 Rust cases. Affected style/license checks pass. New-head hosted CI and
allocator performance remain separate gates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
