[
https://issues.apache.org/jira/browse/IGNITE-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18105259#comment-18105259
]
Evgeny Stanilovsky commented on IGNITE-28980:
---------------------------------------------
[~mroeppis] thank you, fill some comments, plz check them
> BinaryMemoryAllocator$ThreadLocalAllocator$Chunk does not reset maxMsgSize
> after shrink check
> ---------------------------------------------------------------------------------------------
>
> Key: IGNITE-28980
> URL: https://issues.apache.org/jira/browse/IGNITE-28980
> Project: Ignite
> Issue Type: Bug
> Components: general
> Affects Versions: 2.18
> Environment: * Apache Ignite 2.18.0
> * {{BinaryMemoryAllocator$ThreadLocalAllocator$Chunk}}
> * Java 25.0.3+9
> * Linux 6.8.0-100-generic amd64
> Reporter: Mathias Roeppischer
> Assignee: Mathias Roeppischer
> Priority: Major
> Fix For: 2.19
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {{BinaryMemoryAllocator$ThreadLocalAllocator$Chunk}} keeps track of the
> maximum message size detected between checks in {{maxMsgSize}}.
> The value is updated as follows:
> {code:java}
> if (maxMsgSize > this.maxMsgSize)
> this.maxMsgSize = maxMsgSize;
> {code}
> The shrink logic periodically checks this value based on {{CHECK_FREQ}}:
> {code:java}
> if (CommonUtils.nanosToMillis(nowNanos - lastCheckNanos) >= CHECK_FREQ) {
> int halfSize = data.length >> 1;
> if (this.maxMsgSize < halfSize)
> this.data = new byte[halfSize];
> lastCheckNanos = nowNanos;
> }
> {code}
> However, {{this.maxMsgSize}} is never reset after the check.
> As a result, the value effectively represents the maximum message size seen
> during the entire lifetime of the {{Chunk}}, rather than the maximum message
> size detected between checks as documented.
> h3. Impact
> A single large serialization can permanently prevent the chunk from shrinking
> again.
> For example, if a chunk grows to 128 MB due to a large message and
> {{maxMsgSize}} reaches 100 MB, subsequent small messages cannot lower
> {{maxMsgSize}} below 100 MB. Consequently, the shrink condition
> {code:java}
> this.maxMsgSize < halfSize
> {code}
> may never become true, even if the chunk is subsequently used only for much
> smaller messages.
> This can cause large byte arrays to remain retained by thread-local
> allocators. In applications using thread pools, a number of worker threads
> can therefore retain large buffers after processing occasional large
> messages, resulting in unnecessarily high heap usage.
> h3. Expected behavior
> {{maxMsgSize}} should represent the maximum message size observed during the
> current check interval.
> After the shrink check has been performed, the recorded maximum should be
> reset so that the next interval starts with a fresh measurement.
> h3. Proposed fix
> Reset {{maxMsgSize}} after each {{CHECK_FREQ}} check:
> {code:java}
> if (CommonUtils.nanosToMillis(nowNanos - lastCheckNanos) >= CHECK_FREQ) {
> int halfSize = data.length >> 1;
> if (this.maxMsgSize < halfSize)
> this.data = new byte[halfSize];
> this.maxMsgSize = 0;
> lastCheckNanos = nowNanos;
> }
> {code}
> This preserves the existing design of collecting the maximum message size
> between checks while allowing the chunk to shrink when subsequent
> serialization workloads are smaller.
> h3. Additional Information
> Discussion on the Ignite User Mailing list:
> https://lists.apache.org/thread/cr2rvdpyh1njn2wghvg1mq7gr66wgjls
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
