QiuYucheng2003 opened a new issue, #18942:
URL: https://github.com/apache/druid/issues/18942
### Affected Version
Current Master / Trunk (Identified in
org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java)
### Description
Problem Summary:
A potential memory leak exists in ConcurrentGrouper. The class uses a
ThreadLocal<SpillingGrouper> to manage thread-specific aggregation buffers but
fails to clean it up in the close() method.
Detailed Analysis:
Initialization: A ThreadLocal field named threadLocalGrouper is initialized
in the constructor (Line 184) to bind SpillingGrouper instances to processing
threads.
The Bug: The close() method (Line 479) iterates through the groupers list to
close them, but misses calling threadLocalGrouper.remove().
Impact: In Druid's thread-pool environment (Historical/Broker nodes), the
processing threads remain alive after the query finishes. Because .remove() is
not called, the SpillingGrouper instances (and their underlying ByteBuffers)
remain referenced by the threads' ThreadLocalMap, preventing Garbage
Collection. This leads to a gradual Memory Leak.
Steps to reproduce the problem:
1. Inspect
org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java.
2. Observe that threadLocalGrouper is created via
ThreadLocal.withInitial(...).
3. Check the close() implementation; it lacks the necessary cleanup call.
Relevant Code Snippet:
// Location:
org.apache.druid.query.groupby.epinephelinae.ConcurrentGrouper.java
@Override
public void close()
{
if (!closed) {
closed = true;
groupers.forEach(Grouper::close);
// MISSING: threadLocalGrouper.remove(); <--- LEAK SOURCE
}
}
Proposed Fix:
Add threadLocalGrouper.remove() inside the close() method to ensure
resources are detached from the thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
