SEZ9 commented on issue #12456: URL: https://github.com/apache/seatunnel/issues/12456#issuecomment-5902826430
@wangbing505 Thanks for the Create/Release classloader log lines for job 23199974236576. The log shows both connector loaders (MongoDB and Hive) being created and then released for that job, and together with your observation that Metaspace usage keeps increasing without coming back down, I agree this looks like a classloader leak rather than an undersized Metaspace; raising the Metaspace limit would only delay the OOM, not fix it. What the release log does not tell us is what still holds a strong reference to the released `SeaTunnelChildFirstClassLoader` instances so they cannot be collected. To pin that down, it would help to have: 1. From a heap dump taken after the job finished (and after a forced GC), the textual MAT "Path to GC Roots" (excluding weak/soft references) for one surviving loader, including every field/object edge and the owning thread or static holder at the root. 2. The loader count and Metaspace used right after that job, and again after running one more identical job, so we can see whether it grows per job. @knight6236 Thanks for the detailed A/B data. The retention paths you list (MongoDB buffer-pool maintenance, Hadoop `CodecPool` / `ReflectionUtils` / token renewer `ServiceLoader`, shared DFS/RPC objects via thread context or configuration references) are exactly the kind of roots that would explain a loader surviving after release. As you note, those come from your environment, so we should not assume every path applies here; the GC-root output from this deployment will show which of them (if any) are present before we decide on a fix upstream. <!-- streview-comment:1415 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
