SEZ9 commented on issue #12456: URL: https://github.com/apache/seatunnel/issues/12456#issuecomment-6392941947
Thanks @wangbing505 for the 0/5/10-job comparison and the full GC-root paths — that is the key evidence here. The repeated root through Hadoop `ReflectionUtils.CONSTRUCTOR_CACHE` (holding the shaded `SnappyCodec` class defined by the released `SeaTunnelChildFirstClassLoader`) is a process-wide static cache, and the two #12456 cleanup commits already on `dev` (`35b9d5e26d2` and `097101c21be`, as of `4ae4fee30165e9f082f80b05ec5a8f3bb810b474`) do not cover it. They handle the MongoDB pruner, `WritableComparator`, and the `CodecPool` registries, so this remaining root is expected to keep growing per job even on current `dev`. I can't consider the issue resolved until it is handled. @modelpath-dev — waiting for another snapshot after the job finishes won't change the outcome for this particular root: a static cache entry keyed by the connector's class keeps the loader reachable regardless of job state, so the loader count will not drop on its own. The fix needs to be at the same last-reference release point used by the existing cleaner, removing only `CONSTRUCTOR_CACHE` entries whose key class was defined by the released loader. It must not clear or replace the whole cache, since entries belonging to running jobs or ancestor loaders have to survive. Concrete remaining asks: 1. A follow-up change that prunes `ReflectionUtils.CONSTRUCTOR_CACHE` entries owned by the released non-cached job loader, alongside the existing registry cleanup. 2. A regression test with two isolated child class loaders: releasing one must remove only its cache entries while the other loader's entries remain intact. 3. @wangbing505, once that is in place, please rerun the same 0/5/10-job experiment on a build from current `dev`, trigger a full GC before each reading, and post the exact `SeaTunnelChildFirstClassLoader` counts and Metaspace figures. If any loaders remain, the GC-root paths for those would again be very helpful. <!-- streview-comment:2019 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
