[
https://issues.apache.org/jira/browse/FLINK-20614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17252423#comment-17252423
]
Kezhu Wang commented on FLINK-20614:
------------------------------------
[~chesnay] Sorry for ping you again. I thought about suggested fix in
[debugging_classloading|https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/debugging/debugging_classloading.html#unloading-of-dynamically-loaded-classes-in-user-code],
I think it is unfriendly to end users as it requires users to change their
deployment and treats jdbc drivers specially than other dependencies. Comparing
to other two cases "Lingering Threads" and "Interners", users are kind of
innocent to this issue, they are not writing bad-practice or buggy code.
Though, this is actually introduced by JDK, but as a framework, it would be
good for users that the dirty work is done at Flink side not their sides. It is
an once-for-all solution, we do it once in Flink, all users will benefit from
then.
Aside from this discussion, I will dig into JDK to find why [these
issues|https://ci.apache.org/projects/tomcat/tomcat85/docs/jndi-datasource-examples-howto.html#DriverManager,_the_service_provider_mechanism_and_memory_leaks]
were not fixed and is there any possibility to fix it in future JDK version.
It may take a long time, I will update what I get here.
> Registered sql drivers not deregistered after task finished in session cluster
> ------------------------------------------------------------------------------
>
> Key: FLINK-20614
> URL: https://issues.apache.org/jira/browse/FLINK-20614
> Project: Flink
> Issue Type: Bug
> Components: Connectors / JDBC, Runtime / Task
> Affects Versions: 1.12.0, 1.13.0
> Reporter: Kezhu Wang
> Priority: Major
>
> {{DriverManager}} keeps registered drivers in its internal data structures
> which prevents they from gc after task finished. I confirm it in standalone
> session cluster by observing that {{ChildFirstClassLoader}} could not be
> reclaimed after several {{GC.run}}, it should exist in all session clusters.
> Tomcat documents
> [this|https://ci.apache.org/projects/tomcat/tomcat85/docs/jndi-datasource-examples-howto.html#DriverManager,_the_service_provider_mechanism_and_memory_leaks]
> and fixes/circumvents this with
> [JdbcLeakPrevention|https://github.com/apache/tomcat/blob/master/java/org/apache/catalina/loader/JdbcLeakPrevention.java#L30].
> Should we solve this in runtime ? Or treat it as connector and clients'
> responsibility to solve it using
> {{RuntimeContext.registerUserCodeClassLoaderReleaseHookIfAbsent}} or similar ?
> Personally, it would be nice to solve in runtime as a catch-all to avoid
> memory-leaking and provide consistent behavior to clients cross per-job and
> session mode.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
