[
https://issues.apache.org/jira/browse/FLINK-20333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17414278#comment-17414278
]
Puneet Duggal edited comment on FLINK-20333 at 9/13/21, 3:58 PM:
-----------------------------------------------------------------
Hi, Is this issue resolved for java client... Asking this because in prod task
manager with 256Mb jvm metaspace... i am getting metaspace OOM by deploying 4-5
jobs on single task manager irrespective of their parallelism. I am using flink
1.12.0 version. [~dian.fu]
was (Author: pd17):
Hi, Is this issue resolved for java client... Asking this because in prod task
manager with 256Mb jvm metaspace... i am getting metaspace OOM by deploying 4-5
jobs on single task manager irrespective of their parallelism. I am using flink
1.12.0 version.
> Flink standalone cluster throws metaspace OOM after submitting multiple
> PyFlink UDF jobs.
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-20333
> URL: https://issues.apache.org/jira/browse/FLINK-20333
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Reporter: Wei Zhong
> Assignee: Wei Zhong
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.11.3, 1.12.0
>
>
> Currently the Flink standalone cluster will throw metaspace OOM after
> submitting multiple PyFlink UDF jobs. The root cause is that currently the
> PyFlink classes are running in user classloader and so each job creates a
> separate user class loader to load PyFlink related classes. There are many
> soft references and Finalizers in memory (introduced by the underlying
> Netty), which prevents the garbage collection of the user classloader of
> already finished PyFlink jobs.
> Due to their existence, it needs multiple full gc to reclaim the classloader
> of the completed job. If only one full gc is performed before the metaspace
> space is insufficient, then OOM will occur.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
