[
https://issues.apache.org/jira/browse/FLINK-20333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dian Fu closed FLINK-20333.
---------------------------
Fix Version/s: 1.11.3
Resolution: Fixed
Fixed in
- master via c354f7bd679b9fa8c1e0d75feb3827ccca7f317b
- release-1.11 via 18392118c38dc6a1b3a76a76bc2d0fb3ee2f8d7f
> Flink standalone cluster throws metaspace OOM after submitting multiple
> PyFlink UDF jobs.
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-20333
> URL: https://issues.apache.org/jira/browse/FLINK-20333
> Project: Flink
> Issue Type: Bug
> Components: API / Python
> Reporter: Wei Zhong
> Assignee: Wei Zhong
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.12.0, 1.11.3
>
>
> Currently the Flink standalone cluster will throw metaspace OOM after
> submitting multiple PyFlink UDF jobs. The root cause is that currently the
> PyFlink classes are running in user classloader and so each job creates a
> separate user class loader to load PyFlink related classes. There are many
> soft references and Finalizers in memory (introduced by the underlying
> Netty), which prevents the garbage collection of the user classloader of
> already finished PyFlink jobs.
> Due to their existence, it needs multiple full gc to reclaim the classloader
> of the completed job. If only one full gc is performed before the metaspace
> space is insufficient, then OOM will occur.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
