[
https://issues.apache.org/jira/browse/BEAM-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16748279#comment-16748279
]
Maximilian Michels commented on BEAM-6460:
------------------------------------------
Update on this: Not a Beam issue. Flink leaks classloaders when libraries are
loaded through the Flink root classloader which have static caches. For
example, when Flink's "yarn-cluster" mode is used, the user jar is part of the
Flink root classloader.
The subtlety of this issue warrants that we clear the Jackson cache to avoid
other users running into this. Ultimately, this should be fixed upstream by
Flink.
> Jackson Cache may hold on to Classloader after pipeline restart
> ---------------------------------------------------------------
>
> Key: BEAM-6460
> URL: https://issues.apache.org/jira/browse/BEAM-6460
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.7.0
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Blocker
> Fix For: 2.10.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> It looks like Jackson has an internal cache which may continue to hold the
> Flink application classloader through its {{TypeFactory}} class. When the
> pipeline is restarted due to a failure, a new classloader is created which
> can result in too many classes being loaded.
> Reported on the user mailing list:
> https://lists.apache.org/thread.html/e201891684ef3dcffce48d20d1f9be0e19fc2294334362cc7092c0ff@%3Cuser.beam.apache.org%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
