[
https://issues.apache.org/jira/browse/FLINK-13752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16910819#comment-16910819
]
Tzu-Li (Gordon) Tai commented on FLINK-13752:
---------------------------------------------
Cherry-picked for 1.9.0: 04e95278777611519f5d14813dec4cbc533e2934
> TaskDeploymentDescriptor cannot be recycled by GC due to referenced by an
> anonymous function
> --------------------------------------------------------------------------------------------
>
> Key: FLINK-13752
> URL: https://issues.apache.org/jira/browse/FLINK-13752
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.9.0
> Reporter: Yun Gao
> Assignee: Yun Gao
> Priority: Critical
> Fix For: 1.9.0
>
>
> When comparing the 1.8 and 1.9.0-rc2 on a test streaming job, we found that
> the performance on 1.9.0-rc2 is much lower than that of 1.8. By comparing the
> two versions, we found that the count of Full GC of TaskExecutor process on
> 1.9.0-rc2 is much more than that on 1.8.
> A further analysis found that the difference is due to in
> _TaskExecutor#setupResultPartitionBookkeeping_, the anonymous function in
> _taskTermimationWithResourceCleanFuture_ has referenced the
> _TaskDeploymentDescriptor_, since this function will be kept till the task is
> terminated, _TaskDeploymentDescriptor_ will also be kept referenced in the
> closure and cannot be recycled by GC. In this job, _TaskDeploymentDescriptor_
> of some tasks are as large as 10M, and the total heap is about 113M, thus the
> kept _TaskDeploymentDescriptors_ will cause relatively large impact on GC and
> performance.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
