[
https://issues.apache.org/jira/browse/FLINK-23833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu reassigned FLINK-23833:
-------------------------------
Assignee: Zhilong Hong
> Cache of ShuffleDescriptors should be individually cleaned up
> -------------------------------------------------------------
>
> Key: FLINK-23833
> URL: https://issues.apache.org/jira/browse/FLINK-23833
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.14.0
> Reporter: Zhilong Hong
> Assignee: Zhilong Hong
> Priority: Critical
> Fix For: 1.14.0
>
>
> {color:#172b4d}In FLINK-23005, we introduce the cache of compressed
> serialized value for ShuffleDescriptors to improve the performance of
> deployment. To make sure the cache wouldn't stay too long and become a burden
> for GC, the cache would be cleaned up when the partition is released or reset
> for new execution. In the implementation, the cache of the entire
> IntermediateResult is cleaned up because a partition is released only when
> the entire IntermediateResult is released. {color}
> {color:#172b4d}However, after FLINK-22017, the BLOCKING result partition is
> allowed to be consumable individually. It also means that the result
> partition doesn't need to wait for other result partitions and can be
> released individually. After this change, there may be a scene: when a result
> partition is finished, the cache of IntermediateResult on the blob is
> deleted, while other result partitions corresponding to this
> IntermediateResult is just deployed to the TaskExecutor. Then when
> TaskExecutors are trying to download TDD from the blob, they will find the
> blob is deleted and get stuck.{color}
> {color:#172b4d}This bug only happens for jobs with POINTWISE BLOCKING edge.
> Also, the {{blob.offload.minsize}} is set to be a extremely small value,
> since the size of ShuffleDescriptors of POINTWISE BLOCKING edges is usually
> small. To solve this issue, we just need to clean up the cache of
> ShuffleDescriptors individually.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
