[
https://issues.apache.org/jira/browse/FLINK-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16999895#comment-16999895
]
Zhu Zhu commented on FLINK-15325:
---------------------------------
[~trohrmann] do you think we should have such a config to switch on/off input
location preferences, so that there is a way to force tasks to be evenly
distributed with it as well as "cluster.evenly-spread-out-slots"?
> Input location preference which affects task distribution may make certain
> job performance worse
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-15325
> URL: https://issues.apache.org/jira/browse/FLINK-15325
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Priority: Major
> Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png
>
>
> When running TPC-DS jobs in a session cluster, we observed that sometimes
> tasks are not evenly distributed in TMs. The root cause turned out to be that
> the downstream tasks tend to be TM or host local with its input tasks. This
> helps to reduce network shuffle.
> However, in certain cases, like the topology presented in the attached image,
> jamming the input task's TM and machine with downstream tasks would affect
> the performance. In this case, respecting input location preferences is
> causing troubles more than bringing benefits.
> So I'm wondering whether we should introduce a config so that users can
> disable input location preferences?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
