[
https://issues.apache.org/jira/browse/FLINK-15325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000886#comment-17000886
]
Till Rohrmann commented on FLINK-15325:
---------------------------------------
If there is a good need for this feature, then one could introduce such a
config option. I assume this would pretty much be a power user feature as I
don't see our regular users to change it.
How exactly would this config option affect the normal scheduling and the
spread out strategy? Simply setting the collection of input preferences to
{{empty}}?
> Input location preference which affects task distribution may make certain
> job performance worse
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-15325
> URL: https://issues.apache.org/jira/browse/FLINK-15325
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Priority: Major
> Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png
>
>
> When running TPC-DS jobs in a session cluster, we observed that sometimes
> tasks are not evenly distributed in TMs. The root cause turned out to be that
> the downstream tasks tend to be TM or host local with its input tasks. This
> helps to reduce network shuffle.
> However, in certain cases, like the topology presented in the attached image,
> jamming the input task's TM and machine with downstream tasks would affect
> the performance. In this case, respecting input location preferences is
> causing troubles more than bringing benefits.
> So I'm wondering whether we should introduce a config so that users can
> disable input location preferences?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
