Zhu Zhu created FLINK-15325:
-------------------------------
Summary: Input location preference which affects task distribution
may make certain job performance worse
Key: FLINK-15325
URL: https://issues.apache.org/jira/browse/FLINK-15325
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.10.0
Environment: When running TPC-DS jobs in a session cluster, we
observed that sometimes tasks are not evenly distributed in TMs. The root cause
turned out to be that the downstream tasks tend to be TM or host local with its
input tasks. This helps to reduce network shuffle.
However, in certain cases, like the topology presented in the attached image,
jamming the input task's TM and machine with downstream tasks would affect the
performance. In this case, respecting input location preferences is causing
troubles more than bringing benefits.
So I'm wondering whether we should introduce a config to disable input location
preference?
Reporter: Zhu Zhu
Attachments: D58ADB03-7187-46B1-B077-91E5005FD463.png
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
