[
https://issues.apache.org/jira/browse/FLINK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger closed FLINK-11166.
----------------------------------
Resolution: Abandoned
> Slot Placement Constraint
> -------------------------
>
> Key: FLINK-11166
> URL: https://issues.apache.org/jira/browse/FLINK-11166
> Project: Flink
> Issue Type: New Feature
> Components: Runtime / Coordination
> Reporter: Abandoned Account
> Priority: Major
>
> In many cases, users may want Flink to schedule their job tasks following
> certain locality preferences. E.g., colocating upstream/downstream tasks to
> reduce data transmission costs, dispersing tasks of certain pattern (e.g.,
> I/O intensive) to avoid resource competitions, running tasks in exclusive
> TaskExecutor-s for task level resource consumption measurements, etc.
> Currently, there are two ways in Flink to specify such locality preferences:
> specifying preferred locations in the slot request, or setting slot sharing
> group for the task. In both ways the preferences are specified when
> requesting slots from the SlotPool and can affect how tasks are placed among
> the slots allocated to the JobMaster.
> However, there is no guarantee that such preferences can always be satisfied,
> especially when slots are customized with different resource profiles for
> different kinds of tasks. E.g., in cases where two tasks A and B are
> preferred to be scheduled onto a same TaskExecutor, it is possible that none
> of the slots customized for A offered to the JobMaster are collocated with
> slots customized for B.
> To better support locality preferences with various slot resource
> specifications, it is necessary to allow JobMaster-s to request slots
> subjected to certain placement constraints from the ResourceManager.
> In addition, most underlying frameworks Flink runs on (Yarn, Kubernetes,
> Mesos) already have individual supports for container level placement
> constraints. It is a great opportunity for Flink to leverage such underlying
> supports and enable scheduling tasks with rich locality preferences.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
