[
https://issues.apache.org/jira/browse/FLINK-11166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721074#comment-16721074
]
Tony Xintong Song commented on FLINK-11166:
-------------------------------------------
Hi [~till.rohrmann], I'd like to work on this issue myself. Could you please
kindly give me the contributor permission? Many thanks.
> Slot Placement Constraint
> -------------------------
>
> Key: FLINK-11166
> URL: https://issues.apache.org/jira/browse/FLINK-11166
> Project: Flink
> Issue Type: New Feature
> Components: ResourceManager
> Reporter: Tony Xintong Song
> Priority: Major
>
> In many cases, users may want Flink to schedule their job tasks following
> certain locality preferences. E.g., colocating upstream/downstream tasks to
> reduce data transmission costs, dispersing tasks of certain pattern (e.g.,
> I/O intensive) to avoid resource competitions, running tasks in exclusive
> TaskExecutor-s for task level resource consumption measurements, etc.
>
> Currently, there are two ways in Flink to specify such locality preferences:
> specifying preferred locations in the slot request, or setting slot sharing
> group for the task. In both ways the preferences are specified when
> requesting slots from the SlotPool and can affect how tasks are placed among
> the slots allocated to the JobMaster.
> However, there is no guarantee that such preferences can always be satisfied,
> especially when slots are customized with different resource profiles for
> different kinds of tasks. E.g., in cases where two tasks A and B are
> preferred to be scheduled onto a same TaskExecutor, it is possible that none
> of the slots customized for A offered to the JobMaster are collocated with
> slots customized for B.
> To better support locality preferences with various slot resource
> specifications, it is necessary to allow JobMaster-s to request slots
> subjected to certain placement constraints from the ResourceManager.
> In addition, most underlying frameworks Flink runs on (Yarn, Kubernetes,
> Mesos) already have individual supports for container level placement
> constraints. It is a great opportunity for Flink to leverage such underlying
> supports and enable scheduling tasks with rich locality preferences.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
