[
https://issues.apache.org/jira/browse/FLINK-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15859645#comment-15859645
]
ASF GitHub Bot commented on FLINK-3163:
---------------------------------------
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/3249
Added note specifying NUMA support is applicable for standalone only.
This is a much harder feature to support in a multi-application
environment, which is likely why none of these cluster managers have added
support.
> Configure Flink for NUMA systems
> --------------------------------
>
> Key: FLINK-3163
> URL: https://issues.apache.org/jira/browse/FLINK-3163
> Project: Flink
> Issue Type: Improvement
> Components: Startup Shell Scripts
> Affects Versions: 1.0.0
> Reporter: Greg Hogan
> Assignee: Greg Hogan
> Fix For: 1.3.0
>
>
> On NUMA systems Flink can be pinned to a single physical processor ("node")
> using {{numactl --membind=$node --cpunodebind=$node <command>}}. Commonly
> available NUMA systems include the largest AWS and Google Compute instances.
> For example, on an AWS c4.8xlarge system with 36 hyperthreads the user could
> configure a single TaskManager with 36 slots or have Flink create two
> TaskManagers bound to each of the NUMA nodes, each with 18 slots.
> There may be some extra overhead in transferring network buffers between
> TaskManagers on the same system, though the fraction of data shuffled in this
> manner decreases with the size of the cluster. The performance improvement
> from only accessing local memory looks to be significant though difficult to
> benchmark.
> The JobManagers may fit into NUMA nodes rather than requiring full systems.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
