[
https://issues.apache.org/jira/browse/SAMZA-1013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Logvinenko updated SAMZA-1013:
-------------------------------------
Description:
YARN Node labels were introduced in Hadoop version 2.6, which allows to group
nodes with similar characteristics and allows applications to specify where to
run. This patch adds support for YARN node labels in Samza.
In this implementation, node labels are defined directly in yarnConfig in
YarnClusterResourceManager. It might be better to have node labels as a part of
SamzaResourceRequest and SamzaResource classes, but
org.apache.hadoop.yarn.api.records.Container class doesn't contain node label
and hence we have nothing to pass to the SamzaResource constructor in
onContainersAllocated method of YarnClusterResourceManager class.
RB Code review: https://reviews.apache.org/r/51633/
was:
YARN Node labels were introduced in Hadoop version 2.6, which allows to group
nodes with similar characteristics and allows applications to specify where to
run. This patch adds support for YARN node labels in Samza.
In this implementation, node labels are defined directly in yarnConfig in
YarnClusterResourceManager. It might be better to have node labels as a part of
SamzaResourceRequest and SamzaResource classes, but
org.apache.hadoop.yarn.api.records.Container class doesn't contain node label
and hence we have nothing to pass to the SamzaResource constructor in
onContainersAllocated method of YarnClusterResourceManager class.
> Add YARN Node label support
> ---------------------------
>
> Key: SAMZA-1013
> URL: https://issues.apache.org/jira/browse/SAMZA-1013
> Project: Samza
> Issue Type: Improvement
> Reporter: Maksim Logvinenko
> Attachments: 0001-Add-YARN-Node-label-support.patch
>
>
> YARN Node labels were introduced in Hadoop version 2.6, which allows to group
> nodes with similar characteristics and allows applications to specify where
> to run. This patch adds support for YARN node labels in Samza.
> In this implementation, node labels are defined directly in yarnConfig in
> YarnClusterResourceManager. It might be better to have node labels as a part
> of SamzaResourceRequest and SamzaResource classes, but
> org.apache.hadoop.yarn.api.records.Container class doesn't contain node label
> and hence we have nothing to pass to the SamzaResource constructor in
> onContainersAllocated method of YarnClusterResourceManager class.
> RB Code review: https://reviews.apache.org/r/51633/
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
