[
https://issues.apache.org/jira/browse/SLIDER-82?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Kartha updated SLIDER-82:
--------------------------------
Attachment: SLIDER-82.001.patch
Based on the discussion on the Slider dev list:
https://www.mail-archive.com/[email protected]&q=subject:%22Re\%3A+A+basic+approach+towards+implementing+the+ANTI\-AFFINITY+in+Slider%22&o=newest&f=1
here is an early patch that tries to implement ANTI_AFFINITY_REQUIRED using:
1) The updateBlacklist() method from AMRMClientAsyncImpl introduced in
YARN-1723, which can blacklist the nodes on which the component with
ANTI_AFFINITY_REQUIRED is already running
2) Additional check for containers to be on already requested nodes and
discarding them during restart
Needless to say, this patch requires Hadoop 2.7.0 to build.
My testing with the Slider-HBase package using REGION_SERVER with
ANTI_AFFINITY_REQUIRED flag does seem to function as expected with all the
checks added, getting enforced during flex. Also any containers that got
allocated on already requested nodes during restart were being discarded.
Will investigate and add some unit tests soon.
Meanwhile, it will be useful, if someone can review the patch and add their
thought/comments.
> Support ANTI_AFFINITY_REQUIRED option
> -------------------------------------
>
> Key: SLIDER-82
> URL: https://issues.apache.org/jira/browse/SLIDER-82
> Project: Slider
> Issue Type: Task
> Components: appmaster
> Reporter: Steve Loughran
> Fix For: Slider 2.0.0
>
> Attachments: SLIDER-82.001.patch
>
>
> slider has an anti-affinity flag in roles (visible in resources.json?), which
> is ignored.
> YARN-1042 promises this for YARN, slider will need
> # flag in resources.json
> # use in container requests
> we may also want two policies: anti-affinity-desired, and -required. Then if
> required nodes get >1 container for the same component type on the same node,
> it'd have to request a new one and return the old one (Risk: getting the same
> one back).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
