[
https://issues.apache.org/jira/browse/SAMZA-1334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055960#comment-16055960
]
ASF GitHub Bot commented on SAMZA-1334:
---------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/samza/pull/228
> When host-affinity is turned off, ContainerAllocator should ignore any
> previous container locality info
> -------------------------------------------------------------------------------------------------------
>
> Key: SAMZA-1334
> URL: https://issues.apache.org/jira/browse/SAMZA-1334
> Project: Samza
> Issue Type: Bug
> Reporter: Yi Pan (Data Infrastructure)
>
> Consider a case where host affinity is turned on once for a job, and the
> locality info is written to the coordinator stream. Then the user may turn
> off the host affinity feature.
> That triggers a bug in ContainerAllocator:
> 1) it gets the locality map from JobModel which has the list of preferred
> hosts from the coordinator stream. Hence, ContainerAllocator is making
> preferred host resource requests.
> 2) At the end, ContainerAllocator finishes launching all containers and tries
> to release all extra containers mapping to ANY_HOST. However, all preferred
> host resource responses are kept under the specific host's entry. Hence, it
> failed to release those containers.
> The end result is: the job is still successfully launched. However, YARN RM
> reports a lot of reserved memory/containers not released by the job. In some
> extreme cases, the reserved memory/container can be huge and affects the
> availability of the whole cluster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
