Bharath Kumarasubramanian created SAMZA-2580:
------------------------------------------------
Summary: Evaluate when to update locality mapping
Key: SAMZA-2580
URL: https://issues.apache.org/jira/browse/SAMZA-2580
Project: Samza
Issue Type: Improvement
Reporter: Bharath Kumarasubramanian
Currently, we update the locality mapping as soon as the container starts
barring few components (metrics, admins & offset manager). However, the stores
get started after we update the locality.
Consider the following scenario
# A stateful container is running on host A and has a large local state.
# The container fails and AM tries to bring it up and fails to get back on the
same allocated hosts.
# After max retries, it chooses a new host (host B) to run the container and
the container fails during state restoration.
# Based on how far the state restoration has progressed it might be optimal to
retry to place the container on host A on subsequent attempt rather host B for
faster restore & startup time.
The move on the first sight looks theoretically better. However, if it isn't
then we should consider moving it outside the container start so simplify
wiring locality manager in multiple places. i.e. LocalContainerRunner could
write locality and doesn't need to pass it to container.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
