Prateek Maheshwari created SAMZA-2439:
-----------------------------------------
Summary: Remove LocalityManager and container location information
from JobModel
Key: SAMZA-2439
URL: https://issues.apache.org/jira/browse/SAMZA-2439
Project: Samza
Issue Type: Improvement
Reporter: Prateek Maheshwari
Goal: Make JobModel immutable by removing locality information (container to
host mappint) from the JobModel.
Motivation:
1. Enables us to move JobModel to public APIs and expose it in JobContext.
2. Enables us to cache and serve serialized JobModel from the AM servlet to
reduce AM overhead (memory, open connections, num threads) during container
startup, esp. for jobs with a large number of containers.
3. Removes tech debt: models should be immutable, and should not update
themselves.
4. Removes tech debt: makes current container location a first class concept
for container scheduling / placement , and for tools like dashboard,
samza-rest, auto-scaling, diagnostics etc.
Proposal:
Move locality information (and LocalityManager) out of the JobModel. New
JobModel should only contain Config and ContainerModel.
Add a new AM HTTP endpoint for getting the container location information for
external tools.
Update Samza Dashboard and any other tools that rely on locality information to
get it from the new endpoint instead of the JobModel.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
