[
https://issues.apache.org/jira/browse/SAMZA-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prateek Maheshwari updated SAMZA-2439:
--------------------------------------
Description:
Goal: Make JobModel immutable by removing locality information (container to
host mappint) from the JobModel.
Motivation:
1. Enables us to move JobModel to public APIs and expose it in JobContext.
2. Enables us to cache and serve serialized JobModel from the AM servlet to
reduce AM overhead (memory, open connections, num threads) during container
startup, esp. for jobs with a large number of containers (See:
https://github.com/apache/samza/pull/1241)
3. Removes tech debt: models should be immutable, and should not update
themselves.
4. Removes tech debt: makes current container location a first class concept
for container scheduling / placement , and for tools like dashboard,
samza-rest, auto-scaling, diagnostics etc.
Proposal:
Move locality information (and LocalityManager) out of the JobModel. New
JobModel should only contain Config and ContainerModel.
Add a new AM HTTP endpoint for getting the container location information for
external tools.
Update Samza Dashboard and any other tools that rely on locality information to
get it from the new endpoint instead of the JobModel.
was:
Goal: Make JobModel immutable by removing locality information (container to
host mappint) from the JobModel.
Motivation:
1. Enables us to move JobModel to public APIs and expose it in JobContext.
2. Enables us to cache and serve serialized JobModel from the AM servlet to
reduce AM overhead (memory, open connections, num threads) during container
startup, esp. for jobs with a large number of containers.
3. Removes tech debt: models should be immutable, and should not update
themselves.
4. Removes tech debt: makes current container location a first class concept
for container scheduling / placement , and for tools like dashboard,
samza-rest, auto-scaling, diagnostics etc.
Proposal:
Move locality information (and LocalityManager) out of the JobModel. New
JobModel should only contain Config and ContainerModel.
Add a new AM HTTP endpoint for getting the container location information for
external tools.
Update Samza Dashboard and any other tools that rely on locality information to
get it from the new endpoint instead of the JobModel.
> Remove LocalityManager and container location information from JobModel
> -----------------------------------------------------------------------
>
> Key: SAMZA-2439
> URL: https://issues.apache.org/jira/browse/SAMZA-2439
> Project: Samza
> Issue Type: Improvement
> Reporter: Prateek Maheshwari
> Priority: Major
>
> Goal: Make JobModel immutable by removing locality information (container to
> host mappint) from the JobModel.
> Motivation:
> 1. Enables us to move JobModel to public APIs and expose it in JobContext.
> 2. Enables us to cache and serve serialized JobModel from the AM servlet to
> reduce AM overhead (memory, open connections, num threads) during container
> startup, esp. for jobs with a large number of containers (See:
> https://github.com/apache/samza/pull/1241)
> 3. Removes tech debt: models should be immutable, and should not update
> themselves.
> 4. Removes tech debt: makes current container location a first class concept
> for container scheduling / placement , and for tools like dashboard,
> samza-rest, auto-scaling, diagnostics etc.
> Proposal:
> Move locality information (and LocalityManager) out of the JobModel. New
> JobModel should only contain Config and ContainerModel.
> Add a new AM HTTP endpoint for getting the container location information for
> external tools.
> Update Samza Dashboard and any other tools that rely on locality information
> to get it from the new endpoint instead of the JobModel.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
