Rayman created SAMZA-2501:
-----------------------------
Summary: Optimizing startpoint manager to not make successive
bootstrapMessage calls to coordinator-store
Key: SAMZA-2501
URL: https://issues.apache.org/jira/browse/SAMZA-2501
Project: Samza
Issue Type: Bug
Reporter: Rayman
Symptom: Currently the startpoint manager queries metastore for task-ssps and
for each ssp queries coordinator-store, issuing a re-read on the coordinator
store. This causes increased AM startup time and depending on the number of
input SSPs to a job (thousands in case of regex), YARN may timeout the AM.
Cause: Above.
Fix: This change reduces to number of metastore reads to 2; one for startpoints
keyed by ssp and the other for startpoints keyed by ssp+taskname.
API changes: None
Upgrade Instructions: None
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
