mynameborat opened a new pull request #1449:
URL: https://github.com/apache/samza/pull/1449


   **Description**: 
   With AM HA, the new AM will regenerate job model, do job planning and 
rewrite configurations and may have to spin up containers. The new rewritten 
configurations, job model can change from the existing metadata on which the 
containers are running. In order to handle this change of metadata, we need a 
way to identify lifecycle of the AM and store some information on the previous 
snapshot of the metadata.
   
   **Changes**: 
   1. Epoch identifier of the AM attempt so that we can determine if the new 
attempt is part of the same lifecycle of AM or a new deployment lifecycle
   2. Metadata on the job model, configuration so, that we can use this 
information to decide the course of action if current metadata changed in 
comparison with the previous snapshot.
   3. Metrics capturing the changes in metadata
   4. Coordinator stream message updates for new metadata
   
   The code path as is not exercised actively and will be leveraged by the PRs 
to follow.
   
   **Tests**: In progress
   
   **API Changes**: 
   - Added new Coordinator stream message type
   - Introduced data model for job coordinator metadata internal to samza
   
   **Upgrade Instructions**: None
   
   **Usage Instructions**: None


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to