lakshmi-manasa-g opened a new pull request #1448: URL: https://github.com/apache/samza/pull/1448
Feature: Main feature is YARN AM high availability. The feature ensures that the new AM can establish connection with already running containers to avoid restarting all running containers when AM dies. This PR enables the new AM to accept the list fo already running container provided by the resource manager and launch only those containers that are part of the job model but not in the running container list. Changes: 1. ClientHelper: job submit to RM indicates to keep containers alive across attempts 2. SamzaYarnAppMasterLifecycle: new AM uses the yarnid-samza id mapping and accepts the list of running containers given by RM and builds its internal state (SamzaApplicationState and YarnAppState) correctly 3. ContainerProcessManager: removes running containers from the needed processor list prior to placing resource requests Tests: added unit tests and deployed hello-samza API changes: None Usage instructions: to enable AM HA set config "yarn.am.high-availability.enabled" to "true", default value is "false". Upgrade instructions: None ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
