[CONF] Apache Samza > SEP-22: Container Placements in Samza

[Sanil Jain (Confluence)]

Title: Message Title

There's 6 inline comment updates and 1 new edit on this page

SEP-22: Container Placements in Samza 6 inline comment updates Boris Shkolnik Hot StandBy It'd be nice to provide a link to a page that describe this feature. Sanil Jain Done added: SEP-19: Hot standby state for Samza applications View comment • Like Boris Shkolnik support for Canar please provide a short description on how this will work with Canary. Sanil Jain Please see the options in 2.1 after API description View comment • Like Boris Shkolnik deployment id what will be used as the deployment id? Sanil Jain "run.id" generated by the Job planner View comment • Like Sanil Jain resolved we attempt to start t do we really need to something special here? If the container is down, can't we just let Yarn take care of this? View comment Sanil Jain rewording this to fallback to source host View comment • Like Boris Shkolnik ContainerProcessManager can you please clarify why do we need a separate thread for the notifications, instead of callbacks to the caller? Sanil Jain ContainerPlacementRequestAllocator thread is periodically reading from metastore and relaying control messages to the ContainerManager  Container allocator thread is periodically trying to fulfill each request with allocated resources from the ClusterManager We do not need a seprate thread for notifications! View comment • Like 1 new edit Sanil Jain edited this page Here's what changed: ... Selectively Spin StandBy Containers: Samza has a feature of Hot StandBy Containers for reducing stateful restoration times. Enabling this feature for a job involves doubling the containers at the least (simplest case where every container has 1 standby replica enabled). Customers are reluctant to enable this since doubling the containers increases the cost to serve. To improve the adoption for this feature we can build the ability to spin up StandBy Containers for a single or a subset of containers while the job is running, these StandBy Containers then can be used for failover to reduce downtime.   ... If the preferred resources are not able to be acquired the active container is never stopped and a failure notification is sent for the ContainerPacementRequest If the ContainerPlacementManager is not able to stop the active container (3.1 #1 above fails) in that  case the request is marked failed & a failure notification is sent for the ContainerPacementRequest If ClusterResourceManager fails to start the stopped active container on the accrued destination host, then we attempt to start the container back on the container fallbacks to source host and a failure notification is are sent for the ContainerPacementRequest. If a container fails to start on source host then an attempt is made to start on ANY_HOST ... Go to page history View page

Stop watching space • Manage notifications This message was sent by Atlassian Confluence 7.1.2