The intent to move a container is to improve service-to-service locality, co-locate services to reduce bottlenecks introduced by network. The idea that a certain parallel process should be moved comes from an external service.
Speculation might not be effective here as I see, since the intent to move a container can be triggered hourly, daily, weekly or monthly - containers contain long running services, data pipelines. Or am I on the wrong line here? Can I simulate something like this with the current speculation? Thanks for help! On Wed, Mar 30, 2016 at 11:17 PM Eric Payne <[email protected]> wrote: > I think it would help if I knew what the criteria is for wanting to move > the container. In other words, was the container started on an undesirable > node in the first place? Or, did the node become undesirable after the > container started. > > Speculation could be considered a "move" operation for containers. If a > container isn't finishing fast enough, the default speculator will start > another container on a different node. Would it be possible to create a > specialized speculator that understood your criteria for needing to move > containers? If so, it could be done automatically / programatically. > > Thanks, > -Eric > > > ------------------------------ > *From:* Zoltán Zvara <[email protected]> > *To:* Vinod Kumar Vavilapalli <[email protected]> > *Cc:* [email protected] > *Sent:* Wednesday, March 30, 2016 9:10 AM > *Subject:* Re: YARN re-locate container > > How is this achieved? As far as I see it now, after stopping a container, > the AM must reallocate the same container with the same resource vector but > with locality preferences pointed to the new, target node. After the new > leash has been acquired, then the AM can take it to the new node and > initiate a `startContainers` message. > Our use-case with Ericsson would require a more simple API, where (for > example) a `moveContainer` call from the AM would ask the RM or NM to move > a container from one node to another (or to any of the specified set of > preferred nodes). Move would simply kill the container and restart it on > another node at any given time whenever it is possible - I feel questions > around scheduling: how container moves should be handled? Probably not like > simple allocations. > > Am I understanding the architecture correctly here? > > On Tue, Mar 29, 2016 at 7:31 PM Vinod Kumar Vavilapalli < > [email protected]> wrote: > > Containers can be restarted on other machines already today - YARN just > leaves it up to the applications to do so. > > Are you looking for anything more specifically? > > +Vinod > > > On Mar 29, 2016, at 9:45 AM, Zoltán Zvara <[email protected]> > wrote: > > > > Dear Hadoop Community, > > > > Is there any feature available, or on the road map to support the > relocation of containers? (Simply restart the container on another machine.) > > > > Thanks, > > Zoltán > > > >
