Sorry, but I can't see on TD's comments how to allocate executors on demand. It seems to me that he's talking about resources within an executor, mapping shards to cores. I want to be able to decommission executors/workers/machines.
On Sat, May 23, 2015 at 3:31 AM, Ted Yu <yuzhih...@gmail.com> wrote: > For #1, the answer is yes. > > For #2, See TD's comments on SPARK-7661 > > Cheers > > > On Fri, May 22, 2015 at 6:58 PM, Saiph Kappa <saiph.ka...@gmail.com> > wrote: > >> Hi, >> >> 1. Dynamic allocation is currently only supported with YARN, correct? >> >> 2. In spark streaming, it is possible to change the number of executors >> while an application is running? If so, can the allocation be controlled by >> the application, instead of using any already defined automatic policy? >> That is, I want to be able to get more executors or decommission executors >> on demand. Is there some way to achieve this? >> >> Thanks. >> > >
