That should do. Cheers
On Fri, May 22, 2015 at 8:28 PM, Saiph Kappa <saiph.ka...@gmail.com> wrote: > Or should I shutdown the streaming context gracefully and then start it > again with a different number of executors? > > On Sat, May 23, 2015 at 4:00 AM, Saiph Kappa <saiph.ka...@gmail.com> > wrote: > >> Sorry, but I can't see on TD's comments how to allocate executors on >> demand. It seems to me that he's talking about resources within an >> executor, mapping shards to cores. I want to be able to decommission >> executors/workers/machines. >> >> On Sat, May 23, 2015 at 3:31 AM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> For #1, the answer is yes. >>> >>> For #2, See TD's comments on SPARK-7661 >>> >>> Cheers >>> >>> >>> On Fri, May 22, 2015 at 6:58 PM, Saiph Kappa <saiph.ka...@gmail.com> >>> wrote: >>> >>>> Hi, >>>> >>>> 1. Dynamic allocation is currently only supported with YARN, correct? >>>> >>>> 2. In spark streaming, it is possible to change the number of executors >>>> while an application is running? If so, can the allocation be controlled by >>>> the application, instead of using any already defined automatic policy? >>>> That is, I want to be able to get more executors or decommission executors >>>> on demand. Is there some way to achieve this? >>>> >>>> Thanks. >>>> >>> >>> >> >
