Re: Dynamic allocation not releasing executors after unpersisting all cached data

2018-07-10 Thread Jeffrey Charles
Thanks for the suggestion. I gave it a try but the executor still isn't being released several minutes after running that. On Mon, Jul 9, 2018 at 3:51 PM Vadim Semenov wrote: > Try doing `unpersist(blocking=true)` > On Mon, Jul 9, 2018 at 2:59 PM Jeffrey Charles > wrote: > > > > I'm persisting

Re: Dynamic allocation not releasing executors after unpersisting all cached data

2018-07-09 Thread Vadim Semenov
Try doing `unpersist(blocking=true)` On Mon, Jul 9, 2018 at 2:59 PM Jeffrey Charles wrote: > > I'm persisting a dataframe in Zeppelin which has dynamic allocation enabled > to get a sense of how much memory the dataframe takes up. After I note the > size, I unpersist the dataframe. For some

Dynamic allocation not releasing executors after unpersisting all cached data

2018-07-09 Thread Jeffrey Charles
I'm persisting a dataframe in Zeppelin which has dynamic allocation enabled to get a sense of how much memory the dataframe takes up. After I note the size, I unpersist the dataframe. For some reason, Yarn is not releasing the executors that were added to Zeppelin. If I don't run the persist and