Re: [Analytics] SparkContext stopped and cannot be restarted

Hey Neil, there were two Yarn jobs running related to your notebooks, I just killed them, let's see if it solves the problem (you might need to restart again your notebook). If not, let's open a task and investigate :) Luca Il giorno gio 6 feb 2020 alle ore 02:08 Neil Shah-Quinn < nshahqu...@wik

Re: [Analytics] SparkContext stopped and cannot be restarted

Whoa—I just got the same stopped SparkContext error on the query even after restarting the notebook, without an intermediate Java heap space error. That seems very strange to me. On Wed, 5 Feb 2020 at 16:09, Neil Shah-Quinn wrote: > Hey there! > > I was running SQL queries via PySpark (using the

[Analytics] SparkContext stopped and cannot be restarted

Hey there! I was running SQL queries via PySpark (using the wmfdata package ) on SWAP when one of my queries failed with "java.lang.OutofMemoryError: Java heap space". After that, when I tried to call the spark.sql function again