Spark 2.4.0 worker can't find work/app/folderNo directory for logs

2018-11-29 Thread flyingmeatball
I'm running a 10 node standalone cluster and I'm having issues with a stage completing - it keeps hanging somewhere between 196 and 199/200 blocks completed, but never errors and doesn't move forward. If I look at the

Spark 2.4.0 worker can't find work/app/folderNo directory for logs

2018-11-29 Thread flyingmeatball
I'm running a 10 node standalone cluster and I'm having issues with a stage completing - it keeps hanging somewhere between 196 and 199/200 blocks completed, but never errors and doesn't move forward. If I look at the