Solved, is SPARK_PID_DIR from spark-env.sh. Changing this directory from /tmp to smthg different actually changed the error that I got, now showing where the actual error was coming from (a null pointer in my program). The first error was not helpful at all though.
-- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Shuffle-strange-error-tp23179p23180.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
