Are you running Spark in cluster mode by any chance? (It always helps to show the command line you're actually running, and if there's an exception, the first few frames of the stack trace.)
On Mon, Mar 30, 2015 at 4:11 PM, Tom Hubregtsen <thubregt...@gmail.com> wrote: > Updated spark-defaults and spark-env: > "Log directory /home/hduser/spark/spark-events does not exist." > (Also, in the default /tmp/spark-events it also did not work) > > On 30 March 2015 at 18:03, Marcelo Vanzin <van...@cloudera.com> wrote: >> >> Are those config values in spark-defaults.conf? I don't think you can >> use "~" there - IIRC it does not do any kind of variable expansion. >> >> On Mon, Mar 30, 2015 at 3:50 PM, Tom <thubregt...@gmail.com> wrote: >> > I have set >> > spark.eventLog.enabled true >> > as I try to preserve log files. When I run, I get >> > "Log directory /tmp/spark-events does not exist." >> > I set >> > spark.local.dir ~/spark >> > spark.eventLog.dir ~/spark/spark-events >> > and >> > SPARK_LOCAL_DIRS=~/spark >> > Now I get: >> > "Log directory ~/spark/spark-events does not exist." >> > I am running spark as "hduser", which I also use on the cmd(as verified >> > in >> > the stdout "Set(hduser); users with modify permissions: Set(hduser)"). I >> > am >> > able to cd into this directory. I can also create, view and delete files >> > in >> > this directory, logged in as hduser. I checked the folder, it is owned >> > by >> > hduser. I even performed chmod 777, but Spark keeps on crashing when I >> > run >> > with spark.eventLog.enabled. It works without. Any hints? >> > >> > Thanks, >> > >> > Tom >> > >> > >> > >> > -- >> > View this message in context: >> > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-events-does-not-exist-error-while-it-does-with-all-the-req-rights-tp22308.html >> > Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> > For additional commands, e-mail: user-h...@spark.apache.org >> > >> >> >> >> -- >> Marcelo > > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
