"spark.eventLog.dir" should contain the full HDFS URL. In general, this should be sufficient:
spark.eventLog.dir=hdfs:/user/spark/applicationHistory On Wed, Apr 8, 2015 at 6:45 AM, Vijayasarathy Kannan <kvi...@vt.edu> wrote: > I am trying to run a Spark application using spark-submit on a cluster using > Cloudera manager. I get the error > > "Exception in thread "main" java.io.IOException: Error in creating log > directory: file:/user/spark/applicationHistory//app-20150408094126-0008" > > Adding the below lines in /etc/spark/conf/spark-defaults.conf wouldn't > resolve it. > > spark.eventLog.dir=/user/spark/applicationHistory > spark.eventLog.enabled=true > > Any idea on what is missing? > > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
