spark.eventLog.dir should contain the full HDFS URL. In general,
this should be sufficient:
spark.eventLog.dir=hdfs:/user/spark/applicationHistory
On Wed, Apr 8, 2015 at 6:45 AM, Vijayasarathy Kannan kvi...@vt.edu wrote:
I am trying to run a Spark application using spark-submit on a cluster using
Cloudera manager. I get the error
Exception in thread main java.io.IOException: Error in creating log
directory: file:/user/spark/applicationHistory//app-20150408094126-0008
Adding the below lines in /etc/spark/conf/spark-defaults.conf wouldn't
resolve it.
spark.eventLog.dir=/user/spark/applicationHistory
spark.eventLog.enabled=true
Any idea on what is missing?
--
Marcelo
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
