"spark.eventLog.dir" should contain the full HDFS URL. In general, this should be sufficient:
spark.eventLog.dir=hdfs:/user/spark/applicationHistory On Wed, Apr 8, 2015 at 6:45 AM, Vijayasarathy Kannan <[email protected]> wrote: > I am trying to run a Spark application using spark-submit on a cluster using > Cloudera manager. I get the error > > "Exception in thread "main" java.io.IOException: Error in creating log > directory: file:/user/spark/applicationHistory//app-20150408094126-0008" > > Adding the below lines in /etc/spark/conf/spark-defaults.conf wouldn't > resolve it. > > spark.eventLog.dir=/user/spark/applicationHistory > spark.eventLog.enabled=true > > Any idea on what is missing? > > -- Marcelo --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
