Come across such issue in our project and got it resolved by clearing the space under hdfs directory - "/user/spark". Please check if you have enough space/privileges for this hdfs directory - "/user/spark"
-- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/