Hi Eric (or rather: anyone who's experiencing similar situation), I think your problem was, that the /--files/ parameter was provided after the application jar. Your command should have looked like this, instead:
./bin/spark-submit --class edu.bjut.spark.SparkPageRank --master yarn-cluster --num-executors 5 --executor-memory 2g --executor-cores 1 *--files log4j.properties* /data/hadoopspark/MySparkTest.jar hdfs://master:8000/srcdata/searchengine/* 5 5 hdfs://master:8000/resultdata/searchengine/2014102001/ -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/how-to-set-log-level-of-spark-executor-on-YARN-using-yarn-cluster-mode-tp16528p26505.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
