Hi ,
When I am submitting spark job as --master yarn-cluster with below
command/options getting driver
memory error-
spark-submit --jars
./libs/mysql-connector-java-5.1.17.jar,./libs/log4j-1.2.17.jar --files
datasource.properties,log4j.properties --master yarn-cluster --num-executors
1
Try this
./bin/spark-submit -v --master yarn-cluster --jars
./libs/mysql-connector-java-5.1.17.jar,./libs/log4j-1.2.17.jar --files
datasource.properties,log4j.properties --num-executors 1 --driver-memory
4g *--driver-java-options -XX:MaxPermSize=1G* --executor-memory 2g
--executor-cores 1
