Klaus Ma created SPARK-28673: -------------------------------- Summary: Failed to start executors for spark-sql-perf on kubernetes Key: SPARK-28673 URL: https://issues.apache.org/jira/browse/SPARK-28673 Project: Spark Issue Type: Improvement Components: Kubernetes, SQL Affects Versions: 2.4.3 Reporter: Klaus Ma
Start Spark SQL Perf test with following command, the `spark.executor.instances` is 10, but no executors was start :( {code:java} spark-submit \ --master k8s://https://XXX \ --deploy-mode cluster \ --name k82cn-spark-test-per \ --conf "spark.local.dir=/myfs/tmp" \ --conf "spark.executor.extraJavaOptions=-Djava.io.tmpdir=/myfs/tmp" \ --conf "spark.driver.extraJavaOptions=-Djava.io.tmpdir=/myfs/tmp" \ --class org.apache.spark.sql.execution.benchmark.TPCDSQueryBenchmark \ --conf spark.kubernetes.volcano.enable=false \ --conf spark.driver.memory=3G \ --conf spark.executor.instances=10 \ --conf spark.kubernetes.container.image.pullSecrets=default-secret \ --conf spark.kubernetes.container.image=XXX/spark-test:origin-4 \ local:///opt/spark/jars/spark-sql_2.11-2.4.4-SNAPSHOT-tests.jar \ --data-location file://XXXXX/spark-tpcds-data --query-filter "q2,q5" {code} -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org