[ https://issues.apache.org/jira/browse/SPARK-30445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-30445. ----------------------------------- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27118 [https://github.com/apache/spark/pull/27118] > Accelerator aware scheduling handle setting configs to 0 better > --------------------------------------------------------------- > > Key: SPARK-30445 > URL: https://issues.apache.org/jira/browse/SPARK-30445 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.0.0 > Reporter: Thomas Graves > Assignee: Thomas Graves > Priority: Major > Fix For: 3.0.0 > > > If you set the resource configs to 0, it errors with divide by zero. While I > think ideally the user should just remove the configs we should handle the 0 > better. > > {color:#1d1c1d}$ spark-submit --conf spark.driver.resource.gpu.amount=0 > {color}*--conf spark.executor.resource.gpu.amount=0*{color:#1d1c1d} > {color}*--conf spark.task.resource.gpu.amount=0*{color:#1d1c1d} --conf > spark.driver.resource.gpu.discoveryScript=/shared/tools/get_gpu_resources.sh > --conf > spark.executor.resource.gpu.discoveryScript=/shared/tools/get_gpu_resources.sh > test.py{color} > {color:#1d1c1d}20/01/07 05:36:42 WARN NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes where > applicable{color} > {color:#1d1c1d}Using Spark’s default log4j profile: > org/apache/spark/log4j-defaults.properties{color} > {color:#1d1c1d}20/01/07 05:36:43 INFO SparkContext: {color}*Running Spark > version 3.0.0-preview* > {color:#1d1c1d}20/01/07 05:36:43 INFO ResourceUtils: > =============================================================={color} > {color:#1d1c1d}20/01/07 05:36:43 INFO ResourceUtils: Resources for > spark.driver:{color} > *gpu -> [name: gpu, addresses: 0]* > {color:#1d1c1d}20/01/07 05:36:43 INFO ResourceUtils: > =============================================================={color} > {color:#1d1c1d}20/01/07 05:36:43 INFO SparkContext: Submitted application: > test.py{color} > {color:#1d1c1d}......{color} > {color:#1d1c1d}20/01/07 05:36:43 ERROR SparkContext: Error initializing > SparkContext.{color} > *java.lang.ArithmeticException: / by zero* > {color:#1d1c1d}at > org.apache.spark.SparkContext$.$anonfun$createTaskScheduler$3(SparkContext.scala:2793){color} > {color:#1d1c1d}at > org.apache.spark.SparkContext$.$anonfun$createTaskScheduler$3$adapted(SparkContext.scala:2775){color} > {color:#1d1c1d}at scala.collection.Iterator.foreach(Iterator.scala:941){color} > {color:#1d1c1d}at > scala.collection.Iterator.foreach$(Iterator.scala:941){color} > {color:#1d1c1d}at > scala.collection.AbstractIterator.foreach(Iterator.scala:1429){color} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org