[
https://issues.apache.org/jira/browse/SPARK-55757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-55757.
-----------------------------------
Fix Version/s: 4.2.0
Resolution: Fixed
Issue resolved by pull request 54559
[https://github.com/apache/spark/pull/54559]
> Improve `spark.task.cpus` validation
> ------------------------------------
>
> Key: SPARK-55757
> URL: https://issues.apache.org/jira/browse/SPARK-55757
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 4.2.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.2.0
>
>
> Currently, Apache Spark throws `java.lang.ArithmeticException` for the
> invalid `spark.task.cpus`. We had better provide actionable direction.
> **BEFORE**
> {code}
> $ bin/spark-shell -c spark.task.cpus=0
> 26/02/28 09:00:56 ERROR SparkContext: Error initializing SparkContext.
> java.lang.ArithmeticException: / by zero
> at
> org.apache.spark.resource.ResourceUtils$.warnOnWastedResources(ResourceUtils.scala:444)
> {code}
> **AFTER**
> {code}
> $ bin/spark-shell -c spark.task.cpus=0
> ...
> 26/02/28 09:02:15 ERROR SparkContext: Error initializing SparkContext.
> org.apache.spark.SparkIllegalArgumentException:
> [INVALID_CONF_VALUE.REQUIREMENT] The value '0' in the config
> "spark.task.cpus" is invalid. The number of task CPUs must be positive.
> SQLSTATE: 22022
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
