[
https://issues.apache.org/jira/browse/SPARK-33539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allison Wang updated SPARK-33539:
---------------------------------
Description:
In the SPIP: Standardize Exception Messages in Spark, there are three major
improvements proposed:
# Group error messages in dedicated files.
# Establish an error message guideline for developers.
# Improve error message quality.
The first step is to centralize error messages for each component into its own
dedicated file(s). This can help with auditing error messages and subsequent
tasks to establish a guideline and improve message quality in the future.
A general rule of thumb for grouping exceptions:
* AnalysisException => QueryCompilationErrors
* SparkException, RuntimeException(UnsupportedOperationException,
IllegalStateException...) => QueryExecutionErrors
* ParseException => QueryParsingErrors
(A special case for `Command`: since commands are executed eagerly, users can
immediately see the errors even if the exceptions are thrown in
`SparkPlan.execute`. In this case, we should group the exceptions in commands
as QueryCompilationErrors.)
Here is an example RP to group all `AnalysisExcpetion` in Analyzer into
QueryCompilationErrors: [SPARK-32670|https://github.com/apache/spark/pull/29497]
Please see the
[SPIP|https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing]
for more details.
Exceptions per component:
||component||exception||
|sql|714|
|core|334|
|mllib|161|
|streaming|43|
|resource-managers|42|
|external|35|
|examples|20|
|mllib-local|10|
|graphx|6|
|repl|5|
|hadoop-cloud|1|
was:
In the SPIP: Standardize Exception Messages in Spark, there are three major
improvements proposed:
# Group error messages in dedicated files.
# Establish an error message guideline for developers.
# Improve error message quality.
The first step is to centralize error messages for each component into its own
dedicated file(s). This can help with auditing error messages and subsequent
tasks to establish a guideline and improve message quality in the future.
A general rule of thumb for grouping exceptions:
* AnalysisException => QueryCompilationErrors
* SparkException, RuntimeException(UnsupportedOperationException,
IllegalStateException...) => QueryExecutionErrors
* ParseException => QueryParsingErrors
Here is an example RP to group all `AnalysisExcpetion` in Analyzer into
QueryCompilationErrors: [SPARK-32670|https://github.com/apache/spark/pull/29497]
Please see the
[SPIP|https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing]
for more details.
Exceptions per component:
||component||exception||
|sql|714|
|core|334|
|mllib|161|
|streaming|43|
|resource-managers|42|
|external|35|
|examples|20|
|mllib-local|10|
|graphx|6|
|repl|5|
|hadoop-cloud|1|
> Standardize exception messages in Spark
> ---------------------------------------
>
> Key: SPARK-33539
> URL: https://issues.apache.org/jira/browse/SPARK-33539
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core, SQL
> Affects Versions: 3.1.0
> Reporter: Allison Wang
> Priority: Major
>
> In the SPIP: Standardize Exception Messages in Spark, there are three major
> improvements proposed:
> # Group error messages in dedicated files.
> # Establish an error message guideline for developers.
> # Improve error message quality.
> The first step is to centralize error messages for each component into its
> own dedicated file(s). This can help with auditing error messages and
> subsequent tasks to establish a guideline and improve message quality in the
> future.
> A general rule of thumb for grouping exceptions:
> * AnalysisException => QueryCompilationErrors
> * SparkException, RuntimeException(UnsupportedOperationException,
> IllegalStateException...) => QueryExecutionErrors
> * ParseException => QueryParsingErrors
> (A special case for `Command`: since commands are executed eagerly, users can
> immediately see the errors even if the exceptions are thrown in
> `SparkPlan.execute`. In this case, we should group the exceptions in commands
> as QueryCompilationErrors.)
> Here is an example RP to group all `AnalysisExcpetion` in Analyzer into
> QueryCompilationErrors:
> [SPARK-32670|https://github.com/apache/spark/pull/29497]
> Please see the
> [SPIP|https://docs.google.com/document/d/1XGj1o3xAFh8BA7RCn3DtwIPC6--hIFOaNUNSlpaOIZs/edit?usp=sharing]
> for more details.
> Exceptions per component:
> ||component||exception||
> |sql|714|
> |core|334|
> |mllib|161|
> |streaming|43|
> |resource-managers|42|
> |external|35|
> |examples|20|
> |mllib-local|10|
> |graphx|6|
> |repl|5|
> |hadoop-cloud|1|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
