This is an automated email from the ASF dual-hosted git repository.
maxgekk pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new fea905acea2 [SPARK-41139][SQL] Improve error class:
`PYTHON_UDF_IN_ON_CLAUSE`
fea905acea2 is described below
commit fea905acea2e8eedb10f86d4cea6565f19066023
Author: itholic <[email protected]>
AuthorDate: Wed Nov 16 19:13:52 2022 +0300
[SPARK-41139][SQL] Improve error class: `PYTHON_UDF_IN_ON_CLAUSE`
### What changes were proposed in this pull request?
This PR proposes to improve the error message and test for
`PYTHON_UDF_IN_ON_CLAUSE`
### Why are the changes needed?
The current error message is not clear enough to let user understand the
solve the problem.
We can provide more information to improve the usability.
Also, we should test the error class with `checkError` for better
testability.
### Does this PR introduce _any_ user-facing change?
The error message is improved with additional detailed information.
From
```
Python UDF in the ON clause of a <joinType> JOIN.
```
To
```
Python UDF in the ON clause of a <joinType> JOIN. In case of an INNNER JOIN
consider rewriting to a CROSS JOIN with a WHERE clause.
```
### How was this patch tested?
Manually tested for fixed test case.
Closes #38657 from itholic/SPARK-41139.
Authored-by: itholic <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
---
core/src/main/resources/error/error-classes.json | 2 +-
.../optimizer/ExtractPythonUDFFromJoinConditionSuite.scala | 8 +++++---
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/core/src/main/resources/error/error-classes.json
b/core/src/main/resources/error/error-classes.json
index 32083c23df8..d5d6e938ad1 100644
--- a/core/src/main/resources/error/error-classes.json
+++ b/core/src/main/resources/error/error-classes.json
@@ -1248,7 +1248,7 @@
},
"PYTHON_UDF_IN_ON_CLAUSE" : {
"message" : [
- "Python UDF in the ON clause of a <joinType> JOIN."
+ "Python UDF in the ON clause of a <joinType> JOIN. In case of an
INNNER JOIN consider rewriting to a CROSS JOIN with a WHERE clause."
]
},
"REPEATED_PIVOT" : {
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ExtractPythonUDFFromJoinConditionSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ExtractPythonUDFFromJoinConditionSuite.scala
index 0b215818d36..854a3e8f7a7 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ExtractPythonUDFFromJoinConditionSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/ExtractPythonUDFFromJoinConditionSuite.scala
@@ -187,9 +187,11 @@ class ExtractPythonUDFFromJoinConditionSuite extends
PlanTest {
condition = Some(unevaluableJoinCond))
Optimize.execute(query.analyze)
}
- assert(e.message ==
- "[UNSUPPORTED_FEATURE.PYTHON_UDF_IN_ON_CLAUSE] The feature is not
supported: " +
- s"""Python UDF in the ON clause of a ${joinType.sql} JOIN.""")
+ checkError(
+ exception = e,
+ errorClass = "UNSUPPORTED_FEATURE.PYTHON_UDF_IN_ON_CLAUSE",
+ parameters = Map("joinType" -> joinType.sql)
+ )
val query2 = testRelationLeft.join(
testRelationRight,
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
