dongjoon-hyun commented on code in PR #52238:
URL: https://github.com/apache/spark/pull/52238#discussion_r2323163698
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/PythonUDF.scala:
##########
@@ -51,6 +51,27 @@ object PythonUDF {
// support new types in the future, e.g, N -> N transform.
e.isInstanceOf[PythonUDAF]
}
+
+ def correctEvalType(udf: PythonUDF): Int = {
+ if (udf.evalType == PythonEvalType.SQL_ARROW_BATCHED_UDF) {
+ if (conf.pythonUDFArrowFallbackOnUDT &&
Review Comment:
Can we simply get `conf.pythonUDFArrowFallbackOnUDT` as a parameter? It
looks too much for me to bring `SQLConfHelper` simply in order to use `conf` in
this object. WDYT, @peter-toth ?
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/PythonUDF.scala:
##########
@@ -51,6 +51,27 @@ object PythonUDF {
// support new types in the future, e.g, N -> N transform.
e.isInstanceOf[PythonUDAF]
}
+
+ def correctEvalType(udf: PythonUDF): Int = {
+ if (udf.evalType == PythonEvalType.SQL_ARROW_BATCHED_UDF) {
+ if (conf.pythonUDFArrowFallbackOnUDT &&
Review Comment:
Can we simply get the value of `conf.pythonUDFArrowFallbackOnUDT` as a
parameter? It looks too much for me to bring `SQLConfHelper` simply in order to
use `conf` in this object. WDYT, @peter-toth ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
