[GitHub] [spark] maropu commented on a change in pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

[GitBox]

maropu commented on a change in pull request #29054:
URL: https://github.com/apache/spark/pull/29054#discussion_r452758095



##
File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala
##
@@ -69,49 +69,56 @@ private[sql] class HiveSessionCatalog(
 // Current thread context classloader may not be the one loaded the class. 
Need to switch
 // context classloader to initialize instance properly.
 Utils.withContextClassLoader(clazz.getClassLoader) {
-  Try(super.makeFunctionExpression(name, clazz, input)).getOrElse {
-var udfExpr: Option[Expression] = None
-try {
-  // When we instantiate hive UDF wrapper class, we may throw 
exception if the input
-  // expressions don't satisfy the hive UDF, such as type mismatch, 
input number
-  // mismatch, etc. Here we catch the exception and throw 
AnalysisException instead.
-  if (classOf[UDF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveSimpleUDF(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if (classOf[GenericUDF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveGenericUDF(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if 
(classOf[AbstractGenericUDAFResolver].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveUDAFFunction(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if (classOf[UDAF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveUDAFFunction(
-  name,
-  new HiveFunctionWrapper(clazz.getName),
-  input,
-  isUDAFBridgeRequired = true))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if (classOf[GenericUDTF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveGenericUDTF(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.asInstanceOf[HiveGenericUDTF].elementSchema // Force 
it to check data types.
+  Try(super.makeFunctionExpression(name, clazz, input)) match {

Review comment:
   If so, how about using `try` instead of `Try`?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

[GitBox]

maropu commented on a change in pull request #29054:
URL: https://github.com/apache/spark/pull/29054#discussion_r452627861



##
File path: 
sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDAFSuite.scala
##
@@ -161,6 +161,21 @@ class HiveUDAFSuite extends QueryTest
   checkAnswer(sql("select histogram_numeric(a,2) from abc where a=3"), 
Row(null))
 }
   }
+
+  test("Hive mode use spark udaf should show error") {
+val functionName = "longProductSum"
+val functionClass = "org.apache.spark.sql.hive.execution.LongProductSum"
+withUserDefinedFunction(functionName -> true) {
+  sql(s"CREATE TEMPORARY FUNCTION $functionName AS '$functionClass'")
+  val e1 = intercept[AnalysisException] {

Review comment:
   nit: `e1` -> `e`





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] [spark] maropu commented on a change in pull request #29054: [SPARK-32243][SQL]HiveSessionCatalog call super.makeFunctionExpression should show error message

[GitBox]

maropu commented on a change in pull request #29054:
URL: https://github.com/apache/spark/pull/29054#discussion_r452627431



##
File path: 
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala
##
@@ -69,49 +69,56 @@ private[sql] class HiveSessionCatalog(
 // Current thread context classloader may not be the one loaded the class. 
Need to switch
 // context classloader to initialize instance properly.
 Utils.withContextClassLoader(clazz.getClassLoader) {
-  Try(super.makeFunctionExpression(name, clazz, input)).getOrElse {
-var udfExpr: Option[Expression] = None
-try {
-  // When we instantiate hive UDF wrapper class, we may throw 
exception if the input
-  // expressions don't satisfy the hive UDF, such as type mismatch, 
input number
-  // mismatch, etc. Here we catch the exception and throw 
AnalysisException instead.
-  if (classOf[UDF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveSimpleUDF(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if (classOf[GenericUDF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveGenericUDF(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if 
(classOf[AbstractGenericUDAFResolver].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveUDAFFunction(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if (classOf[UDAF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveUDAFFunction(
-  name,
-  new HiveFunctionWrapper(clazz.getName),
-  input,
-  isUDAFBridgeRequired = true))
-udfExpr.get.dataType // Force it to check input data types.
-  } else if (classOf[GenericUDTF].isAssignableFrom(clazz)) {
-udfExpr = Some(HiveGenericUDTF(name, new 
HiveFunctionWrapper(clazz.getName), input))
-udfExpr.get.asInstanceOf[HiveGenericUDTF].elementSchema // Force 
it to check data types.
+  Try(super.makeFunctionExpression(name, clazz, input)) match {

Review comment:
   +1





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org