wuyi created SPARK-32238:
----------------------------
Summary: "Malformed class name" error from ScalaUDF
Key: SPARK-32238
URL: https://issues.apache.org/jira/browse/SPARK-32238
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.0, 2.4.0
Reporter: wuyi
{code:java}
object MalformedClassObject extends Serializable {
class MalformedFunction extends (String => Int) with Serializable {
override def apply(v1: String): Int = v1.toInt / 0
}
}
Seq("20").toDF("col").select(udf(new
MalformedFunction).apply(Column("col"))).collect()
An exception or error caused a run to abort: Malformed class name
java.lang.InternalError: Malformed class name
at java.lang.Class.getSimpleName(Class.java:1330)
at
org.apache.spark.sql.catalyst.expressions.ScalaUDF.udfErrorMessage$lzycompute(ScalaUDF.scala:1157)
at
org.apache.spark.sql.catalyst.expressions.ScalaUDF.udfErrorMessage(ScalaUDF.scala:1155)
at
org.apache.spark.sql.catalyst.expressions.ScalaUDF.doGenCode(ScalaUDF.scala:1077)
at
org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$genCode$3(Expression.scala:147)
at scala.Option.getOrElse(Option.scala:189)
at
org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:142)
at
org.apache.spark.sql.catalyst.expressions.Alias.genCode(namedExpressions.scala:160)
at
org.apache.spark.sql.execution.ProjectExec.$anonfun$doConsume$1(basicPhysicalOperators.scala:69)
....
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
