Valentin Ursu created ZEPPELIN-1524:
---------------------------------------
Summary: User defined functions in Scala cannot be applied
Key: ZEPPELIN-1524
URL: https://issues.apache.org/jira/browse/ZEPPELIN-1524
Project: Zeppelin
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Valentin Ursu
Related to stackoverflow question
http://stackoverflow.com/questions/39857803/class-not-found-when-applying-udf-in-apache-zeppelin
and issue ZEPPELIN-150
Applying an UDF in the scala interpreter generates a class not found exception.
The following code works as expected in spark-shell (both 1.6.0 and 2.0.0) and
can be used to reproduce the issue in Zeppelin 0.6.0:
import org.apache.spark.sql.functions._
val data = sqlContext.read.json(sc.parallelize(Seq("{'location' : 'london'}",
"{'location': 'tokyo'}")))
val cities = Seq("london", "paris")
val urbf = udf { city: String => if (cities.contains(city)) 1 else 0 }
sqlc.udf.register("sentiment", sentiment _)
data.select($"location", urbf($"location")).show
The error is:
java.lang.ClassNotFoundException:
$iwC$$iwC$$iwC$$iwC$$iwC$$$$cd8ca64e19a07f9732e2b3de5fd5fc88$$$$$iwC$$iwC$$iwC$$iwC$$anonfun$1
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
