Hi, I'm trying to run simple UDF functions from Zeppelin and fail. I'm using Zeppelin 0.5.6 and it's not even working on the simple Tutorial example, even though it does work from shell:
This is what I run: def sentiment(s:String) : String = { val positive = Array("like", "love", "good", "great", "happy", "cool", "the", "one", "that") val negative = Array("hate", "bad", "stupid", "is") var st = 0; val words = s.split(" ") positive.foreach(p => words.foreach(w => if(p==w) st = st+1 ) ) negative.foreach(p=> words.foreach(w=> if(p==w) st = st-1 ) ) if(st>0) "positivie" else if(st<0) "negative" else "neutral" } // Below line works only in spark 1.3.0. // For spark 1.1.x and spark 1.2.x, // use sqlc.registerFunction("sentiment", sentiment _) instead. sqlc.udf.register("sentiment", sentiment _) This is the Error I get: import scala.util.matching.Regex import org.apache.spark.sql.functions.udf lengthString: (element_path: String)Int sentiment: (s: String)String lengthString_udf: org.apache.spark.sql.UserDefinedFunction = UserDefinedFunction(<function1>,IntegerType,List(StringType)) res177: org.apache.spark.sql.UserDefinedFunction = UserDefinedFunction(<function1>,StringType,List(StringType)) org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 140.0 failed 4 times, most recent failure: Lost task 0.3 in stage 140.0 (TID 19712, hdn02): java.lang.ClassNotFoundException: $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$anonfun$1 Any idea what causes it and how to fix? Thank you!