Re: Using UDFs in Java without registration

Would like to bring this back for consideration again. I'm open to adding types for all the parameters, but it does seem onerous, and in the case of Python, we don't do that. Do you feel strongly about adding them? On Sat, May 30, 2015 at 8:04 PM Reynold Xin wrote: > We

Re: Using UDFs in Java without registration

We added all the typetags for arguments but haven't got around to use them yet. I think it'd make sense to have them and do the auto cast, but we can have rules in analysis to forbid certain casts (e.g. don't auto cast double to int). On Sat, May 30, 2015 at 7:12 AM, Justin Uang

Re: Using UDFs in Java without registration

I think you are right that there is no way to call Java UDF without registration right now. Adding another 20 methods to functions would be scary. Maybe the best way is to have a companion object for UserDefinedFunction, and define UDF there? e.g. object UserDefinedFunction { def define(f:

Re: Using UDFs in Java without registration

The idea of asking for both the argument and return class is interesting. I don't think we do that for the scala APIs currently, right? In functions.scala, we only use the TypeTag for RT. def udf[RT: TypeTag, A1: TypeTag](f: Function1[A1, RT]): UserDefinedFunction = { UserDefinedFunction(f,

Using UDFs in Java without registration

I would like to define a UDF in Java via a closure and then use it without registration. In Scala, I believe there are two ways to do this: myUdf = functions.udf({ _ + 5}) myDf.select(myUdf(myDf(age))) or myDf.select(functions.callUDF({_ + 5}, DataTypes.IntegerType, myDf(age)))