Re: Spark 2 + Java + UDF + unknown return type...

2017-02-05 Thread Koert Kuipers
A UDF that does not return a single type is not supported. and spark has no concept of union types. On Feb 2, 2017 16:05, "Jean Georges Perrin" wrote: Hi fellow Sparkans, I am building a UDF (in Java) that can return various data types, basically the signature of the function

Re: Spark 2 + Java + UDF + unknown return type...

2017-02-02 Thread Jörn Franke
Not sure what your udf is exactly doing, but why not on udf / type ? You avoid issues converting it, it is more obvious for the user of your udf etc You could of course return a complex type with one long, one string and one double and you fill them in the udf as needed, but this would be

Spark 2 + Java + UDF + unknown return type...

2017-02-02 Thread Jean Georges Perrin
Hi fellow Sparkans, I am building a UDF (in Java) that can return various data types, basically the signature of the function itself is: public Object call(String a, Object b, String c, Object d, String e) throws Exception When I register my function, I need to provide a type, e.g.: