RE: Spark SQL UDF returning a list?

2014-12-03 Thread Cheng, Hao
, 2014 9:05 AM To: Jerry Raj Cc: user Subject: Re: Spark SQL UDF returning a list? Hi, On Wed, Dec 3, 2014 at 4:31 PM, Jerry Raj mailto:jerry@gmail.com>> wrote: Exception in thread "main" java.lang.RuntimeException: [1.57] failure: ``('' expected but identifier

Re: Spark SQL UDF returning a list?

2014-12-03 Thread Tobias Pfeiffer
Hi, On Wed, Dec 3, 2014 at 4:31 PM, Jerry Raj wrote: > > Exception in thread "main" java.lang.RuntimeException: [1.57] failure: > ``('' expected but identifier myudf found > > I also tried returning a List of Ints, that did not work either. Is there > a way to write a UDF that returns a list? >

Spark SQL UDF returning a list?

2014-12-03 Thread Jerry Raj
Hi, Can a UDF return a list of values that can be used in a WHERE clause? Something like: sqlCtx.registerFunction("myudf", { Array(1, 2, 3) }) val sql = "select doc_id, doc_value from doc_table where doc_id in myudf()" This does not work: Exception in thread "main" java.l