Re: UDF returning generic Seq

2016-07-26 Thread Chris Beavers
itrary type. > > > https://issues.apache.org/jira/browse/SPARK-16279 > > > Yong > > > -- > *From:* Chris Beavers > *Sent:* Monday, July 25, 2016 10:32 PM > *To:* user@spark.apache.org > *Subject:* UDF returning generic Seq > > Hey there, > > Interested i

UDF returning generic Seq

2016-07-25 Thread Chris Beavers
Hey there, Interested in writing a UDF that returns an ArrayType column of unknown subtype. My understanding is that this translated JVM-type-wise be a Seq of generic templated type: Seq[Any]. I seem to be hitting the constraint at https://github.com/apache/spark/blob/master/sql/catalyst/src/main/

Re: PySpark Client

2015-01-22 Thread Chris Beavers
lient mode), which would be the first step towards what you > want. > > -Andrew > > 2015-01-20 8:36 GMT-08:00 Chris Beavers : > > Hey all, >> >> Is there any notion of a lightweight python client for submitting jobs to >> a Spark cluster remotely? If I essentia

PySpark Client

2015-01-20 Thread Chris Beavers
Hey all, Is there any notion of a lightweight python client for submitting jobs to a Spark cluster remotely? If I essentially install Spark on the client machine, and that machine has the same OS, same version of Python, etc., then I'm able to communicate with the cluster just fine. But if Python