Re: UDF with column value comparison fails with PySpark

2016-11-10 Thread Perttu Ranta-aho
So it was something obvious, thanks! -Perttu to 10. marraskuuta 2016 klo 21.19 Davies Liu kirjoitti: > On Thu, Nov 10, 2016 at 11:14 AM, Perttu Ranta-aho > wrote: > > Hello, > > > > I want to create an UDF which modifies one column value depending on >

Re: UDF with column value comparison fails with PySpark

2016-11-10 Thread Davies Liu
On Thu, Nov 10, 2016 at 11:14 AM, Perttu Ranta-aho wrote: > Hello, > > I want to create an UDF which modifies one column value depending on value > of some other column. But Python version of the code fails always in column > value comparison. Below are simple examples, scala

UDF with column value comparison fails with PySpark

2016-11-10 Thread Perttu Ranta-aho
Hello, I want to create an UDF which modifies one column value depending on value of some other column. But Python version of the code fails always in column value comparison. Below are simple examples, scala version works as expected but Python version throws an execption. Am I missing something