Re: should I file a bug? Re: trouble implementing Transformer and calling DataFrame.withColumn()

2015-12-22 Thread Jeff Zhang
a code. Here is my > original python > > binomial = udf(lambda labelStr: labelStr if (labelStr == "noise") else > “signal", StringType()) > ret = dataFrame.withColumn(newColName, binomial(dataFrame["label"])) > > > Any suggestions would be greatly appreciat

Re: trouble implementing Transformer and calling DataFrame.withColumn()

2015-12-21 Thread Jeff Zhang
In your case, I would suggest you to extends UnaryTransformer which is much easier. Yeah, I have to admit that there's no document about how to write a custom Transformer, I think we need to add that, since writing custom Transformer is a very typical work in machine learning. On Tue, Dec 22,

trouble implementing Transformer and calling DataFrame.withColumn()

2015-12-21 Thread Andy Davidson
I am trying to port the following python function to Java 8. I would like my java implementation to implement Transformer so I can use it in a pipeline. I am having a heck of a time trying to figure out how to create a Column variable I can pass to DataFrame.withColumn(). As far as I know