Re: functools.partial as UserDefinedFunction

2015-03-26 Thread Karlson 

Hi,

I've filed a JIRA (https://issues.apache.org/jira/browse/SPARK-6553) and 
suggested a fix (https://github.com/apache/spark/pull/5206).



On 2015-03-25 19:49, Davies Liu wrote:

It’s good to support functools.partial, could you file a JIRA for it?


On Wednesday, March 25, 2015 at 5:42 AM, Karlson wrote:



Hi all,

passing a functools.partial-function as a UserDefinedFunction to
DataFrame.select raises an AttributeException, because 
functools.partial

does not have the attribute __name__. Is there any alternative to
relying on __name__ in pyspark/sql/functions.py:126 ?


-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org 
(mailto:dev-unsubscr...@spark.apache.org)
For additional commands, e-mail: dev-h...@spark.apache.org 
(mailto:dev-h...@spark.apache.org)





-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

functools.partial as UserDefinedFunction

2015-03-25 Thread Karlson 


Hi all,

passing a functools.partial-function as a UserDefinedFunction to 
DataFrame.select raises an AttributeException, because functools.partial 
does not have the attribute __name__. Is there any alternative to 
relying on __name__ in pyspark/sql/functions.py:126 ?



-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org