[
https://issues.apache.org/jira/browse/SPARK-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15586325#comment-15586325
]
Tobi Bosede commented on SPARK-10915:
-------------------------------------
Hmm...I don't think percent_rank does what I thought. It looks like it returns
the percentile of a value rather than the value at a specific percentile, say
25th or 75th. Really surprised spark has no quantile or percentile function in
the traditional sense like at the below links for python.
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.quantile.html
https://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.percentile.html
> Add support for UDAFs in Python
> -------------------------------
>
> Key: SPARK-10915
> URL: https://issues.apache.org/jira/browse/SPARK-10915
> Project: Spark
> Issue Type: Improvement
> Components: PySpark, SQL
> Reporter: Justin Uang
>
> This should support python defined lambdas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
