[
https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15095868#comment-15095868
]
Sun Rui commented on SPARK-6817:
--------------------------------
If we think that column-oriented UDF is more important, I can do it with a
higher priority. Just need some help on technical insights. I doubt UDAF is not
exact match, because UDAF only returns only one value for a column. But in R,
operations on a column may still return another non-scalar column. And in order
to seamlessly use existing R functions, a column in question need to be loaded
as a whole into memory as a vector or list, which may suffer OOM if the column
has too many rows.
> DataFrame UDFs in R
> -------------------
>
> Key: SPARK-6817
> URL: https://issues.apache.org/jira/browse/SPARK-6817
> Project: Spark
> Issue Type: New Feature
> Components: SparkR, SQL
> Reporter: Shivaram Venkataraman
> Attachments: SparkR UDF Design Documentation v1.pdf
>
>
> This depends on some internal interface of Spark SQL, should be done after
> merging into Spark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
