[
https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15110359#comment-15110359
]
Sun Rui commented on SPARK-6817:
--------------------------------
for dapply(), user can call repartition() to set an appropriate number of
partitions before calling dapply().
for gapply(), the SQL conf "spark.sql.shuffle.partitions" could be used to tune
the partitions number after shuffle. I am also hoping SPARK-9850 Adaptive
execution in Spark could help.
> DataFrame UDFs in R
> -------------------
>
> Key: SPARK-6817
> URL: https://issues.apache.org/jira/browse/SPARK-6817
> Project: Spark
> Issue Type: New Feature
> Components: SparkR, SQL
> Reporter: Shivaram Venkataraman
>
> This depends on some internal interface of Spark SQL, should be done after
> merging into Spark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
