Github user sun-rui commented on a diff in the pull request:
https://github.com/apache/spark/pull/12493#discussion_r60394761
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1125,6 +1125,52 @@ setMethod("summarize",
agg(x, ...)
})
+#' dapply
+#'
+#' Apply a function to each partition of a DataFrame.
+#'
+#' @param x A DataFrame
+#' @param func A function to be applied to each partition of the DataFrame.
+#' @param schema Optional. The schema of the resulting DataFrame after the
function is applied.
+#' If NULL, the content of the resulting DataFrame is
serialized R data.
--- End diff --
sure.
If schema is not specified, each partition of the result DataFrame will be
serialized in R into a single byte array. Such resulting DataFrame can be
processed by successive calls to dapply() and collect() on it can be
supported. For now collect() on it only returns RAW data, in future PR we can
deserialize the RAW data into R objects.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
