Github user shivaram commented on a diff in the pull request:
https://github.com/apache/spark/pull/10220#discussion_r60786195
--- Diff: R/pkg/R/DataFrame.R ---
@@ -1357,17 +1357,46 @@ setMethod("mutate",
function(.data, ...) {
x <- .data
cols <- list(...)
- stopifnot(length(cols) > 0)
+ if (length(cols) <= 0) {
+ return(x)
+ }
+
stopifnot(class(cols[[1]]) == "Column")
+
+ # Check if there is any duplicated column name in the DataFrame
+ dfCols <- columns(x)
+ if (length(unique(dfCols)) != length(dfCols)) {
+ stop("Error: found duplicated column name in the DataFrame")
+ }
+
+ # TODO: simplify the implementation of this method after
SPARK-12225 is resolved.
+
+ # The last column of the same name in the specific columns
takes effect
ns <- names(cols)
- if (!is.null(ns)) {
- for (n in ns) {
- if (n != "") {
- cols[[n]] <- alias(cols[[n]], n)
- }
+ deDupCols <- list()
+ for (i in 1:length(cols)) {
+ if (!is.null(ns) && ns[[i]] != "") {
+ deDupCols[[ns[[i]]]] <- alias(cols[[i]], ns[[i]])
+ } else {
+ # TODO: how to check if there are columns of the same name
in unnamed Columns.
+ deDupCols[[length(deDupCols) + 1]] <- cols[[i]]
--- End diff --
I'm not sure I quite understand when we go into the else case (compared to
what we had before). So if I'm not wrong, before we were just dropping the
non-empty column names and now we want to include them ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
