Repository: spark
Updated Branches:
refs/heads/branch-2.0 15a2dba66 -> bcad1d13f
[SPARK-15439][SPARKR] Failed to run unit test in SparkR
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
There are some failures when running SparkR unit tests.
In this PR, I fixed two of these failures in test_context.R and test_sparkSQL.R
The first one is due to different masked name. I added missed names in the
expected arrays.
The second one is because one PR removed the logic of a previous fix of missing
subset method.
The file privilege issue is still there. I am debugging it. SparkR shell can
run the test case successfully.
test_that("pipeRDD() on RDDs", {
actual <- collect(pipeRDD(rdd, "more"))
When using run-test script, it complains no such directories as below:
cannot open file '/tmp/Rtmp4FQbah/filee2273f9d47f7': No such file or directory
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests,
manual tests)
Manually test it
Author: [email protected] <[email protected]>
Closes #13284 from wangmiao1981/R.
(cherry picked from commit 06bae8af17d9478c889d206a4556a697b5d629e7)
Signed-off-by: Shivaram Venkataraman <[email protected]>
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bcad1d13
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bcad1d13
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bcad1d13
Branch: refs/heads/branch-2.0
Commit: bcad1d13f58a119948e3374072824f70a14a6d34
Parents: 15a2dba
Author: [email protected] <[email protected]>
Authored: Wed May 25 21:08:03 2016 -0700
Committer: Shivaram Venkataraman <[email protected]>
Committed: Wed May 25 21:08:17 2016 -0700
----------------------------------------------------------------------
R/pkg/R/DataFrame.R | 6 +++++-
R/pkg/inst/tests/testthat/test_context.R | 6 +++++-
2 files changed, 10 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/bcad1d13/R/pkg/R/DataFrame.R
----------------------------------------------------------------------
diff --git a/R/pkg/R/DataFrame.R b/R/pkg/R/DataFrame.R
index 0c2a194..f719173 100644
--- a/R/pkg/R/DataFrame.R
+++ b/R/pkg/R/DataFrame.R
@@ -1445,7 +1445,11 @@ setMethod("[", signature(x = "SparkDataFrame"),
#' }
setMethod("subset", signature(x = "SparkDataFrame"),
function(x, subset, select, drop = F, ...) {
- x[subset, select, drop = drop]
+ if (missing(subset)) {
+ x[, select, drop = drop, ...]
+ } else {
+ x[subset, select, drop = drop, ...]
+ }
})
#' Select
http://git-wip-us.apache.org/repos/asf/spark/blob/bcad1d13/R/pkg/inst/tests/testthat/test_context.R
----------------------------------------------------------------------
diff --git a/R/pkg/inst/tests/testthat/test_context.R
b/R/pkg/inst/tests/testthat/test_context.R
index 0e5e15c..95258ba 100644
--- a/R/pkg/inst/tests/testthat/test_context.R
+++ b/R/pkg/inst/tests/testthat/test_context.R
@@ -27,6 +27,11 @@ test_that("Check masked functions", {
namesOfMasked <- c("describe", "cov", "filter", "lag", "na.omit", "predict",
"sd", "var",
"colnames", "colnames<-", "intersect", "rank", "rbind",
"sample", "subset",
"summary", "transform", "drop", "window", "as.data.frame")
+ namesOfMaskedCompletely <- c("cov", "filter", "sample")
+ if (as.numeric(R.version$major) == 3 && as.numeric(R.version$minor) > 2) {
+ namesOfMasked <- c("endsWith", "startsWith", namesOfMasked)
+ namesOfMaskedCompletely <- c("endsWith", "startsWith",
namesOfMaskedCompletely)
+ }
expect_equal(length(maskedBySparkR), length(namesOfMasked))
expect_equal(sort(maskedBySparkR), sort(namesOfMasked))
# above are those reported as masked when `library(SparkR)`
@@ -36,7 +41,6 @@ test_that("Check masked functions", {
any(grepl("=\"ANY\"",
capture.output(showMethods(x)[-1])))
}))
maskedCompletely <- masked[!funcHasAny]
- namesOfMaskedCompletely <- c("cov", "filter", "sample")
expect_equal(length(maskedCompletely), length(namesOfMaskedCompletely))
expect_equal(sort(maskedCompletely), sort(namesOfMaskedCompletely))
})
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
