[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15302525#comment-15302525 ] Shivaram Venkataraman commented on SPARK-15439: --- Hmm - I think the check needs to be `R.version$minor >= 3` and similarly the major version should be `>=3` ? > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang >Assignee: Miao Wang > Fix For: 2.0.0 > > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[
https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15301735#comment-15301735
]
Felix Cheung commented on SPARK-15439:
--
I think we have an issue here - I"m running R 3.2.2 and the mask tests are
failing because:
> R.version$minor
[1] "2.2"
And this is not strict enough?
if (as.numeric(R.version$major) == 3 && as.numeric(R.version$minor) > 2) {
namesOfMasked <- c("endsWith", "startsWith", namesOfMasked)
namesOfMaskedCompletely <- c("endsWith", "startsWith",
namesOfMaskedCompletely)
}
> Failed to run unit test in SparkR
> -
>
> Key: SPARK-15439
> URL: https://issues.apache.org/jira/browse/SPARK-15439
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.0.0
>Reporter: Kai Jiang
>Assignee: Miao Wang
> Fix For: 2.0.0
>
>
> Failed to run ./R/run-tests.sh around recent commit (May 19, 2016)
> It might be related to permission. It seems I used `sudo ./R/run-tests.sh`
> and it worked sometimes. Without permission, maybe we couldn't access /tmp
> directory. However, the SparkR unit testing is still brittle.
> [error
> message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299432#comment-15299432 ] Felix Cheung commented on SPARK-15439: -- Possibly with R version or package versions. Can you check `installed.packages()`? > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[
https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299143#comment-15299143
]
Miao Wang commented on SPARK-15439:
---
[~felixcheung] For the test case below:
test_that("Check masked functions", {
# Check that we are not masking any new function from base, stats, testthat
unexpectedly
masked <- conflicts(detail = TRUE)$`package:SparkR`
expect_true("describe" %in% masked) # only when with testthat..
func <- lapply(masked, function(x) { capture.output(showMethods(x))[[1]] })
funcSparkROrEmpty <- grepl("\\(package SparkR\\)$|^$", func)
maskedBySparkR <- masked[funcSparkROrEmpty]
namesOfMasked <- c("describe", "cov", "filter", "lag", "na.omit", "predict",
"sd", "var",
"colnames", "colnames<-", "intersect", "rank", "rbind",
"sample", "subset",
"summary", "transform", "drop", "window", "as.data.frame",
"endsWith", "startsWith")
When I run it in local, it has "endsWith" and "startsWith". However, when I
submit PR, Jekins test doesn't include these two.
I don't know what is the difference when running in local and running auto
test. Can you help checking it? Thanks!
> Failed to run unit test in SparkR
> -
>
> Key: SPARK-15439
> URL: https://issues.apache.org/jira/browse/SPARK-15439
> Project: Spark
> Issue Type: Bug
> Components: SparkR
>Affects Versions: 2.0.0
>Reporter: Kai Jiang
>
> Failed to run ./R/run-tests.sh around recent commit (May 19, 2016)
> It might be related to permission. It seems I used `sudo ./R/run-tests.sh`
> and it worked sometimes. Without permission, maybe we couldn't access /tmp
> directory. However, the SparkR unit testing is still brittle.
> [error
> message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299078#comment-15299078 ] Apache Spark commented on SPARK-15439: -- User 'wangmiao1981' has created a pull request for this issue: https://github.com/apache/spark/pull/13284 > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299011#comment-15299011 ] Miao Wang commented on SPARK-15439: --- It seems that this one is caused by https://github.com/apache/spark/pull/11318 Let me change some of this PR for test. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15299004#comment-15299004 ] Miao Wang commented on SPARK-15439: --- [~shivaram]I fixed that failure. But I see a new failure in test_sparkSQL.R#922 2. Error: subsetting (@test_sparkSQL.R#922) argument "subset" is missing, with no default 1: subset(df, select = "name") at /Users/mwang/spark_ws_0904/R/lib/SparkR/tests/testthat/test_sparkSQL.R:922 2: subset(df, select = "name") 3: .local(x, ...) 4: x[subset, select, drop = drop] I am fixing it now. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298679#comment-15298679 ] Miao Wang commented on SPARK-15439: --- I think I found the reason of failure. Will post a PR soon. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15298579#comment-15298579 ] Shivaram Venkataraman commented on SPARK-15439: --- I think the `maskedCompletely` failures might be related to some R version or package versions. At a high level those tests check how many base R functions are masked by SparkR. cc [~felixcheung] who knows more about those unit tests. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297781#comment-15297781 ] Kai Jiang commented on SPARK-15439: --- I can reproduce this. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297774#comment-15297774 ] Miao Wang commented on SPARK-15439: --- Per discussion with Yanbo, the directory not found is a known issue. [~yanboliang] Can you explain it? For the other Failures, I will try to debug it to understand whether it is related to the known issue or not. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297714#comment-15297714 ] Miao Wang commented on SPARK-15439: --- I am still investigating the errors. Now, I see new errors without using sudo, like below: Failed - 1. Failure: Check masked functions (@test_context.R#30) length(maskedBySparkR) not equal to length(namesOfMasked). 1/1 mismatches [1] 22 - 20 == 2 2. Failure: Check masked functions (@test_context.R#31) sort(maskedBySparkR) not equal to sort(namesOfMasked). Lengths differ: 22 vs 20 3. Failure: Check masked functions (@test_context.R#40) length(maskedCompletely) not equal to length(namesOfMaskedCompletely). 1/1 mismatches [1] 5 - 3 == 2 4. Failure: Check masked functions (@test_context.R#41) sort(maskedCompletely) not equal to sort(namesOfMaskedCompletely). Lengths differ: 5 vs 3 I am learning how R works. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15297176#comment-15297176 ] Shivaram Venkataraman commented on SPARK-15439: --- [~wm624] Any luck in figuring out why this is happening ? Would be good to fix this before the 2.0 RC goes out cc [~sunrui] > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15294522#comment-15294522 ] Miao Wang commented on SPARK-15439: --- Reproduced. Now I am analyzing the reason. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15293914#comment-15293914 ] Miao Wang commented on SPARK-15439: --- I see an error: Error in library(testthat) : there is no package called ‘testthat’ Execution halted > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292815#comment-15292815 ] Kai Jiang commented on SPARK-15439: --- Cool! Let me know if you find a way to solve this. > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-15439) Failed to run unit test in SparkR
[ https://issues.apache.org/jira/browse/SPARK-15439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15292814#comment-15292814 ] Miao Wang commented on SPARK-15439: --- Let me investigate this bug. Thanks! > Failed to run unit test in SparkR > - > > Key: SPARK-15439 > URL: https://issues.apache.org/jira/browse/SPARK-15439 > Project: Spark > Issue Type: Bug > Components: SparkR >Affects Versions: 2.0.0 >Reporter: Kai Jiang > > Failed to run ./R/run-tests.sh around recent commit (May 19, 2016) > It might be related to permission. It seems I used `sudo ./R/run-tests.sh` > and it worked sometimes. Without permission, maybe we couldn't access /tmp > directory. However, the SparkR unit testing is still brittle. > [error > message|https://gist.github.com/vectorijk/71f4ff34e3d34a628b8a3013f0ca2aa2] -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
