GitHub user felixcheung reopened a pull request:
https://github.com/apache/spark/pull/16720
[SPARK-19387][SPARKR] Tests do not run with SparkR source package in CRAN
check
## What changes were proposed in this pull request?
- this is cause by changes in SPARK-18444, SPARK-18643 that we no longer
install Spark when `master = ""` (default), but also related to SPARK-18449
since the real `master` value is not known at the time the R code in
`sparkR.session` is run. (`master` cannot default to "local" since it could be
overridden by spark-submit commandline or spark config)
- as a result, while running SparkR as a package in IDE is working fine,
CRAN check is not as it is launching it via non-interactive script
- fix is to add check to the beginning of each test and vignettes; the same
would also work by changing `sparkR.session()` to `sparkR.session(master =
"local")` in tests, but I think being more explicit is better.
## How was this patch tested?
Tested this by reverting version to 2.1, since it needs to download the
release jar with matching version. But since there are changes in 2.2
(specifically around SparkR ML) that are incompatible with 2.1, some tests are
failing in this config. Will need to port this to branch-2.1 and retest with
2.1 release jar.
manually as:
```
# modify DESCRIPTION to revert version to 2.1.0
SPARK_HOME=/usr/spark R CMD build pkg
# run cran check without SPARK_HOME
R CMD check --as-cran SparkR_2.1.0.tar.gz
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/felixcheung/spark rcranchecktest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16720.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16720
----
commit 318ecc876c64b7085df46618278861c2ea9ff422
Author: Felix Cheung <[email protected]>
Date: 2017-01-27T19:03:15Z
make sure Spark is installed
commit f51f504acb3a64da27bf0bddbb156c68d62d89bb
Author: Felix Cheung <[email protected]>
Date: 2017-01-28T06:55:54Z
simplify
commit cd1394a8aab39b20c9c747b5ddfe5a003081510a
Author: Felix Cheung <[email protected]>
Date: 2017-02-13T00:11:18Z
change install.spark
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
