Github user felixcheung commented on the issue:
https://github.com/apache/spark/pull/17516
There are two parts to the branch-2.1 fix
First, the reason why the test failed was because `SPARK_HOME` was not set
before calling `spark.install()` when running as a package. This would not be a
problem when in Jenkins, but only when running with `R CMD check SparkR*.tgz`.
The fix was to move `spark.install` to earlier.
Second, even after the change, while testing it, I found that `R CMD check`
was getting `spark-warehouse` etc in the `testthat` directory, NOT in
`SPARK_HOME` - therefore that test would be essentially a no-op or always
passes anyway. I made the call to disable it (with `skip_if_cran`), but that
had the unintended effect of also turning off that test in Jenkins, as we are
testing with `--as-cran` (as explained above)
And so the attempt here in this PR to fix this for real in master. Since we
are rolling our RC anytime, I don't want to delay the first fix (install.spark)
only to sort out the 2nd part, which could come a bit later.
If you feel that's safer, we could also add `skip_if_cran` to this test in
master - just know that it will also turn off this test in Jenkins. Since with
`R CMD check` the `spark-warehouse` and `metastore_db` are not written to
`SPARK_HOME`, but to `testthat`, this test will pass during the package test
with `R CMD check` - so long as we merge this PR to move `install.spark` first
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
