GitHub user BryanCutler opened a pull request:
https://github.com/apache/spark/pull/22273
[SPARK-25272][PYTHON][TEST] Add test to better indicate pyarrow is
installed and related tests will run
## What changes were proposed in this pull request?
Currently, Arrow tests in python are skipped if pyarrow is not installed
and there is clear Jenkins outputs that these tests were skipped. When it is
installed, the tests are run but there will be no output. This adds a test that
will be skipped and output a message in the Jenkins log that pyarrow and pandas
required versions are installed. This way it is clear that Arrow tests will be
run. The output given when pyarrow and pandas are installed is:
`test_required_pyarrow_pandas_not_installed
(pyspark.sql.tests.HaveArrowTests) ... skipped 'Required pyarrow and pandas
were found, Arrow tests will run'
## How was this patch tested?
Added new test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/BryanCutler/spark
arrow-test-pyarrow-installed-SPARK-25272
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22273.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22273
----
commit cd4ca3cb91462335f7aae6ca683110f021e00851
Author: Bryan Cutler <cutlerb@...>
Date: 2018-08-29T20:42:13Z
add test that will output a skipped message when pyarrow is installed
commit c341ff83f957b69032f67778177617a62baa1a14
Author: Bryan Cutler <cutlerb@...>
Date: 2018-08-29T20:50:11Z
added to msg
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
