GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/20128
[SPARK-21893][SPARK-22142][BUILD][FOLLOWUP] Enables PySpark tests for Flume
and Kafka in Jenkins
## What changes were proposed in this pull request?
This PR proposes to enable PySpark tests for Flume and Kafka in Jenkins by
explicitly setting the environment variables in `modules.py`.
Seems we are not taking the dependencies into account when calculating
environment variables:
https://github.com/apache/spark/blob/3a07eff5af601511e97a05e6fea0e3d48f74c4f0/dev/run-tests.py#L554-L561
## How was this patch tested?
Manual tests with Jenkins in https://github.com/apache/spark/pull/20126.
**Before** -
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85559/consoleFull
```
[info] Setup the following environment variables for tests:
...
```
**After** -
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85560/consoleFull
```
[info] Setup the following environment variables for tests:
ENABLE_KAFKA_0_8_TESTS=1
ENABLE_FLUME_TESTS=1
...
```
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/spark SPARK-21893
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20128.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20128
----
commit 3b1533a400ecf2eeee77cf0e18b8cb76a14305b0
Author: hyukjinkwon <gurwls223@...>
Date: 2017-12-31T15:35:54Z
Enables PySpark tests for Flume and Kafka in Jenkins
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
