GitHub user HyukjinKwon opened a pull request:
https://github.com/apache/spark/pull/20834
[SPARK-23695][PYTHON] Fix the error message for Kinesis streaming tests
## What changes were proposed in this pull request?
```bash
ENABLE_KINESIS_TESTS=1 SPARK_TESTING=1 bin/pyspark pyspark.streaming.tests
```
Before:
```
Skipped test_flume_stream (enable by setting environment variable
ENABLE_FLUME_TESTS=1Skipped test_kafka_stream (enable by setting environment
variable ENABLE_KAFKA_0_8_TESTS=1Traceback (most recent call last):
File
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 72, in _run_code
exec code in run_globals
File "/.../spark/python/pyspark/streaming/tests.py", line 1572, in
<module>
% kinesis_asl_assembly_dir) +
NameError: name 'kinesis_asl_assembly_dir' is not defined
```
After:
```
Skipped test_flume_stream (enable by setting environment variable
ENABLE_FLUME_TESTS=1Skipped test_kafka_stream (enable by setting environment
variable ENABLE_KAFKA_0_8_TESTS=1Traceback (most recent call last):
File
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File
"/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py",
line 72, in _run_code
exec code in run_globals
File "/.../spark/python/pyspark/streaming/tests.py", line 1576, in
<module>
"You need to build Spark with 'build/sbt -Pkinesis-asl "
Exception: Failed to find Spark Streaming Kinesis assembly jar in
/.../spark/external/kinesis-asl-assembly. You need to build Spark with
'build/sbt -Pkinesis-asl assembly/package
streaming-kinesis-asl-assembly/assembly'or 'build/mvn -Pkinesis-asl package'
before running this test.
```
## How was this patch tested?
Manually tested.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HyukjinKwon/spark minor-variable
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20834.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20834
----
commit d2fe1e9a3dfdba9958b9ffa893fd903e902b4033
Author: hyukjinkwon <gurwls223@...>
Date: 2018-03-15T11:48:48Z
Fix the error message for Kinesis streaming tests
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
