Hyukjin Kwon created SPARK-23695: ------------------------------------ Summary: Confusing error message for PySpark's Kinesis tests when its jar is missing but enabled Key: SPARK-23695 URL: https://issues.apache.org/jira/browse/SPARK-23695 Project: Spark Issue Type: Improvement Components: PySpark Affects Versions: 2.3.0 Reporter: Hyukjin Kwon
Currently if its jar is missing but the Kinesis tests are enabled: {code} ENABLE_KINESIS_TESTS=1 SPARK_TESTING=1 bin/pyspark pyspark.streaming.tests Skipped test_flume_stream (enable by setting environment variable ENABLE_FLUME_TESTS=1Skipped test_kafka_stream (enable by setting environment variable ENABLE_KAFKA_0_8_TESTS=1Traceback (most recent call last): File "/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/local/Cellar/python/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/.../spark/python/pyspark/streaming/tests.py", line 1572, in <module> % kinesis_asl_assembly_dir) + NameError: name 'kinesis_asl_assembly_dir' is not defined {code} It shows a confusing error message. Seems a mistake. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org