Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/21198#discussion_r185148928
--- Diff: python/pyspark/streaming/tests.py ---
@@ -1549,7 +1549,9 @@ def search_kinesis_asl_assembly_jar():
kinesis_jar_present = True
jars = "%s,%s,%s" % (kafka_assembly_jar, flume_assembly_jar,
kinesis_asl_assembly_jar)
- os.environ["PYSPARK_SUBMIT_ARGS"] = "--jars %s pyspark-shell" % jars
+ existing_args = os.environ.get("PYSPARK_SUBMIT_ARGS")
--- End diff --
@vanzin, I think this will be broken if `PYSPARK_SUBMIT_ARGS` is not set
and it returns `None`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
