Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/19134#discussion_r137923181
--- Diff: python/pyspark/streaming/tests.py ---
@@ -1051,6 +1062,9 @@ def test_kafka_stream(self):
def test_kafka_direct_stream(self):
"""Test the Python direct Kafka stream API."""
+ if not are_kafka_tests_enabled:
--- End diff --
@vanzin ah I was distracted by how it's also handled in KinesisStreamTests.
Done, I think.
I think that yes there is no 0.10 test, as I don't think Pyspark supports
0.10 (right @jerryshao ?):
https://github.com/apache/spark/pull/14340#issuecomment-235457517
So all Kafka Pyspark tests should be disabled, yeah.
But, does this mean that Pyspark Kafka support is effectively deprecated,
entirely?
(I need to add deprecation messages to 0.8 code)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
