GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/1751
[SPARK-1022][Streaming] Add Kafka real unit test
This PR is a updated version of (https://github.com/apache/spark/pull/557)
to actually test sending and receiving data through Kafka, and fix previous
flaky issues.
@tdas, would you mind reviewing this PR? Thanks a lot.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark kafka-unit-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/1751.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1751
----
commit 796d4ca6ed2e420c73e1beab9518a324df74457c
Author: jerryshao <[email protected]>
Date: 2014-01-24T10:51:21Z
Add real Kafka streaming test
Conflicts:
external/kafka/src/test/java/org/apache/spark/streaming/kafka/JavaKafkaStreamSuite.java
external/kafka/src/test/scala/org/apache/spark/streaming/kafka/KafkaStreamSuite.scala
project/SparkBuild.scala
commit 860f649be9b3052688c1d6b4540458657bc2e759
Author: jerryshao <[email protected]>
Date: 2014-02-10T02:09:43Z
Minor style changes, and tests ignored due to flakiness
Conflicts:
project/SparkBuild.scala
commit 45593105ba131366a1cee43e4c9c79e89735b5b7
Author: jerryshao <[email protected]>
Date: 2014-04-30T01:07:06Z
Minor changes for Kafka unit test
commit 5525f10006fe17e36b6d328a6e1bbc040d760e90
Author: jerryshao <[email protected]>
Date: 2014-07-23T01:42:40Z
Fix flaky issue of Kafka real unit test
commit 5222330879f7310e424a334a568b7fbd89790e9e
Author: jerryshao <[email protected]>
Date: 2014-08-03T08:26:32Z
Change JavaKafkaStreamSuite to better test it
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
