GitHub user dragos opened a pull request:
https://github.com/apache/spark/pull/4270
[SPARK-4631][streaming] Wait for a receiver to start before publishing test
data.
This fixes two sources of non-deterministic failures in this test:
- wait for a receiver to be up before pushing data through MQTT
- gracefully handle the case where the MQTT client is overloaded. Thereâs
a hard-coded limit of 10 in-flight messages, and this test may hit it.
Instead of crashing, we retry sending the message.
Both of these are needed to make the test pass reliably on my machine.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dragos/spark issue/fix-flaky-test-SPARK-4631
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/4270.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4270
----
commit d408a8eb36fe59f64c7b3ce9eb9d22cee6dbd684
Author: Iulian Dragos <[email protected]>
Date: 2015-01-29T16:06:42Z
Install callback before connecting to MQTT broker.
According to the MQTT client docs the callback needs to be installed
before the client connects or subscribes to a topic. Otherwise
messages may be lost.
commit 261653fad1abd4b40931663e9fc8aa7510f635db
Author: Iulian Dragos <[email protected]>
Date: 2015-01-29T16:11:14Z
[SPARK-4631][streaming] Wait for a receiver to start before publishing test
data.
This fixes two sources of non-deterministic failures in this test:
- wait for a receiver to be up before pushing data through MQTT
- gracefully handle the case where the MQTT client is overloaded. Thereâs
a hard-coded limit of 10 in-flight messages, and this test may hit it.
Instead of crashing, we retry sending the message.
Both of these are needed to make the test pass reliably on my machine.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
