GitHub user srdo opened a pull request:
https://github.com/apache/storm/pull/2890
STORM-3268: Improve integration test stability
https://issues.apache.org/jira/browse/STORM-3268
Basically the integration test can't handle logs rolling during the test.
To fix this we added a bit of time between spout emits, but I don't think it
was enough. I've put a limit on how many tuples the spouts will emit during the
tests, which should prevent the logs from growing too much.
Some tests use multiple bolt/spout tasks, and don't wait for all executors
to be started. In some cases tests could fail because e.g. two spout instances
were supposed to run, but one executor was slow to start so the log for that
executor wasn't picked up when the test asked Nimbus for logs for the spout
component. The tests now check that Nimbus has all expected executors listed.
The tests extract spout/bolt logs by looking for a magic string. If
multiple components were running in the same worker, the log reading logic
could pick up e.g. logs from the bolt while looking for logs from the spout.
The magic string now includes the component id the log reader is looking for.
Some tests were slow due to building lots of useless error messages, the
error messages are constructed lazily now.
The tests aren't perfectly stable, but I think this is due to an issue in
Storm where tuples can be lost if the spout starts emitting before the bolt
executor is ready to receive. I've seen the spout emit a bunch of tuples before
the bolt worker started, and those tuples end up timing out.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srdo/storm STORM-3268
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2890.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2890
----
commit ef9d907d457d9d5ce9af42b88c8a11d16b7facfe
Author: Stig Rohde Døssing <srdo@...>
Date: 2018-10-21T10:55:49Z
STORM-3267: Disable Java 10 build
commit 5b066609b0dbeec2bb842ddd5a81ae8c32ac9719
Author: Stig Rohde Døssing <srdo@...>
Date: 2018-10-20T20:17:26Z
STORM-3268: Try to improve integration test stability
----
---
