GitHub user pnowojski opened a pull request:
https://github.com/apache/flink/pull/4470
[FLINK-7343] Simulate network failures in kafka at-least-once test
We shouldn't fail KafkaServers directly, because they might not be able to
flush the data (`log.flush.interval.***` properties). Since we don't want to
test how well Kafka implements at-least-once/exactly-once semantic, it is a
better idea (and hopefully more reliable) to just simulate network failure
between Flink and Kafka in our at-least-once tests. To achieve that I have
implemented `NetworkFailuresProxy` class.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pnowojski/flink network-failures
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4470.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4470
----
commit 0e28327619893cfbf793fa842be3d965f649516c
Author: Piotr Nowojski <[email protected]>
Date: 2017-08-01T14:05:49Z
[FLINK-7343][kafka] Increase Xmx for tests
Sometimes 1000m was not enough memory to run at-least-once tests with
broker failures on Travis
commit 8d820c3d0e77624a945e074f4a1bc476b5fd0f75
Author: Piotr Nowojski <[email protected]>
Date: 2017-08-01T16:11:27Z
[FLINK-7343] Add network proxy utility to simulate network failures
commit 967e1dfc87846b4011652bbaefab696900abc8dd
Author: Piotr Nowojski <[email protected]>
Date: 2017-08-03T07:25:04Z
fixup! [FLINK-7343][kafka] Increase Xmx for tests
commit 27b20f2ec3770231d95c3c7918c9313ce58b5e18
Author: Piotr Nowojski <[email protected]>
Date: 2017-08-03T09:27:12Z
[FLINK-7343] Use NetworkFailureProxy in kafka tests
We shouldn't fail KafkaServers directly, because they might not be able
to flush the data. Since we don't want to test how well Kafka implements
at-least-once/exactly-once semantic, we just simulate network failure
between Flink and Kafka in our at-least-once tests.
commit 692b5944f16b98aafe716ca1d18a04fa8a033798
Author: Piotr Nowojski <[email protected]>
Date: 2017-08-03T09:35:26Z
[hotfix][Kafka] Clean up getKafkaServer method
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
