GitHub user srdo opened a pull request:
https://github.com/apache/storm/pull/2907
STORM-2990, STORM-3279: Fix issue where Kafka Trident spout could ignâ¦
â¦ore EARLIEST and LATEST, and make EARLIEST and LATEST only take effect
on topology deploy
See https://issues.apache.org/jira/browse/STORM-2990 and
https://issues.apache.org/jira/browse/STORM-3279.
3279 is fixed by always returning a metadata from the emitter, even if no
tuples were emitted. If no tuples are emitted, the metadata will now indicate
that `consumer.position() - 1` was emitted.
2990 is fixed by inspecting the lastBatchMeta topology id, and deciding
whether to start over based on that.
I expanded the emitter tests to cover these cases as well, and tried to
make them a bit more readable by splitting them into multiple classes. The
Kafka client dependency has been bumped to 0.11.0.0, since the MockConsumer
wasn't behaving properly in 0.10.1.0.
The only other change is to replace the KafkaUnit internals so we no longer
depend on Kafka's internal test tooling.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/srdo/storm STORM-2990
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2907.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2907
----
commit 9247e4a59a2f8829238179066a7299fa1cafdbaa
Author: Stig Rohde Døssing <srdo@...>
Date: 2018-11-14T15:17:03Z
STORM-2990, STORM-3279: Fix issue where Kafka Trident spout could ignore
EARLIEST and LATEST, and make EARLIEST and LATEST only take effect on topology
deploy
----
---
