GitHub user arunmahadevan opened a pull request:
https://github.com/apache/storm/pull/2829
STORM-3222: Fix KafkaSpout internals to use LinkedList instead of ArrayList
KafkaSpout internally maintains a waitingToEmit list per topic partition
and keeps removing the first item to emit during each nextTuple. The
implementation uses an ArrayList which results in un-necessary traversal and
copy for each tuple.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/arunmahadevan/storm STORM-3222
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/2829.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2829
----
commit a5384aa845496a7e584bd947cace968a18b7ffdf
Author: Arun Mahadevan <arunm@...>
Date: 2018-09-12T22:36:24Z
STORM-3222: Fix KafkaSpout internals to use LinkedList instead of ArrayList
KafkaSpout internally maintains a waitingToEmit list per topic partition
and keeps removing the first item to emit during each nextTuple. The
implementation uses an ArrayList which results in un-necessary traversal and
copy for each tuple.
Also I am not sure why the nextTuple only emits a single tuple wheres
ideally it should emit whatever it can emit in a single nextTuple call which is
more efficient. However the logic appears too complicated to refactor.
----
---
