Abhishek,
OutputCollector is thread safe for most methods, and for all methods in 0.11.0+
of storm. In versions prior to 0.11.0 emitting to a shuffle grouping was not
thread safe because of the grouping code itself was not thread safe, hence the
synchronization on the collector to support older versions of storm.
If async is false the callback is null, so no acks/fails will happen there and
the ack will happen right after calling producer.send so I don't think acks
will happen twice.
As far as performance in concerned we can guess, but without real numbers it is
hard to tell what will really happen in any situation. If you have a
performance benchmark that we can use to profile different kafka bolt
implementations in terms of both throughput and latency that would be great,
but without that until someone complains I am not going to worry about it too
much. For me personally my biggest concern is the huge amount of object
creation that is happening. But I don't see a reason to invest more resources
unless someone is complaining or there is a need. If you have that need feel
free to file a JIRA, write a benchmark and start profiling. - Bobby
On Tuesday, November 24, 2015 9:11 AM, Abhishek Agarwal
<[email protected]> wrote:
I was just looking at STORM-826 <https://github.com/apache/storm/pull/572>and
was wondering if this is the correct approach for doing acks and fails
asynchronously.
> Callback will happen on a different thread. Wouldn't it lead to race
conditions as the disruptor send queues are single producer queues?
> If async is set to false, wouldn't the acks/fails performed twice? Since
callback is always executed by the background thread.
Also if bolt operations are high latency and IO bound, will a non-blocking
implementation without batching lead to increased throughput ?
--
Regards,
Abhishek Agarwal
