Thanks Bobby for the clarification. Though I would like to point out that kafka bolt is not backward compatible with versions prior to 0.10.x. TupleUtils class, which is being referred to by KafkaBolt, is not available in 0.9.x releases.
If this break was not intentional, I could refactor the bolt to remove the TupleUtils class. As a workaround, people can also put TupleUtils class in the application with appropriate package name. This is what I am doing to make storm-kafka-0.10.0 work with storm-core 0.9.5 On Tue, Nov 24, 2015 at 9:16 PM, Bobby Evans <[email protected]> wrote: > Abhishek, > OutputCollector is thread safe for most methods, and for all methods in > 0.11.0+ of storm. In versions prior to 0.11.0 emitting to a shuffle > grouping was not thread safe because of the grouping code itself was not > thread safe, hence the synchronization on the collector to support older > versions of storm. > If async is false the callback is null, so no acks/fails will happen there > and the ack will happen right after calling producer.send so I don't think > acks will happen twice. > > As far as performance in concerned we can guess, but without real numbers > it is hard to tell what will really happen in any situation. If you have a > performance benchmark that we can use to profile different kafka bolt > implementations in terms of both throughput and latency that would be > great, but without that until someone complains I am not going to worry > about it too much. For me personally my biggest concern is the huge amount > of object creation that is happening. But I don't see a reason to invest > more resources unless someone is complaining or there is a need. If you > have that need feel free to file a JIRA, write a benchmark and start > profiling. - Bobby > > > On Tuesday, November 24, 2015 9:11 AM, Abhishek Agarwal < > [email protected]> wrote: > > > I was just looking at STORM-826 <https://github.com/apache/storm/pull/572 > >and > was wondering if this is the correct approach for doing acks and fails > asynchronously. > > > Callback will happen on a different thread. Wouldn't it lead to race > conditions as the disruptor send queues are single producer queues? > > If async is set to false, wouldn't the acks/fails performed twice? Since > callback is always executed by the background thread. > > Also if bolt operations are high latency and IO bound, will a non-blocking > implementation without batching lead to increased throughput ? > > -- > Regards, > Abhishek Agarwal > > > > -- Regards, Abhishek Agarwal
