Hi Peter, Thank you for helping me out!!! I went through the provided link and it has helped me a lot. Now i do get that there will be a intra worker messaging and another messaging system for pulling data into storm. Can you also tell me whether i need to have specific knowledge about the external queing system(RabbitMQ,ActiveMQ,kafka) or it will just be a connector which i could use to ingest data. I wanna know whether i need to develop specific skills for the queing systems like RabbitMQ?
Thanks, Siddharth ubale On Wed, Jul 2, 2014 at 6:33 PM, Richards Peter <[email protected]> wrote: > Hi, > > I think you are slightly confused. Let me try to explain it in a simple > manner. Storm has a transport layer for communication between spouts and > bolts. At present (release >=storm 0.9) there are two transport layer > implementations provided by storm - zeromq and netty based > implementations.You can select anyone of them. Installation of zeromq needs > some special steps. It is already documented in storm project. I haven't > tried Netty based implementation. However netty based implementation is > supposed to give you better performance. > http://yahooeng.tumblr.com/post/64758709722/making-storm-fly-with-netty > > Now let me try to answer your question related to ActiveMQ. For most of > the realtime usecases people like to process messages from a queue. For > such usecases the messages are read from these queues by the spout. You can > use any queuing system(Kafka/RabbitMQ/ActiveMQ/...) for this activity. All > that you need is a consumer in your spout to read these messages. > > Regards, > Richards Peter. >
