That’s correct, the code running in Storm decides where messages are headed - kafka is a scalable speedy FIFO buffer that can sit between any nodes in the network you want it to. 1-to-1, 1-to-many, or many-to-many.
Steve Blackmon Director, Data Sciences W2O Group 3000 E Cesar Chavez St. Austin, Texas 78701 cell 512.965.0451 | work 512.402.6366 twitter @steveblackmon On Aug 28, 2013, at 3:37 PM, Danny Sullivan <[email protected]<mailto:[email protected]>> wrote: I understood that Storm, not Kafka, would overtake the responsibilities of Camel through Spouts and Bolts: https://github.com/nathanmarz/storm/wiki/Tutorial Date: Wed, 28 Aug 2013 13:19:34 -0700 Subject: Re: Integration with Storm and Cassandra Java Driver From: [email protected] To: [email protected] On Wed, Aug 28, 2013 at 1:19 PM, Chris Geer <[email protected]> wrote: On Wed, Aug 28, 2013 at 1:15 PM, Jason Letourneau <[email protected] wrote: Matt's desire might create an even more complex architecture, though a wonderful goal. I wouldn't be opposed to just picking a solution and designing around it for the purposes of getting the end to end solution running and then revisiting the appropriate abstractions. I had seen Kafka when starting out on the initial code base, I thought it held serious promise for Streams. Advantage of Camel is the ability to deploy as part of the web archive vs a standalone service, if Kafka/Storm bring that as well, sounds cool to me. They seem to bring the high performance hammer of doom - rock on. \\m// I using Kafka gets you out of using something like Camel (or custom code) to orchestrate your routes and other end points (i.e. Twitter). Should have read "I'm not sure using...." On Wed, Aug 28, 2013 at 3:40 PM, Matt Franklin <[email protected]> wrote: On Wed, Aug 28, 2013 at 2:14 PM, Danny Sullivan <[email protected] wrote: Thanks for the info Steve. As I understand, Kafka would take the place of the functionality of what ActiveMQ does now. Storm would take place of what Camel does now. I think in the long term we need to have a flexible architecture with a few implementations. The way I see it, we need collection, orchestration, processing pipeline, persistence and exposure. If there is a way that we can define each of these components loosely coupled enough to where we could have a Kafka OR AMQP routing implementation that would be ideal. I haven't thought through exactly how to do this myself, but wanted to offer that things may not be mutually exclusive. CONFIDENTIALITY NOTICE: This e-mail, along with any documents, files, or attachments, may contain information that is confidential, privileged, or otherwise exempt from disclosure. If you are not the intended recipient or person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, printing, distribution or use of any information contained in or attached to this e-mail is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and delete the original e-mail and its attachments without reading, printing, or saving in any manner. This e-mail message should not be interpreted to include a digital or electronic signature that can be used to authenticate an agreement, contract or other legal document, nor to reflect an intention to be bound to any legally-binding agreement or contract. Your cooperation is appreciated. Thank you.
