On Wed, Aug 28, 2013 at 12:40 PM, Matt Franklin <[email protected]>wrote:
> On Wed, Aug 28, 2013 at 2:14 PM, Danny Sullivan <[email protected] > >wrote: > > > Thanks for the info Steve. > > As I understand, Kafka would take the place of the functionality of what > > ActiveMQ does now. Storm would take place of what Camel does now. > > > > I think in the long term we need to have a flexible architecture with a few > implementations. The way I see it, we need collection, orchestration, > processing pipeline, persistence and exposure. If there is a way that we > can define each of these components loosely coupled enough to where we > could have a Kafka OR AMQP routing implementation that would be ideal. I > haven't thought through exactly how to do this myself, but wanted to offer > that things may not be mutually exclusive. > I agree Matt. I've had some ideas for a while but haven't had the time to test them out. At a high level what I think would be nice is to get rid of the API interfaces and make it all message based. The core of the streams server could essentially dynamically build camel routes based on the configuration. That way you could rearrange the various processing steps any way you see fit as long as the input/output messages were compatible. This would also allow you to split steps across servers or even different implementations. Chris
