Thanks Jacob. I'm using kafka as a distributed queue and my data pipeline
is a several components which connected together via a stream abstraction.
each component has a input and output stream. Basically the source of
this pipeline can be anything and the output can be anything to.
On Tue, Mar 13, 2018 at 1:27 PM, Jacob Sheck <shec0...@gmail.com> wrote:
> Sameer when you say that you need to "consume from and produce to a topic"
> to me that seems like a good fit for Kafka Streams. Streaming your data
> out of Kafka for a transform and back in has some fundamental costs and
> operational challenges involved. Are the events in your stream stateless?
> If it isn't stateless streams will ensure a consistent playback of events
> if needed. Without knowing more about your pipeline it is hard to make
> recommendations. Are you possibly using Kafka as a distributed queue?
> On Tue, Mar 13, 2018 at 6:29 AM Sameer Rahmani <lxsame...@gmail.com>
> > Hi folks,
> > I need to consume from and produce to a topic. I have my own data
> > to process the data.
> > So I was wondering beside the stores and StreamDSL what does Kafka
> > brings to the table
> > that might be useful to me ?