Beside state and DSL Kafka Stream also allow for fully fault-tolerant,
salable, and elastic deployment.


On 3/13/18 10:55 AM, Hans Jespersen wrote:
> "If your system is stateless and the transformations are not interdependent"
> then I would just look at using Kafka Connect's Single Message Transform
> (SMT) feature.
> -hans
> /**
>  * Hans Jespersen, Director Systems Engineering, Confluent Inc.
>  * (650)924-2670
>  */
> On Tue, Mar 13, 2018 at 9:18 AM, Jacob Sheck <> wrote:
>> If you are augmenting streaming data with dimensional data, or if you can
>> transform your data with a map, filter or join operation Streams will be a
>> good option.  If your system is stateless and the transformations are not
>> interdependent, you may want to look into using one of the queue
>> technologies like AcitveMQ.
>> On Tue, Mar 13, 2018 at 10:00 AM Sameer Rahmani <>
>> wrote:
>>> Thanks Jacob. I'm using kafka as a distributed queue and my data pipeline
>>> is a several components which connected together via a stream
>> abstraction.
>>> each component has a input and output stream. Basically the source of
>>> this pipeline can be anything and the output can be anything to.
>>> On Tue, Mar 13, 2018 at 1:27 PM, Jacob Sheck <> wrote:
>>>> Sameer when you say that you need to "consume from and produce to a
>>> topic"
>>>> to me that seems like a good fit for Kafka Streams.  Streaming your
>> data
>>>> out of Kafka for a transform and back in has some fundamental costs and
>>>> operational challenges involved.  Are the events in your stream
>>> stateless?
>>>> If it isn't stateless streams will ensure a consistent playback of
>> events
>>>> if needed.  Without knowing more about your pipeline it is hard to make
>>>> recommendations.  Are you possibly using Kafka as a distributed queue?
>>>> On Tue, Mar 13, 2018 at 6:29 AM Sameer Rahmani <>
>>>> wrote:
>>>>> Hi folks,
>>>>> I need to consume from and produce to a topic. I have my own data
>>>> pipeline
>>>>> to process the data.
>>>>> So I was wondering beside the stores and StreamDSL what does Kafka
>>>> Streams
>>>>> brings to the table
>>>>> that might be useful to me ?

Attachment: signature.asc
Description: OpenPGP digital signature

Reply via email to