Re: Moving data from one cluster to another with Kafka Streams

2018-02-04 Thread Geoffrey Holmes
> Kafka Streams only work with a single cluster. Ok, that’s what I was thinking after I looked at it more. > Thus, you would need to either transform the data first and replicate > the output topic to the target cluster, or replicate first and transform > within the target cluster. I don’t

Re: Moving data from one cluster to another with Kafka Streams

2018-02-02 Thread Matthias J. Sax
Kafka Streams only work with a single cluster. Thus, you would need to either transform the data first and replicate the output topic to the target cluster, or replicate first and transform within the target cluster. Note, for the "intermediate" topic you need, you can set a low retention time

Moving data from one cluster to another with Kafka Streams

2018-02-02 Thread Geoffrey Holmes
I need to get messages from a topic in one Kafka cluster, transform the message payload, and put the messages into topics in another Kafka cluster. Is it possible to do this with Kafka Streams? I don’t see how I can configure the stream to use one cluster for the input and another cluster for