Hi Rob, Kafka Streams do not use a MessageChooser to give priorities to one stream over the other like Samza does, but rely on the event timestamp of the records which can be treated as the "MessageChooser". You can read this section for more details:
http://docs.confluent.io/2.1.0-alpha1/streams/architecture.html#flow-control-with-timestamps In addition, we are considering to add finer flow control beyond just timestamps, but this may not be available in the 0.10.0.0 release: https://issues.apache.org/jira/browse/KAFKA-3478 Guozhang On Thu, May 19, 2016 at 4:37 PM, Robert Crim <[email protected]> wrote: > I'm starting to take a closer look at Kafka Streams, and one of the things > I'd like to be able to is see if I can "migrate" our existing Samza-based** > applications to run on Kafka Streams. > > With Samza, we take advantage of multiple source streams being sent to a > process function that runs in a single thread for stateful processing > (along with a single window function). > > I recon I can emulate this model by mapping a samza-like "envelope" to each > source and forwarding them on to a single processor. > > But -- I don't see if there is support for "bootstrap" streams or any kind > of stream priority tools, akin to Samza's MessageChooser. Is there > something I'm missing, or any ideas how to implement this? I'm not sure if > I can see how to get current lag / topic metadata in my consumer. > > Thanks! > Rob > > ** for context, we use a Clojure-based DSL for writing tasks that run on > Samza. I think I can swap Samza with Kafka Streams in an unobtrusive way. > -- -- Guozhang
