Can I use a single-partition topic for leader election?

2016-07-08 Thread Yi Chen
Hello, I am developing a service so that all clustered nodes form a consumer group. I also need to run some logic on only one of the node. Can I use a special single-partition topic for leader election? That is, in my node I can use ConsumerRebalanceListener to make sure that if the "leader"

Re: Can I access Kafka Streams Key/Value store outside of Processor?

2016-06-29 Thread Yi Chen
Streams is a good one. We are currently > working on having the state stores accessible like you mention and should > be able to share some design docs shortly. > > Thanks > Eno > > > On 19 Jun 2016, at 19:49, Yi Chen <y...@symphonycommerce.com> wrote: > > >

Can I access Kafka Streams Key/Value store outside of Processor?

2016-06-19 Thread Yi Chen
Hello, I am thinking of using the Kafka Steams feature to "unify" our real-time and scheduled workflow. An example is that in our workflow with stages A--> B --> C, the A --> B segment can be achieved in real-time, but B-->C segment is usually a done with a scheduled job, running maybe once per