Re: Event Grouping

2022-05-02 Thread Liam Clarke-Hutchinson
Hi Dave, Okay, fair enough. Given that you've got a key to group on, and a sensors_per_cycle to tell you how many events to expect, contingent on data volumes you could easily aggregate in a regular ol Python map of key to a list of events using Python clients to consume them and then produce the

Re: Event Grouping

2022-05-02 Thread Dave Greeko
Hi Liam, Thanks for the answer and you are right; the question is how to aggregate input events into another topic. I have looked into streams but my initial understanding is that the stream API can be done in Java only. Our development environment is strictly Python and while at it, we also lo

Re: Event Grouping

2022-05-02 Thread Liam Clarke-Hutchinson
Hi Dave, I think you forgot to ask your question :D However, if you're looking to group those events from the input topic and then putting the aggregated event onto another topic, have you looked into Kafka Streams? https://kafka.apache.org/31/documentation/streams/ Cheers, Liam On Tue, 3 May