I have a tricky use case where a user initiates an event (by clicking a button) and then stops it (by clicking it again, losing connection, closing the browser, etc).
Each time the event starts or stops, a notification is sent to a kafka topic, with the user's id as the message key and the current timestamp, and the state of the event (started, or stopped). I'm using Kafka streaming to process these events. Based on the notifications, I need to determine the total time spent 'working', i.e the time between user clicked start, and they stopped. Per hour, per day, etc. E.g total time spent 'working' per hour, per day. Any ideas how this could be achieved, while accounting for messages arriving out of order due to latency, etc (e.g the stop notification may arrive before start)? Would the kafka streams local store be of any use here (all events by the same user will have the same message key), or should i use Redis? Or do I need an hourly job which runs and processes last hour's events?
