Hi Beam Community, I’m reaching out for some guidance on a Beam Flink streaming job I’m working on.
We are reading from a Kafka topic, where the traffic can be either sparse or high-volume, and we need to run a piece of logic periodically, roughly every 5 minutes. We considered using @Timer, but based on the Beam docs, timers require keyed state, which introduces a shuffle. We would like to avoid this shuffle if possible. Is there a way to trigger periodic logic in a Beam pipeline without causing a data shuffle? Thanks, Vamsi
