I've been building pipelines and benchmarking Beam jobs in Dataflow.

Without windowing, latencies look pretty good (reliably sub-second) from
ingest to sink.

Once I introduce windowing even with aggressive watermark it seems it takes
at least a second (often multiple seconds) to see a window fire.

Same appears true for setting events timers in the State API; there seems a
delay between setting a timer at current event time and the timer callback
fires.

Are there any good/canonical latency benchmarks/reports for different
runners?

My next move may be to evaluate Flink in terms of these latencies, but
curious if I should be trying to get sub-second latencies out of Beam
(windowing) at all?

Reply via email to