stankiewicz opened a new pull request, #41395: URL: https://github.com/apache/beam/pull/41395
OutputTimer.set() in the Python FnAPI harness encodes and writes a Timer message to the data/timer output stream on every call without in-memory deduplication. _GroupIntoBatchesDoFn, _StatefulBatchElementsDoFn, and _StatefulWeightedBatchElementsDoFn previously called window_timer.set(window.end) unconditionally on every input element, emitting N redundant watermark timer writes per batch of size N (and setting both window and buffering timers even when the first element immediately reached the target batch size and flushed). Check whether the batch reaches its flush threshold first, and arm window_timer.set(window.end) (along with buffering_timer) only when the first element is processed (count == 1). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
