damccorm opened a new issue, #20706: URL: https://github.com/apache/beam/issues/20706
When using the `--isRateLimited` option for Nexmark, the system is supposed to hold events back until they are supposed to be sent, in accordance to the event generations rule. What happens today is that the events are pushed into the backlog, but are never actually replayed. This leads to only a few events being sent. When I tested this with the direct runner and flink I saw that the generator got resumed all the time, and when it gets resumed the backlog is not restored. I hacked together a solution for this locally, but even with the queue collected, most events still lack. From my tests only 1/10 make it through. I don't really know the nexmark code well enough to benchmark anymore beyond this. Imported from Jira [BEAM-11547](https://issues.apache.org/jira/browse/BEAM-11547). Original Jira may contain additional context. Reported by: rHermes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
