Thanks for the design Damon! And thanks for collaborating with me on getting a high level textual description of the key implementation idea down in writing. I think the solution is pretty elegant.
I do have concerns about how different Runners might handle ProcessContinuations for the Bounded Input case. I know Dataflow famously has two different execution modes under the hood, but I agree with the principle that ProcessContinuation.Resume should largely be in line with the expected delay, though it's by no means guaranteed AFAIK. We should also ensure this is linked from https://s.apache.org/beam-design-docs if not already. Robert Burke Beam Go Busybody On 2024/02/20 14:00:00 Damon Douglas wrote: > Hello Everyone, > > The following describes a Throttle PTransform that holds element throughput > to minimize downstream API overusage. Thank you for reading and your > valuable input. > > https://s.apache.org/beam-throttle-transform > > Best, > > Damon >
