mosche commented on issue #26291: URL: https://github.com/apache/beam/issues/26291#issuecomment-1510894845
> Meaning, there's no way to actually signal processing of the message is done or not - for 2 minutes the message will be considered as "in processing", and afterwards the message will be considered done and then deleted from the queue. Thanks for this feature request @AlonHarell. Would you be willing to contribute a PR to support configuring the processing timeout? > and some messages aren't deleted even though they're done This sounds like a bug. Messages that are done should be deleted in batches once a checkpoint is finalized. Have you observed otherwise? > Also, side note: Extending messages timeout is done by a blocking loop, so it isn't scalable. Absolutely yes, the entire IO still uses the old blocking client API. Re-designing / -implementing that would require quite some effort. Currently there's no such plans, though I'd be happy to provide support. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
