ppawel commented on PR #30218: URL: https://github.com/apache/beam/pull/30218#issuecomment-2039093334
I will try it out today but with this approach I am a bit worried about performance impact. I think waiting for the timeout would mean that during every checkpoint (every 10 messages?) there would be a delay with consuming further messages. This performance impact would be "masked" a bit if you leave the splits to the original logic instead of forcing to 1 but then there another problem with JmsIO comes out - #21075. In my case, this gets really bad - after some hours of streaming, we are reaching the limit of 100 broker connections and there really is no justification to ask for raising the limit because when I force the split to 1 (=one JMS connection), then the workload is handled just fine. So bottom line for my case is that until the connection blowup issue is fixed somehow, I need to force it to 1. Which means I am facing the deadlock issue, and the timeout workaround will probably not be usable for me due to high performance impact. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
