ngibanel commented on PR #37164: URL: https://github.com/apache/beam/pull/37164#issuecomment-3958163802
Hi, I have a couple of concerns regarding the use of the SEMP API in this context. 1. SEMP API is primarily a management API, and in my opinion it should not be used for acknowledging messages. In case of high throughput there is a risk that the API will be a bottleneck. There are also major security limitations such as the lack of fine‑grained per‑user access control, which means a user could potentially execute all management actions on the Message VPN. 2. The number of unacked messages is a critical metric to monitor because it creates direct backpressure on the broker. With high throughput, there is a real risk that the unacked count grows significantly. In fact, with the current implementation, the number of unacked message is already much higher than what we observe when using JmsIO. It's out of scope of this PR but it would be worth investigating whether the acknowledgment latency can be improved to reduce the number of unacked messages. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
