Team, tl;dr I would like to remove the ListenHTTP "Max Data to Receive per Second" capability because it has problems. Thoughts?
I've been experimenting with the ListenHTTP "Max Data to Receive per Second" feature and I've found some concerning results. If I set "Max Data to Receive per Second" to a low value such as "500 KB" and send a 5 MB file using InvokeHTTP, then ListenHTTP receives the file after 10 seconds. Great! But if the size of the file would take longer than the InvokeHTTP "Socket Read Timeout" (default 15 seconds), for example sending a 20 MB file (which would take 40 seconds), then after 15 seconds InvokeHTTP experiences a SocketTimeoutException and routes the file to Failure. After 40 seconds elapses, ListenHTTP completes and receives the 20 MB file! If the InvokeHTTP Failure relationship is retried, then the receiving system gets duplicate files. At larger "Max Data to Receive per Second" values such as "10 MB", SocketTimeoutExceptions are harder to reproduce but I can still make it happen by heavily loading ListenHTTP. As an alternative to this property, if a ListenHTTP is followed by ControlRate with tight back pressure in between, then a limited data rate appears to work much better. When InvokeHTTP is the client, and ListenHTTP experiences backpressure and does things like send a 503 response or deny connections or close connections, InvokeHTTP experiences various errors. This results in files going to either Retry or Failure relationships. When those are retried, then the number of files that InvokeHTTP wants to send exactly equal the number of files that ListenHTTP receives. Also great! So, I propose removing the ListenHTTP "Max Data to Receive per Second" property, with proper migration of properties. And documenting a UseCase on ListenHTTP to explain the recommended approach (backpressure) to rate limiting. Thanks for your thoughts, -- Mike
