> Tracking these metrics consistently will introduce the same problem as
> precisely tracking throttling numbers across multiple controllers, I
think,
> where either there is delay introduced to use remote data, or eventual
> consistency will introduce inaccurate data.
>
If you're talking about limit enforcement, you're right! Regarding the
concurrency on each container though, we are able to accurately track that
and we need to be able to make sure that actual concurrency is always <= C.
>
> I’m interested to know if this accuracy is important as long as actual
> concurrency <= C?
>
I don't think it is as much, no. But how do you keep <= C if you don't
accurately track?
Maybe I should say that while we cannot accurately track, we can still
guarantee <= C, we just cannot guarantee maximizing concurrency up to C.
Since the HTTP requests are done via futures in proxy, the messaging between
pool and proxy doesn't have an accurate way to get exactly C requests in
flight, but can prevent ever sending > C messages that cause the HTTP requests.
The options for this are:
- track in flight requests in the pool; passing C will cause more containers to
be used, but probably the container will always only have < C in flight.
- track in flight requests in the proxy; passing C will cause the message in
proxy to be stashed/delayed until some HTTP requests are completed, and if the
>C state remains, the pool will eventually learn this state and cause more
containers to be used.
(current impl in the PR does the latter)
Thanks
Tyson
