Introduce client flow control and broker overflow protection
------------------------------------------------------------
Key: QPID-942
URL: https://issues.apache.org/jira/browse/QPID-942
Project: Qpid
Issue Type: Bug
Components: Java Broker, Java Client
Affects Versions: M2.1
Reporter: Marnie McCormack
Assignee: Aidan Skinner
Fix For: M3
Certainly the Java, and probably the other clients do not obey flow-control
commands from the broker. The Java broker never sends them to clients either.
This is in the 0-8 spec. so not fully AMQP compliant without it.
Client Work
-------------------
Flow control to be implemented in clients. The clients must also have buffer
limiting in place prior to this, or flow-controlling a client will only shift
OOMEs from the broker to the clients. Bounded buffers plus back-pressure from
the broker will should ultimately lead to the 'send' method blocking once the
system as a whole is completely saturated. This may mean that the broker gets a
needed opportunity to chew its way through the back-log, resulting in healthy
throughputs under saturation.
Broker Work
------------------
A strategy for deciding when to flow control clients from the broker needs to
be decided upon. Flow-control has per-connection granularity, which makes
deciding when to do it on a per route basis awkward. Flow-control may be
triggered by:
1. The broker being low on memory globally across the message store and all
queues.
2. A client attempting to publish to a queue that is beyond its max-depth.
3. Based on history of destinations a client usually publishes to (or has
published to), flow-control client if one of them is beyond max-depth.
A well-conditioned application will not experience 'send' blocking, except
under exceptional loads, whereupon it will act as a safety valve to prevent
either clients or broker from throwing OOME. The other scenario that may cause
back-logs, is slow consumers without TTL.
No time estimate for this yet, as its a fairly large piece of work, and not yet
decided exactly how its to be done. Need a design proposal before starting
work, to be reviewed by the qpid-dev group.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
