[ 
https://issues.apache.org/jira/browse/DISPATCH-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124058#comment-16124058
 ] 

Chuck Rolke commented on DISPATCH-807:
--------------------------------------

Here is an illustration of the issue.
{noformat}

                         +----------------------+
                         |       Sender         |
                         +----------+-----------+
                                    |
 +----------------------------------|------------------------------+
 |Router                            |                              |
 |       +--------------------------|---------------------------+  |
 |       |Proton In                 v                           |  |
 |       |             #####################                    |  |
 |       |incoming-window-->   |  |  |  |  #  Q1                |  |
 |       |             #####################                    |  |
 |       |queue of frames           |                           |  |
 |       +--------------------------|---------------------------+  |
 |                                  |                              |
 |                           +------------+                        |
 |                           |pn_link_recv|                        |
 |                           +------------+                        |
 |                                  |                              |
 |       +-----------------+------------------+-----------------+  |
 |       |Message Handling |qd_message_receive|                 |  |
 |       |                 +--------+---------+                 |  |
 |       |                          v                           |  |
 |       |             #####################                    |  |
 |       |message-bytes-->     |  |  |  |  #  Q2                |  |
 |       |             #####################                    |  |
 |       |queue of dispatch         |                           |  |
 |       |buffers                   |                           |  |
 |       |                 +-------------------+                |  |
 |       |                 |qd_message_send    |                |  |
 |       +-----------------+--------+----------+----------------+  |
 |                                  |                              |
 |                           +------------+                        |
 |                           |pn_link_send|                        |
 |                           +------------+                        |
 |                                  |                              |
 |       +--------------------------|---------------------------+  |
 |       |Proton Out                v                           |  |
 |       |             #####################                    |  |
 |       |outgoing-bytes-->    |  |  |  |  #  Q3                |  |
 |       |             #####################                    |  |
 |       |                          |                           |  |
 |       +--------------------------|---------------------------+  |
 +----------------------------------|------------------------------+
                                    v
                         +----------------------+
                         |       Receiver       |
                         +----------------------+
{noformat}

Here a fast Sender is sending a large message to the router and on to a slow 
Receiver. The message bytes pass through three queues.

*Q1* is the Proton session frame queue. This queue is *not* subject to overflow 
and unlimited memory consumption. Instead this queue is protected by AMQP 
session incoming-window that limits the number of frames that the Sender is 
allowed to send. If function _pn_link_receive_ is not called then the session 
incoming-window will close and the Sender will stop sending.

*Q2* is internal to a dispatch message where bytes are pulled the Proton layer 
and stored in dispatch buffers. Recent changes for streaming limit the size of 
this buffer by pulling message bytes out of the head of this queue while it is 
being filled. In general this queue is not a problem because the when the 
router gets around to sending the message all of the bytes are pulled out of Q2 
and sent.

*Q3* is an outgoing byte queue in Proton. This queue may grow unbounded.

h3. Limiting queue size

Proposal 1 is to add limits to how large queues Q2 and Q3 may grow.
* When Q2 is above its threshold then_qd_message_receive_ stops calling 
_pn_link_recv_ and message data is backed up into Q1.
* When Q3 is above its threshold then _qd_message_send_ stops calling 
_pn_link_send_ and message data is backed up into Q2.

Proposal 2 is to have no limit applied to Q2 and just have a limit for Q3.
* When Q3 is above its threshold then _qd_message_receive_ stops calling 
_pn_link_recv_.

h3. Proposal complexity

Both proposals have to deal with multiple outbound links as message targets. 
That is, the same message bytes in Q2 are sent to multiple links and go into 
multiple Q3 instances.

Proposal 1 has two stages of throttling to restart but each stage is simple. 
* qd_message_receive only cares about the size of Q2
* qd_message_send only cares about the size of the link's session 
outgoing-bytes.

Proposal 2 has one stage of throttling but requires tighter coupling between 
the qd_message_receive and qd_message_send functions. 
* qd_message_receive has to know if any of the message's destination links have 
outgoing-bytes over their respective limits.


> Message handling requires flow control to limit memory consumption
> ------------------------------------------------------------------
>
>                 Key: DISPATCH-807
>                 URL: https://issues.apache.org/jira/browse/DISPATCH-807
>             Project: Qpid Dispatch
>          Issue Type: Bug
>          Components: Router Node
>    Affects Versions: 0.8.0
>            Reporter: Chuck Rolke
>            Assignee: Chuck Rolke
>
> Large messages coming from fast senders and going to slow receivers may 
> consume large amounts of memory.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@qpid.apache.org
For additional commands, e-mail: dev-h...@qpid.apache.org

Reply via email to