[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12997437#comment-12997437 ] Alan Conway commented on QPID-2935: --- Flow control is disabled in a cluster, enabling it is QPID-3076 Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: 0.9 Attachments: QPID-2935.tgz, user_doc.txt To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12992025#comment-12992025 ] Ken Giusti commented on QPID-2935: -- Status update 2/7/2011: Resync'ed to trunk. Fixed: -) Cleanup synchronous completion case in SessionState ~line 267 -) Creates a request for IOProcessing on each completion - need to aggregate these. Additional work item: forgot to add: o) Replication of new queue thresholds for clustering. o) Documentation updates. And still TBD: o) Performance regression testing tuning. o) Sends completion on every execution.sync, else hangs. Can this be fixed? o) Hang in clustered + async store tests. Need debug fix. o) Reviews pending for management changes. o) Reviews pending for queue flow limit changes. Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: 0.9 Attachments: QPID-2935.tgz To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12991431#comment-12991431 ] Ken Giusti commented on QPID-2935: -- As of 2/7/2011: Code functionally complete, and passing simple unit tests. The following TODO items still pending: o) Performance regression testing tuning. o) Cleanup synchronous completion case in SessionState ~line 267 o) Sends completion on every execution.sync, else hangs. Can this be fixed? o) Hang in clustered + async store tests. Need debug fix. o) Creates a request for IOProcessing on each completion - need to aggregate these. o) Reviews pending for management changes. o) Reviews pending for queue flow limit changes. Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: 0.9 Attachments: QPID-2935.tgz To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12984196#action_12984196 ] Ken Giusti commented on QPID-2935: -- created svn branch for development of this feature: http://svn.apache.org/viewvc/qpid/branches/qpid-2935/ Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: Future Attachments: QPID-2935.tgz To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12966573#action_12966573 ] Ken Giusti commented on QPID-2935: -- Marnie - thanks for those links. We'll want the C++ broker's implementation to have the same look and feel as the existing Java functionality. I'll follow up with Rob with some questions. Alan - extra work necessary for clustering - duly noted. I'll be sure to develop some flow control tests against clustered brokers, and have you review them. Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: Future Attachments: QPID-2935.tgz To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12930163#action_12930163 ] Ken Giusti commented on QPID-2935: -- A high-level description of a proposed solution: *) Use credit to prevent queue overflow event. *) Associate watermarks with each Queue instance. Each queue would maintain a high and low watermark corresponding to a capacity level within the queue - the number of queued messages, for example, or the total number of message bytes enqueued. The watermarks would be constrained such that high_watermark = low_watermark: the high_watermark would indicate the level of capacity at and above which the queue is considered likely to overflow. The low_watermark would indicate the level at or below which the queue is no longer considered likely to overflow. *) Associate a state with each Queue instance that reflects the current level of data in the queue with respect to the watermarks. When the current level of data in the queue crosses the high_watermark, the state is set to blocking. When the current level of data in the queue falls below the low_watermark, the state will transition to normal from blocking *) Each message transferred to the broker will maintain a boolean blocked flag. After a message has been enqueued to all of the destination queues, the block flag will be set if one or more of the destination queues are in the blocking state. *) The transfer of any message which has a set blocked flag will not be completed from the point of view of the client until the flag is reset. *) A message's blocked flag will be reset when: 1) the state of all destination queues become normal or 2) the message is consumed from all queues. *) The message is completed once the blocked flag is reset. Issues with this approach: 1) the capacity level configured for a given producer must take into account the high_watermark setting of the potential destination queues. If the producer's capacity level is too high for a given queue (or the sum of all potential producer's capacity), the queue will overflow regardless of this solution. 2) A producer will be blocked based on the destination of the current set of outbound messages. A pending transfer of a message to a different - possibly unblocked - destination would be blocked by the current outstanding messages. This appears to be unavoidable given the 0.10 model. Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: Future Attachments: QPID-2935.tgz To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12930174#action_12930174 ] Marnie McCormack commented on QPID-2935: Hi Ken, FYI - In case you haven't already seen the docs on the flow control implementation on the Java broker: https://cwiki.apache.org/qpid/use-producer-flow-control.html https://cwiki.apache.org/qpid/producer-flow-control.html Implemented by Rob Godfrey. Regards, Marnie Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: Future Attachments: QPID-2935.tgz To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
[jira] Commented: (QPID-2935) Support best effort producer flow control within the AMQP 0.10 implementation.
[ https://issues.apache.org/jira/browse/QPID-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12929805#action_12929805 ] Ken Giusti commented on QPID-2935: -- The 0.10 AMQP spec defines the use of credit as a mechanism to control the amount of data that is transferred between a client and a server (see section 2.6.1. Flow Control in the AMQP spec 0.10). In theory, the broker should be able to throttle the production rate of a client by intelligently managing the rate for which it replenishes credit to that client. For example, the broker could delay the completion of a message transfer should it detect that one or more queues that are destinations of the message transfer are nearing their limits. The current broker implementation supports the notion of asynchronous enqueuing - that there is a potential delay between the receipt of a message and the point in at which the message completes enqueueing to all destination queues. The broker delays the completion of the message transfer (from the producer's point of view) until the enqueueing has completed. This concept may be able to be extended to consider the capacity of the destination queues: do not complete a message transfer until all destination queues have a reasonable amount of capacity available. Support best effort producer flow control within the AMQP 0.10 implementation. Key: QPID-2935 URL: https://issues.apache.org/jira/browse/QPID-2935 Project: Qpid Issue Type: New Feature Components: C++ Broker, C++ Client Affects Versions: 0.9 Environment: any Reporter: Ken Giusti Assignee: Ken Giusti Fix For: Future To what extent, if any, could producer flow control be supported on the existing (pre-1.0) protocol? In the current C++ broker/client implementation, when a queue on the broker fills to the point where it cannot accept any more messages (--default-queue-limit hit), the broker will forcibly disconnect any client that attempts to route a message to that queue. This is an abrupt failure - the producing client is not privy to the queue's remaining capacity. The broker provides no feedback to the producing client, which could be used to throttle the client's message production rate. The purpose of this JIRA is to explore the possible methods for implementing producer throttling on the current 0.10 C++ codebase. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online. - Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:dev-subscr...@qpid.apache.org
