[CONF] Apache Qpid: Network IO Interface (page edited)

[confluence]

Page Edited : qpid : Network IO Interface Network IO Interface has been edited by Aidan Skinner (Jun 26, 2009). (View changes) Content: Objective and Scope. Overview Problem Statement Current Architecture Exclusions: / Assumptions High-Level Technical Architecture Functional Requirements Non Functional Requirements Architecture Design Overview of Design Breakdown of component parts. Testing Impact Compatibility / Migration Implications Risks Problem statement memory usage - unbounded Mina buffers Overall solution fix buffer sizes unprocessed frames in one location only full network buffers on server use tcp flow control to prevent client writing more Areas which need to be addressed one heap per session vs total job heap push back on processing network traffic to client client needs to handle pause in network processing without OOM Specific changes to address those areas broker does not retain frames outside of the network buffer. network buffers are of a fixed, configurable, size. client does not retain frames outside of the network buffer client network writes block if the buffer is full see Producer flow control for more details of the higher level aspects of this, IO work is purely focussed on the network layer, not the AMQP semantics. Current requirements which need to be carried forward Buffer size tuning TCP nodelay toggle SSL - link level encryption, what about cert validation etc? Protect-io Multi-io Write biasing Plugability Define interface Decouple protocol handling from transport Allow arbitrary plugins to be loaded at runtime ? Tasks Common Define common transport interface Client side Port to new common interface Replace MINA ByteBuffer with nio ByteBuffer for portability Broker side Port to new common interface Replace MINA ByteBuffer with nio ByteBuffer for portability Objective and Scope. Overview The broker is prone to heap exhaustion, leading to OutOfMemory errors. To protect against this Producer flow control will be implemented. In order for flow control to be effective the network transport layer needs to be able to support some new requirements. Problem Statement For the broker to effectively manage its memory usage, it needs to be able to know how much is held in the network buffers or at least put an upper bound on it. It also needs to be able to constrain how much memory those buffers are using. Current Architecture The current MINA networking uses unbounded buffers. When the broker is unable to process frames as quickly as they are being sent these buffers begin to fill up and the broker has no way to limit those. It also has no way to know how large those buffers are, making memory-based triggers for flow control difficult to implement. Exclusions: / Assumptions No AMQP semantics are involved. This is purely to bound the network buffers between the client producing AMQP frames and the broker processing them. Higher level information should be determined by the broker itself. Network buffer sizes are fixed at startup and do not change High-Level Technical Architecture Functional Requirements Buffer size tuning TCP options: nodelay, keepalive, window size etc. SSL: link level encryption, do we want to consider things like certificate validation etc here or at a higher level? Non Functional Requirements Runtime loading of transport plugins? Architecture Design Common should have an interface which all transport plugins can implement and which the server and client can use. The interface would include a means to set the standard socket options and to limit it's incoming memory usage. The interface would return Overview of Design This interface should be quite simple. Methods to send, receive, flush, close, open and a method to set TCP options is likely to be sufficient. These would operate on a QpidByteBuffer, essentially MinaByteBuffer to avoid having to fix our use of expanding buffers at the same time. Breakdown of component parts. Common holds a transport layer interface which the existing MINA transport is ported too. We will also port the 0-10 client o.a.q.transport.network.io package to that interface. The server and client both use common for their network layer, and will need to be updated to use the new interface. They will need to pass through the configured socket options. The server will need to be substantively modified to push the MINA specific parts into the appropriate plugin. Testing Testing under load and handling error conditions (unexpected disconnection etc) will need to be carried out. Impact There is a potential effect upon performance. This change should, however, be largely invisible to the user other than increasing the brokers robustness. Compatibility / Migration Implications Older clients connected to a new broker may suffer OOM when tcp flow control kicks in. This seems preferrable to the broker suffering OOM. Clients which upgrade their library may experience a change in behaviour of the send() method, since it may now block if the clients network buffer is full. Risks

Powered by Atlassian Confluence (Version: 2.2.9 Build:#527 Sep 07, 2006) - Bug/feature request

Unsubscribe or edit your notifications preferences

--------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:commits-subscr...@qpid.apache.org