[CONF] Apache Qpid: Network IO Interface (page edited)

[confluence]

Page Edited : qpid : Network IO Interface Network IO Interface has been edited by Aidan Skinner (Jun 29, 2009). (View changes) Content: Objective and Scope. Overview Problem Statement Current Architecture Exclusions: / Assumptions High-Level Technical Architecture Functional Requirements Non Functional Requirements Architecture Design Overview of Design Breakdown of component parts. Testing Impact Compatibility / Migration Implications Risks Objective and Scope. Overview The broker is prone to heap exhaustion, leading to OutOfMemory errors. To protect against this Producer flow control will be implemented. In order for flow control to be effective the network transport layer needs to be able to support some new requirements. Problem Statement For the broker to effectively manage its memory usage, it needs to be able to know how much is held in the network buffers or at least put an upper bound on it. It also needs to be able to constrain how much memory those buffers are using. When the broker is unable to process frames as quickly as they are being sent these buffers begin to fill up and the broker has no way to limit those. It also has no way to know how large those buffers are, making memory-based triggers for flow control difficult to implement. Current Architecture The current MINA networking uses unbounded buffers. We replace over a dozen MINA classes, none of which have any unit test coverage. We failed to get our patches upstream and haven't attempted since then. Existing test coverage is minimal (approx 30%) Improving test coverage is difficult due to poor encapsulation Poor encapsulation has lead to tight coupling of MINA to server The current behaviour of send() leaves the potential for message loss when not using transactions and violates JMS expectations MINA's internal state is currently a black box, leaving no way to determine how much memory is being used by an individual client. The way that we use MINA is suboptimal for our purpouses but is difficult to change due to the tight coupling Supporting alternative transport layers is impossible due to tight coupling Exclusions: / Assumptions No AMQP semantics are involved. This is purely to limt the size of the network buffers between the client producing AMQP frames and the broker processing them. Higher level information should be determined by the broker itself. No policy will be applied beyond blocking reads if the buffer is full. Network buffer sizes are fixed at startup and do not change TCP flow control is the only mechanism used to signal to the client that it should cease to send data High-Level Technical Architecture Functional Requirements Buffer size tuning TCP options: nodelay, keepalive, window size etc. SSL: link level encryption, do we want to consider things like certificate validation etc here or at a higher level? Non Functional Requirements Runtime loading of transport plugins? Architecture Design Common should have an interface which all transport plugins can implement and which the server and client can use. The interface would include a means to set the standard socket options and to limit it's incoming memory usage. The interface would return Overview of Design This interface should be quite simple. Methods to send, receive, flush, close, open and a method to set TCP options is likely to be sufficient. These would operate on a QpidByteBuffer, essentially MinaByteBuffer to avoid having to fix our use of expanding buffers at the same time. When processing the incoming data, one frame at a time will be processed and that frames processing will be completed before the next one is read. There will be no other data structures used to hold unprocessed frames. Breakdown of component parts. Common holds a transport layer interface which the existing MINA transport is ported too. We will also port the 0-10 client o.a.q.transport.network.io package to that interface. The server and client both use common for their network layer, and will need to be updated to use the new interface. They will need to pass through the configured socket options. The server will need to be substantively modified to push the MINA specific parts into the appropriate plugin. Testing Testing under load and handling error conditions (unexpected disconnection etc) will need to be carried out. New unit tests will need to be written to cover the transport plugins and the new interfaces. Existing test coverage in this area is minimal. Impact There is a potential effect upon performance, we will need to measure this once it has been implemented to quantify what effect, if any, it has had. Compatibility / Migration Implications Older clients connected to a new broker may suffer OOM when tcp flow control kicks in. This seems preferrable to the broker suffering OOM. Clients which upgrade their library may experience a change in behaviour of the send() method, since it may now block if the clients network buffer is full. Risks MINA is quite deeply embedded in the server and will require some work to excise it fully. This is somewhat mitigated by the decision to import mina.ByteBuffer and continue using that. Differences in behaviour of transport layer may expose other bugs in the broker which were being hidden before.

Powered by Atlassian Confluence (Version: 2.2.9 Build:#527 Sep 07, 2006) - Bug/feature request

Unsubscribe or edit your notifications preferences

--------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:commits-subscr...@qpid.apache.org