[ http://issues.apache.org/jira/browse/AXIS2C-290?page=comments#action_12435286 ] James Clark commented on AXIS2C-290: ------------------------------------
Here's how I see this all fitting together: - mod_axis2 implements input_buffer on top of bucket brigades - mod_axis2 also implements output_stream on top of bucket brigades; it implements write_input_stream by trying to convert the input_stream to an input_buffer; if it is an input_buffer, then it makes a bucket out of each chunk of the input_buffer, and inserts each bucket into output bucket brigade without copying - there are also a variety implementations of input_buffer and of output_stream independent of Apache2 (eg in-memory, files using read(), files using mmap()) - MIME parsing operates on an input_buffer; it extracts each body part as an input_buffer - XML parsing operates on an input_stream (this fits quite well with what expat needs) - the decryption engine will read from an input_stream and will present the decrypted data as an input_stream - axiom will use an input_stream (which is typically also an input_buffer) to hold the content of binary-optimized element - serialization is split into two phases: serialization into straight XML and optimization of the XML into an MTOM package - the first serialization phase writes to an optimizing_output_stream, so that the chunks to be optimized are explictly and efficiently represented; one implementation of the serialization will support the xmlsig C14N algorithm - the second (MTOM-packaging) phase implements the optimizing_output_stream interface and writes the package to a regular output _stream - the encryption engine writes the binary encryption data to an output_stream - there's an implementation of output_stream that saves the output in an input_buffer; typically the encryption process will use this to store the encrypted data - the signing engine that computes the hash implements the optimizing_output_stream interface; when signing a message body, the serializer feeds into both the hashing computation and into the MTOM-packager; the MTOM-packager will store its output initially as a set of input_buffers, one for each part; the input_buffer for the main body will be wrapped with the header and envelope before being sent to the output_stream > Improve IO architecture to minimize copying > ------------------------------------------- > > Key: AXIS2C-290 > URL: http://issues.apache.org/jira/browse/AXIS2C-290 > Project: Axis2-C > Issue Type: Improvement > Reporter: James Clark > > At the moment, there's a lot of inefficiency in how data moves through the > system. Data (especally binary data) is copied multiple times, and often > multiple copies of potentially large data items are held in memory. The > design needs revisiting so as to minimize copying. > Apache2's native IO interface is "bucket brigades". At the moment, we're not > using this but rather the Apache 1.x compatibility layer that is built on top > of this. This is causing an unnecessary copy of data both on input and > output. The design goals should include: > - the XML parser should be able to operate directly on bytes in a bucket > brigade > - the MIME parser should be able to operate directly on bytes in a bucket > brigade > - when a binary blob in the axiom tree (what is currently called a data > handler) comes from MTOM optimizsed input, it should refer to bytes in the > Apache bucket brigade > - when a binary blob in the axiom tree is output using MTOM, that blob should > be passed for output to Apache without any copying > - when data is encrypted, the encrypted bytes should be stored once and then > passed for output to Apache without any copying > - when the message body is signed, it should be serialized once; this > serialization should both be used for signing and be passed for output to > Apache without copying (this is particularly tricky because the MTOMification > is applied for output but not for signing) > - the design is not too tightly coupled to Apache2's bucket brigades > interface. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
