[
http://issues.apache.org/jira/browse/AXIS2C-290?page=comments#action_12435285 ]
James Clark commented on AXIS2C-290:
------------------------------------
Here's a sketch (in pseudo-C++, with error-handling removed for simplicity) of
a design that I think may work for this.
/* An input_stream is a list of chunks. There is a current position,
which is always between chunks. */
class input_stream {
void free();
/* returns the content of the chunk immediately following the
current position; the returned pointer is valid until the next
call to a non-const method on the object. If there is no chunk
following the current position, ptr gets null and len gets 0. */
void get(const char **ptr, size_t *len) const;
/* Moves the current position forward by one chunk. Does nothing if there
is no chunk following the current position. */
void next();
/* If the input_stream is an input_buffer, return the input_buffer;
otherwise return null */
input_buffer *as_input_buffer();
/* Copies the input_stream starting from the current
position. Typically this will not actually copy data, but will
just bump some reference counts. If this input_stream is an
input_buffer, then the copy will also be an input_buffer; the
position stack in the returned copy will be empty. */
input_stream *copy() const;
};
/* An input_buffer is an input_stream with some additional
functionality. This allows a range of the input_buffer to be
extracted out and used as an independent input_buffer without
copying. */
class input_buffer : input_stream {
/* Splits the chunk immediately following the current position into
chunks. Must have 0 < n < len, where len is the length of that
chunk. The current position is between the two new chunks. */
void split(size_t n);
/* Pushes the current position onto a stack of positions maintained
by the input buffer. */
void push_position();
/* Pops off the top-most member of the position stack. */
void pop_position();
/* Changes the current position to be that of the top-most member of
the position stack. The position stack is unchanged. */
void goto_position();
/* Copies the range of the input_buffer from the top-most position
of the position stack to the current position. The returned input
buffer will have an empty position stack. Typically this will not
actually copy data, but will just bump some reference counts.
The chunk structure of the copied range will be preserved. */
input_buffer *copy_from_position() const;
};
class output_stream {
void free();
/* Writes n bytes of data at ptr. */
void write(const char *ptr, size_t n);
/* Writes the content of the input_stream. Ownership of the
input_stream is transferred. Default implementation can just
iterate over input_stream, calling write and then free the
input_stream. Other implementations might use as_input_buffer. */
void write_input_stream(input_stream *);
void flush();
void close();
};
class optimizing_output_stream : output_stream {
/* Same as write_input_stream, except that the base64 encoding of
the input_stream is written. */
void write_input_stream_base64(input_stream *in);
};
class text_node : node {
/* If the text node has a binary value, then return that binary
value. The value of the text node is unchanged. */
const input_stream *get_binary_value() const;
/* If the text node has a binary value, then return that binary
value and make the value of the node empty. The ownership of the
input_stream passes to the caller. Return null if the node does
not have a binary value. */
input_stream *extract_binary_value();
/* Sets the value of the node to the base64 encoding of the
input_stream. Ownership of the input buffer is transferred. */
void set_binary_value(input_stream *);
};
/* Makes an output_stream and input_buffer pair, where the output
stream writes to the input_buffer; this allows us to save the output
of a serialization and reoutput it later efficiently. */
output_stream *make_output_stream_input_buffer(input_buffer **);
> Improve IO architecture to minimize copying
> -------------------------------------------
>
> Key: AXIS2C-290
> URL: http://issues.apache.org/jira/browse/AXIS2C-290
> Project: Axis2-C
> Issue Type: Improvement
> Reporter: James Clark
>
> At the moment, there's a lot of inefficiency in how data moves through the
> system. Data (especally binary data) is copied multiple times, and often
> multiple copies of potentially large data items are held in memory. The
> design needs revisiting so as to minimize copying.
> Apache2's native IO interface is "bucket brigades". At the moment, we're not
> using this but rather the Apache 1.x compatibility layer that is built on top
> of this. This is causing an unnecessary copy of data both on input and
> output. The design goals should include:
> - the XML parser should be able to operate directly on bytes in a bucket
> brigade
> - the MIME parser should be able to operate directly on bytes in a bucket
> brigade
> - when a binary blob in the axiom tree (what is currently called a data
> handler) comes from MTOM optimizsed input, it should refer to bytes in the
> Apache bucket brigade
> - when a binary blob in the axiom tree is output using MTOM, that blob should
> be passed for output to Apache without any copying
> - when data is encrypted, the encrypted bytes should be stored once and then
> passed for output to Apache without any copying
> - when the message body is signed, it should be serialized once; this
> serialization should both be used for signing and be passed for output to
> Apache without copying (this is particularly tricky because the MTOMification
> is applied for output but not for signing)
> - the design is not too tightly coupled to Apache2's bucket brigades
> interface.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
