Thilina Gunarathne wrote:
Hi Dennis,
Following is what I understood about your proposal for MTOM after
reading the chat log...Please correct me if I'm wrong..
JIBX expects a raw input stream of the SOAP envelope with
xop:include's.JIBX handles the XOP:Include internally, but it needs
Axis2 to do the mime parsing.. The interface you expects Axis2 to
provide an interface with a method to access the InputStream of the
SOAP envelope together with the methods to access
DataHandlers/inputStreams of subsequent mime parts given the
content-id.
Yes, though I'd say that it's up to the transport layer to sort out the
attachments (since attachment support is a transport issue). I haven't
looked into how this is actually implemented in Axis2.
Also it seems that you have the misunderstanding that Axis2 will build
the whole OM if somebody needs to access the data referred in a
XOP:Include..
I'm not sure how you mean this. If you're talking about incoming
messages and the user application accessing the data, then no, I'm
trying to avoid Axis2 building the whole OM. If you're talking about
messages in either direction with parts of the framework code (such as
WS-Security) accessing the data, then yes, I think Axis2 needs to build
the OM.
...
2. In the case of a MIME message (MTOM or SWA) Axis2 reads in the root
mime part containing the envelope and stores it in a DataHandler..Then
takes and InputStream from the DataHandler and gives it to the
StaxReader for the normal XML processing to be done..The data in this
mime part will be directly streamed to a file, If the the file caching
is switched on and the size of the part is bigger than the threshold
value.. Direct streaming of the contents of the Root part to the
StaXReader till the user requests the subsequent mime parts would have
been the ideal scenario. But we decided it to be a post 1.0 goal at
that time..
Okay, I hadn't realized the root part was always being buffered but this
should not be a major performance hit in and of itself.
4.References to the MIMEParser and the attachments are stored in the
MIMEHelper class and the user can access it directly if needed.. This
takes care of deferred parsing of Mime parts and storing the unused
parts in Memory or temp files for future use.. Deferred parsing of
MIME parts means that we will parse the stream only till we get the
required mime parts..
5. Axis2 starts processing of the headers and reads in the
XOP:Include, which in turn would create an OMText instance with just a
pointer to the relevant MIME part. Axis2 is not doing any actual
readings of the relevant MIME part even at this point..
What I'm discussing is the case where a data binding framework is
handling the body content, so as long as the data binding framework has
access to the MIMEHelper class everything should be fine.
6. Somebody needs to access the data in the above created OMText and
calls it's getDataHandler method. At this point Axis2 MIME parser goes
ahead and read the relevant MIME part. But the OM will **not** get
created irrespective of whether the Caching is ON or OFF.. Remaining
InputStream for the Envelope is buffered in a File or in memory as a
FileDataSource of MemoryDataSource depending on the size..
I only said the OM always gets created on output in the current code,
due to the transport code scanning the OM to find optimizable data items.
My suggestion to your proposal is to use the MIMEHelper to access the
attachments. We can also add a method to access the inputStream of the
root part without much trouble.. We are also in the process of adding
attachment streaming capability to the MIMEHelper, which is a blocker
for 1.0..
Functionality of the MIMEHelper might get delegated to SOAPMessage and
it's currently under discussion. Personally i prefer to have
SOAPMessage with well defined consistent API's for attachment
handling...
<Quote Dennis>
The second part of this interface deals with attachments. It gives the
SerializationTarget (which would be transport-dependent, of course) the
control over what actually gets sent as an attachment, and provides the
data to be output as an attachment in the form of either a stream or an
array of bytes. This would allow us to fix the current broken output
behavior which forces generation of a fully-expanded OM tree for every
message being sent, just so the transport code can check for anything it
wants to send as an attachment.
</quote>
Current behaviour of traversing the tree to find whether there are any
optimised parts is a hack till the MTOM policy comes in to play.. When
MTOM policy is there it'll define whether to send the message inside a
MIME envelope or not.. Till that we can use a message level flag like
Glen suggested in the chat to avoid traversing the tree.. Also with
with the ongoing improvements like attachments streaming ability and
Attachment API we can avoid the above behavior even in a SOAP
intermediary..
I'm not sure I completely understand your response on this. The
important issue from my standpoint is that the attachments can be added
directly by user code (or in the main case of interest, by data binding
code), without going through OMText or such.
IMO the ability to decide what data to be optimised should remained on
the hands of the user.. Even in the current impl data to be output
as attachment is given as streams or array of bytes using
DataHandlers.. Please provide a more detailed explanation of the inner
workings of the above proposed way of handling attachments..
There's a separation of concerns issue here. The data binding framework
may know that certain types of items can potentially be handled as
attachments, but doesn't directly know whether (1) attachments are
supported by the transport in use, and (2) whether a particular data
size is worth sending as an attachment. For instance, with MIME over
HTTP sending a 1KB attachment is likely to be slower and bulkier than
just sending the 1KB as base64Encoding. With other transports and
encodings this might not be the case (such as TCP/IP with block sizes).
That's why I wanted to structure things so the transport layer could
make the decisions.
- Dennis
