Re: Streaming RPC calls

[Martin Jericho]

Hi Graham   Thanks for your interest.  I have included some source files to show you how I did the streaming.  It's not a complete set so it doesn't compile, but it should give you an idea how the client side works at least.  (Sorry for posting an attachment everyone!  It's only 10K)   The original BatchItemStreamArray class as generated by WSDL2Java simply contains an array of BatchItem objects called batchItems.  Either the client or server can use this default implementation, at the risk of running into memory problems if the array is large.  The new BatchItemStreamArray class emulates the original class, but extends the StreamArray class, which contains all the hooks necessary to serialize/deserialize on the fly without having to actually create the array.  It's all a bit unrefined at the moment, since I'm just doing a proof of concept as a first step.   The application I am designing must be able to cope with hundreds of thousands or even millions of records.  The size of each record is dynamic, so I can't put an exact figure on the number of MB.   Does Glue have the same limitations as axis in this regard?   ----- Original Message ----- From: graham glass To: [EMAIL PROTECTED] Sent: Monday, September 23, 2002 2:38 PM Subject: RE: Streaming RPC calls hi martin,   out of interest, how would you write an interceptor to process inputs and outputs during the streaming process? one advantage of having a DOM represention is that it's easy to manipulate during inbound/outbound messaging using tools like xalan. would you provide some kind of streaming interceptor interface?   also, out of interest, how large is the data you are sending (on the wire)?   cheers, graham -----Original Message----- From: Martin Jericho [mailto:[EMAIL PROTECTED]] Sent: Sunday, September 22, 2002 11:31 PM To: [EMAIL PROTECTED] Subject: Streaming RPC calls I have implemented a special class which allows streaming of complex bean types in an RPC call, but have found that AXIS stores the whole message in memory before sending / deserializing it anyway, which defeats the whole purpose of what I am trying to do.   The first problem is that AXIS doesn't support chunked http messages, so the content length must be known before sending it.   QUESTION 1:  Are there any plans to implement chunked transfer encoding soon?   The SOAPPart class stores the entire message as a byte array before it is sent across the wire, which apart from the reason above is unnecessary.   QUESTION 2:  Are there any plans to clean this up to allow direct streaming from the serializers onto the wire?   I have only researched the client side to see how and why the entire message is stored in memory before sending it.  I have also noticed that the server stores it all in memory before deserializing it, but I haven't investigated whether there is any good reason for this.   QUESTION 3:  If AXIS won't support this any time soon, which I assume is the case, does anyone know whether the Sun JAXRPC reference implementation suffers the same problem?  I would also like to know whether the MS .NET implementation has the same problem if anyone knows about it.   QUESTION 4:  All I really want to do is send a large array of structured data in a platform independent way.  I would like to use the standard RPC encoding of SOAP to avoid having to define my own XML schemas for the data, but I'm not sure whether today's SOAP implementations are mature enough to use for this purpose.  What do other people do in this situation?  I can't imagine I'm the only one.   I would be more than happy to share my streaming serialization/deserialization code if anyone is interested, although it is not of much use with the current version of axis.   Thanks Martin Jericho

RPCStreaming.zip
Description: Binary data