Daniel Rall wrote: >> That means the same data is buffered in memory (in various >> forms) a minimum of 4 times (!!). If the response contains >> large quantities of data, imagine the repercussions...
I've observed significant overhead when returning large result objects. It used to be worse, before the intermediate values were cleared in XmlRpc.Worker, but it's still wasteful. So far, the response to this has centered on how the possibilities for streaming are limited by the need to produce the response body in memory before writing the Content-length header. However, there are other ways things could be improved. 1. It might be reasonable to perform two passes over the result object, one to compute the content length, and then one to stream it out. 2. In any event, why not have XmlWriter write directly to an OutputStreamWriter that wraps a ByteArrayOutputStream (maybe itself wrapped in a BufferedWriter)? This would eliminate the need for a StringBuffer, which is one of the problem areas that Dan is worried about. 3. And, at the risk of being a one-note Johnny, I'll point out that were we to use List and Map instead of Vector and Hashtable, server side code that, for example, currently constructs a Vector in memory just to hold the results of iterating over some internal data structure might instead be able to use a custom List implementation that would effectively stream those results directly from the List's Iterator without having to form an intermediate array or Vector. Of these, #2 seems to me the easiest to contemplate. --tim
