I did quite a bit of work yesterday to fix some areas where large messages were causing problems. They all pretty much centered around mis-use of the CachedOutputStream (assuming it's always a ByteArrayOutputStream which isn't true once it flips to a file). However, while digging through those, I've noticed a few other potential problems that I'd like to discuss.
1) Logging interceptors - right now, we log the entire message to the Logger. However, the logger requires a String, so the entire message is converted to a String (keep in mind, a String is utf-16 so twice the memory is consumed) for logging. Plus, do you really think logging a 10MB message to the Logger is the best option? I'm going to add a configurable "limit" that will Log just the first "limit" bytes of the message. Should the default be -1 (no limit) or something reasonable like 4K? Also, we could have a configuration to allow it to optionally log to a separate file instead of the Logger. We can then directly copy from stream to stream with very little memory usage. That's going to be a lot trickier though as we'll have to deal with concurrent access to the file from multiple threads and such. 2) JMS transport - the JMS transport always deals with the entire message as either a String (for TextMessage) or byte[]. That seems to be a limitation for standard JMS API's. Some vendors (such as ActiveMQ) do have proprietary extensions to allow some proprietary stuff to help. Examples: http://activemq.apache.org/blob-messages.html http://activemq.apache.org/jms-streams.html It might be good to allow some extra hooks into the JMS transport to allow for easy subclassing to enable some of the hooks. (That said, I'd like to see the JMS transport also changed over to using the Spring JMS configuration.) 3) WS-RM - The RMMessage object holds the message as a byte[]. Thus, it's entirely in memory until it's stored. It would probably be a good idea for this to be passed as the original CachedOutputStream and stored directly to/from that stream. Thoughts? -- J. Daniel Kulp Principal Engineer IONA P: 781-902-8727 C: 508-380-7194 [EMAIL PROTECTED] http://www.dankulp.com/blog
