I think the CXF binding component does support attachments caching to disk to avoid loading them entirely in memory.
On Tue, Sep 23, 2008 at 10:37 AM, Jussi Nummelin <[EMAIL PROTECTED]> wrote: > We're mainly using SOAP as our incoming protocol. In typical cases we > get the actual payload data as attachments which we need to rip of > from the message and send them to further processing in the bus. These > attachments can be hundreds of megs in size. > > We're thinking of some kind of a streaming solution 'cause typically > we do not need to transform the actual payload data. The > transformation and "real" processing happens only for the incoming > message and not for the attachments. > > BR, > - Jussi - > > On Mon, Sep 22, 2008 at 5:24 PM, Guillaume Nodet <[EMAIL PROTECTED]> wrote: >> It mostly depends on the flow and components used. >> Could you give more informations ? >> >> On Mon, Sep 22, 2008 at 3:06 PM, Jussi Nummelin >> <[EMAIL PROTECTED]> wrote: >>> Hi, >>> >>> We're facing the exact same issue. And no, turning DEBUG logging off >>> does not solve it. :-) >>> >>> How to configure the flow(s) to handle huge input messages as streams? >>> Or is this something that should be configured in the binding >>> components in use? >>> >>> BR, >>> - Jussi - >>> >>> On Wed, Sep 3, 2008 at 1:31 PM, Guillaume Nodet <[EMAIL PROTECTED]> wrote: >>>> You need to turn off DEBUG logging, else the exchanges are displayed, >>>> thus the whole xml document is loaded in memory and parsed. >>>> Alternatively, you can set the system property named >>>> "org.apache.servicemix.preserveContent" to "true" if you want to keep >>>> DEBUG logging. >>>> >>>> On Wed, Sep 3, 2008 at 10:53 AM, steff aka sid >>>> <[EMAIL PROTECTED]> wrote: >>>>> >>>>> Hi, >>>>> its me again :) I'am trying to handle a huge amount of data with >>>>> servicemix >>>>> but always get an error message (I'am not so familar with tuning smx >>>>> etc.). >>>>> So what I want to do is the following: >>>>> >>>>> 1. Get an xml file with ftp (see my other posts x) ). This xml files are >>>>> about 100 to 200 of MByte. >>>>> 2. This files are send as a message to a bean component and this bean >>>>> component has to do some transformations by using Smooks. This sending >>>>> fails. >>>>> 3. After the transformation the transformed files which are shrinked at >>>>> about 50Mbyte has to be send to another ftp or folder. >>>>> >>>>> My questions: >>>>> - What is the best flow to use in such a use case, where can I define the >>>>> flow to use? >>>>> - What parameters affect this issue? >>>>> - How can I receive the message as Stream? >>>>> >>>>> Regards >>>>> /Steffen >>>>> >>>>> P.s. Here is the Exception I got: >>>>> >>>>> Exception in thread "pool-flow.seda.servicemix-bean-thread-1" >>>>> java.lang.OutOfMemoryError: Java heap space >>>>> at java.util.Arrays.copyOf(Arrays.java:2882) >>>>> at >>>>> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) >>>>> at >>>>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572) >>>>> at java.lang.StringBuffer.append(StringBuffer.java:320) >>>>> at java.io.StringWriter.write(StringWriter.java:60) >>>>> at >>>>> org.apache.xml.serializer.ToStream.writeAttrString(ToStream.java:1935) >>>>> at >>>>> org.apache.xml.serializer.ToStream.processAttributes(ToStream.java:1902) >>>>> at >>>>> org.apache.xml.serializer.ToStream.endElement(ToStream.java:1989) >>>>> at >>>>> org.apache.xalan.transformer.TransformerIdentityImpl.endElement(TransformerIdentityImpl.java:1101) >>>>> at >>>>> org.apache.xml.serializer.TreeWalker.endNode(TreeWalker.java:458) >>>>> at >>>>> org.apache.xml.serializer.TreeWalker.traverse(TreeWalker.java:149) >>>>> at >>>>> org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:389) >>>>> at >>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toResult(SourceTransformer.java:125) >>>>> at >>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toResult(SourceTransformer.java:103) >>>>> at >>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toString(SourceTransformer.java:140) >>>>> at >>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toString(SourceTransformer.java:149) >>>>> at >>>>> org.apache.servicemix.jbi.messaging.MessageExchangeImpl.display(MessageExchangeImpl.java:769) >>>>> at >>>>> org.apache.servicemix.jbi.messaging.MessageExchangeImpl.toString(MessageExchangeImpl.java:743) >>>>> at java.lang.String.valueOf(String.java:2827) >>>>> at java.lang.StringBuilder.append(StringBuilder.java:115) >>>>> at >>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:136) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) >>>>> at java.lang.Thread.run(Thread.java:619) >>>>> >>>>> I am starting my servicemix with an max Heap of 1024Mbyte. >>>>> >>>>> >>>>> ----- >>>>> Brockhaus GmbH >>>>> COMPETITIVE THROUGH KNOWLEDGE >>>>> >>>>> Web: http://www.brockhaus-gruppe.de www.brockhaus-gruppe.de / >>>>> http://www.brockhaus-group.com www.brockhaus-group.com >>>>> -- >>>>> View this message in context: >>>>> http://www.nabble.com/Huge-amount-of-data-%28about-100-MByte%29-tp19285331p19285331.html >>>>> Sent from the ServiceMix - User mailing list archive at Nabble.com. >>>>> >>>>> >>>> >>>> >>>> >>>> -- >>>> Cheers, >>>> Guillaume Nodet >>>> ------------------------ >>>> Blog: http://gnodet.blogspot.com/ >>>> >>> >> >> >> >> -- >> Cheers, >> Guillaume Nodet >> ------------------------ >> Blog: http://gnodet.blogspot.com/ >> ------------------------ >> Open Source SOA >> http://open.iona.com >> > -- Cheers, Guillaume Nodet ------------------------ Blog: http://gnodet.blogspot.com/ ------------------------ Open Source SOA http://open.iona.com
