Thanks. But how about configurations for different flows, is there something that we should configure to enable streaming for messages within the NMR?
- Jussi - On Tue, Sep 23, 2008 at 12:59 PM, Guillaume Nodet <[EMAIL PROTECTED]> wrote: > I think the CXF binding component does support attachments caching to > disk to avoid loading them entirely in memory. > > On Tue, Sep 23, 2008 at 10:37 AM, Jussi Nummelin > <[EMAIL PROTECTED]> wrote: >> We're mainly using SOAP as our incoming protocol. In typical cases we >> get the actual payload data as attachments which we need to rip of >> from the message and send them to further processing in the bus. These >> attachments can be hundreds of megs in size. >> >> We're thinking of some kind of a streaming solution 'cause typically >> we do not need to transform the actual payload data. The >> transformation and "real" processing happens only for the incoming >> message and not for the attachments. >> >> BR, >> - Jussi - >> >> On Mon, Sep 22, 2008 at 5:24 PM, Guillaume Nodet <[EMAIL PROTECTED]> wrote: >>> It mostly depends on the flow and components used. >>> Could you give more informations ? >>> >>> On Mon, Sep 22, 2008 at 3:06 PM, Jussi Nummelin >>> <[EMAIL PROTECTED]> wrote: >>>> Hi, >>>> >>>> We're facing the exact same issue. And no, turning DEBUG logging off >>>> does not solve it. :-) >>>> >>>> How to configure the flow(s) to handle huge input messages as streams? >>>> Or is this something that should be configured in the binding >>>> components in use? >>>> >>>> BR, >>>> - Jussi - >>>> >>>> On Wed, Sep 3, 2008 at 1:31 PM, Guillaume Nodet <[EMAIL PROTECTED]> wrote: >>>>> You need to turn off DEBUG logging, else the exchanges are displayed, >>>>> thus the whole xml document is loaded in memory and parsed. >>>>> Alternatively, you can set the system property named >>>>> "org.apache.servicemix.preserveContent" to "true" if you want to keep >>>>> DEBUG logging. >>>>> >>>>> On Wed, Sep 3, 2008 at 10:53 AM, steff aka sid >>>>> <[EMAIL PROTECTED]> wrote: >>>>>> >>>>>> Hi, >>>>>> its me again :) I'am trying to handle a huge amount of data with >>>>>> servicemix >>>>>> but always get an error message (I'am not so familar with tuning smx >>>>>> etc.). >>>>>> So what I want to do is the following: >>>>>> >>>>>> 1. Get an xml file with ftp (see my other posts x) ). This xml files are >>>>>> about 100 to 200 of MByte. >>>>>> 2. This files are send as a message to a bean component and this bean >>>>>> component has to do some transformations by using Smooks. This sending >>>>>> fails. >>>>>> 3. After the transformation the transformed files which are shrinked at >>>>>> about 50Mbyte has to be send to another ftp or folder. >>>>>> >>>>>> My questions: >>>>>> - What is the best flow to use in such a use case, where can I define the >>>>>> flow to use? >>>>>> - What parameters affect this issue? >>>>>> - How can I receive the message as Stream? >>>>>> >>>>>> Regards >>>>>> /Steffen >>>>>> >>>>>> P.s. Here is the Exception I got: >>>>>> >>>>>> Exception in thread "pool-flow.seda.servicemix-bean-thread-1" >>>>>> java.lang.OutOfMemoryError: Java heap space >>>>>> at java.util.Arrays.copyOf(Arrays.java:2882) >>>>>> at >>>>>> java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100) >>>>>> at >>>>>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572) >>>>>> at java.lang.StringBuffer.append(StringBuffer.java:320) >>>>>> at java.io.StringWriter.write(StringWriter.java:60) >>>>>> at >>>>>> org.apache.xml.serializer.ToStream.writeAttrString(ToStream.java:1935) >>>>>> at >>>>>> org.apache.xml.serializer.ToStream.processAttributes(ToStream.java:1902) >>>>>> at >>>>>> org.apache.xml.serializer.ToStream.endElement(ToStream.java:1989) >>>>>> at >>>>>> org.apache.xalan.transformer.TransformerIdentityImpl.endElement(TransformerIdentityImpl.java:1101) >>>>>> at >>>>>> org.apache.xml.serializer.TreeWalker.endNode(TreeWalker.java:458) >>>>>> at >>>>>> org.apache.xml.serializer.TreeWalker.traverse(TreeWalker.java:149) >>>>>> at >>>>>> org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:389) >>>>>> at >>>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toResult(SourceTransformer.java:125) >>>>>> at >>>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toResult(SourceTransformer.java:103) >>>>>> at >>>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toString(SourceTransformer.java:140) >>>>>> at >>>>>> org.apache.servicemix.jbi.jaxp.SourceTransformer.toString(SourceTransformer.java:149) >>>>>> at >>>>>> org.apache.servicemix.jbi.messaging.MessageExchangeImpl.display(MessageExchangeImpl.java:769) >>>>>> at >>>>>> org.apache.servicemix.jbi.messaging.MessageExchangeImpl.toString(MessageExchangeImpl.java:743) >>>>>> at java.lang.String.valueOf(String.java:2827) >>>>>> at java.lang.StringBuilder.append(StringBuilder.java:115) >>>>>> at >>>>>> org.apache.servicemix.jbi.nmr.flow.seda.SedaQueue$1.run(SedaQueue.java:136) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) >>>>>> at java.lang.Thread.run(Thread.java:619) >>>>>> >>>>>> I am starting my servicemix with an max Heap of 1024Mbyte. >>>>>> >>>>>> >>>>>> ----- >>>>>> Brockhaus GmbH >>>>>> COMPETITIVE THROUGH KNOWLEDGE >>>>>> >>>>>> Web: http://www.brockhaus-gruppe.de www.brockhaus-gruppe.de / >>>>>> http://www.brockhaus-group.com www.brockhaus-group.com >>>>>> -- >>>>>> View this message in context: >>>>>> http://www.nabble.com/Huge-amount-of-data-%28about-100-MByte%29-tp19285331p19285331.html >>>>>> Sent from the ServiceMix - User mailing list archive at Nabble.com. >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Cheers, >>>>> Guillaume Nodet >>>>> ------------------------ >>>>> Blog: http://gnodet.blogspot.com/ >>>>> >>>> >>> >>> >>> >>> -- >>> Cheers, >>> Guillaume Nodet >>> ------------------------ >>> Blog: http://gnodet.blogspot.com/ >>> ------------------------ >>> Open Source SOA >>> http://open.iona.com >>> >> > > > > -- > Cheers, > Guillaume Nodet > ------------------------ > Blog: http://gnodet.blogspot.com/ > ------------------------ > Open Source SOA > http://open.iona.com >
