We got around this now by zipping the files and changing the send/receive web services; thus CData not necessary, for now, and works... Also I think may have been an issue with the particular snapshot I used; as I see it has other problems.
I do think that CData should be addressed ASAP, in 1.3, so Synapse can work with standard web services. The issue is also one of maintainability. If the synapse we use has to be patched, to add in different libraries for CDATA, after your releases, then things soon gets out of hand, patching patches and multiple releases of components. Very easy for an incompatibility to happen and as we will have a different Synapse to everyone else makes it difficult to report issues. So we will still wait for CData to be implemented before using Synapse in production as customers cannot be expected to change their services, and one main aim of ESB's is legacy integration which means CData becomes a critical feature. Thanks Kim _____________________________________ Kim Horn Enterprise Architect ICSGlobal Limited / THELMA [email protected] Switch: +61 2 9247 2111 Direct: +61 2 9258 7235 Mobile: +61 437 898 060 Fax: +61 2 9247 6122 www.icsglobal.net Level 26, 201 Kent Street Sydney NSW 2000 AUSTRALIA NOTICE: This e-mail and any attachments are private and confidential and may contain legally privileged information. If you are not an authorised recipient, the copying or distribution of this e-mail and any attachments is prohibited and you must not read, print or act in reliance on this e-mail or attachments. This notice should not be removed. -----Original Message----- From: Andreas Veithen [mailto:[email protected]] Sent: Thursday, 14 May 2009 6:19 PM To: [email protected] Subject: Re: Data Loss Issue VFS Kim, I did a quick test, but I'm unable to reproduce this. To investigate this, we will definitely need an example file that reproduces this problem. Andreas On Tue, May 12, 2009 at 09:21, kimhorn <[email protected]> wrote: > > The synapse.xml is in the original post. We have tried synapse with very > large files and other types of data but have never succeeded with this data > type. The data contains many spaces, carriage returns, line feeds etc. I > must admit the results I am getting are weird, but I can't see what I am > doing wrong. I gave up on this a month ago and now have some time to look > into it again. > > Currently our existing systems (and our clients) must use CData to wrap this > data for sends and receives but unfortunately I cannot even get this far to > test this with Synapse. Hence why I added the "*" to see where the > whitespace are disappearing in the data. > > I will also attach a data file, when I can, but can't do that at the moment > as all the data has private info in it and it is not easy to sanitise these > types of files. > > Thanks > Kim > > > > pzfreo wrote: >> >> Kim >> >> Can you please post a simple Synapse.xml and a file that demonstrates >> this. I'm quite surprised because I have had successful use of Synapse >> with similar record oriented data to yours. >> >> Paul >> >> On Tue, May 12, 2009 at 6:53 AM, kimhorn <[email protected]> wrote: >>> >>> I should also mention that not only the data at the end of the file is >>> missing but data in the middle is also gone in a number of places. >>> >>> >>> kimhorn wrote: >>>> >>>> In testing the CDATA issues (another stream) we have come across a more >>>> serious problem. >>>> We have found that synapse is loosing parts of files that have imbedded >>>> CR >>>> and spaces. These are fixed format files. The script at the bottom is >>>> used >>>> to reproduce the problem. Logging the property extracted shows the loss >>>> of >>>> data. As the files are not small (96KB) and are full of spaces it is >>>> hard >>>> to demonstrate or shows the full logs. Here is the start and end of the >>>> data file only; it has 100's of records in between. The data has a "*" >>>> at >>>> the beginning and at the end. >>>> >>>> <Data Start> >>>> *000000141973601M95202YBA000000014197 341316840 >>>> >>>> 010402030000010 >>>> 000190000000000000000000000000000000000000000000000000000000000 >>>> >>>> 006770000000000000000000000000000000000000000000000020000000000 >>>> >>>> * <Data END> >>>> >>>> >>>> Looking at the log the last part of the file is gone when Property DATA >>>> is >>>> logged. >>>> ----------------------------------------------------------------------------- >>>> >>>> 2009-05-12 15:23:50,825 [-] [vfs-Worker-5] INFO LogMediator DATA = >>>> *000000141973601M95202YBA000000014197 341316840 >>>> >>>> 010402030000010 >>>> 000190000000000000000000000000000000000000000000000000000000000 >>>> >>>> >>>> 2009-05-12 15:23:54,387 [-] [vfs-Worker-5] DEBUG LogMediator End : Log >>>> mediator >>>> >>>> Looking at the XML created we get: >>>> ---------------------------------- >>>> >>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope >>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><data>*000000141973601M95202YBA000000014197 >>>> >>>> 000190000000000000000000000000000000000000000000000000000000000</data></soapenv:Body></soapenv:Envelope> >>>> >>>> >>>> Clearly the last part of the file has gone. I thought the error could be >>>> in the Javascript but it appears the data is already lost at the >>>> property >>>> when extracted from the VFS payload. Any suggestions on whats happening >>>> here ? >>>> >>>> Could this be an issue with the XPATH property setting code? >>>> >>>> Version of Synapse is one of recent Stable Snapshots. >>>> >>>> Thanks >>>> Kim >>>> >>>> <definitions xmlns="http://ws.apache.org/ns/synapse";> >>>> <proxy name="FileCheckProxyPRS" transports="vfs"> >>>> <parameter >>>> name="transport.vfs.FileURI">file:///C:/work/sftpagent/testdata/prs/ndc</parameter> >>>> <parameter name="transport.vfs.ContentType">text/plain</parameter> >>>> <parameter name="transport.vfs.FileNamePattern">.*req</parameter> >>>> <parameter name="transport.PollInterval">15</parameter> >>>> <parameter >>>> name="transport.vfs.MoveAfterProcess">file:///C:/work/sftpagent/testdata/prs/ndc/archive</parameter> >>>> <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter> >>>> <target inSequence="inSequencePRS"/> >>>> </proxy> >>>> <sequence name="inSequencePRS"> >>>> <log level="custom"> >>>> <property name="MSG" value=">>>>>>> IN"/> >>>> </log> >>>> <log level="custom"> >>>> <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"; >>>> name="DATA" expression="//axis2ns:text"/> >>>> </log> >>>> <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"; >>>> name="claimData" expression="//axis2ns:text"/> >>>> <log level="full"/> >>>> <script language="js"><![CDATA[ var claimData = >>>> mc.getProperty("claimData").toString(); >>>> mc.setPayloadXML(<data>{claimData}</data>); >>>> ]]></script> >>>> <send> >>>> <endpoint> >>>> <address uri="vfs:file:///C:/work/sftpagent/testdata/prs"/> >>>> </endpoint> >>>> </send> >>>> </sequence> >>>> </definitions> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23496604.html >>> Sent from the Synapse - Dev mailing list archive at Nabble.com. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> >> >> >> -- >> Paul Fremantle >> Co-Founder and CTO, WSO2 >> Apache Synapse PMC Chair >> OASIS WS-RX TC Co-chair >> >> blog: http://pzf.fremantle.org >> [email protected] >> >> "Oxygenating the Web Service Platform", www.wso2.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >> > > -- > View this message in context: > http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23497440.html > Sent from the Synapse - Dev mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
