The synapse.xml is in the original post. We have tried synapse with very large files and other types of data but have never succeeded with this data type. The data contains many spaces, carriage returns, line feeds etc. I must admit the results I am getting are weird, but I can't see what I am doing wrong. I gave up on this a month ago and now have some time to look into it again.
Currently our existing systems (and our clients) must use CData to wrap this data for sends and receives but unfortunately I cannot even get this far to test this with Synapse. Hence why I added the "*" to see where the whitespace are disappearing in the data. I will also attach a data file, when I can, but can't do that at the moment as all the data has private info in it and it is not easy to sanitise these types of files. Thanks Kim pzfreo wrote: > > Kim > > Can you please post a simple Synapse.xml and a file that demonstrates > this. I'm quite surprised because I have had successful use of Synapse > with similar record oriented data to yours. > > Paul > > On Tue, May 12, 2009 at 6:53 AM, kimhorn <[email protected]> wrote: >> >> I should also mention that not only the data at the end of the file is >> missing but data in the middle is also gone in a number of places. >> >> >> kimhorn wrote: >>> >>> In testing the CDATA issues (another stream) we have come across a more >>> serious problem. >>> We have found that synapse is loosing parts of files that have imbedded >>> CR >>> and spaces. These are fixed format files. The script at the bottom is >>> used >>> to reproduce the problem. Logging the property extracted shows the loss >>> of >>> data. As the files are not small (96KB) and are full of spaces it is >>> hard >>> to demonstrate or shows the full logs. Here is the start and end of the >>> data file only; it has 100's of records in between. The data has a "*" >>> at >>> the beginning and at the end. >>> >>> <Data Start> >>> *000000141973601M95202YBA000000014197 341316840 >>> >>> 010402030000010 >>> 000190000000000000000000000000000000000000000000000000000000000 >>> >>> 006770000000000000000000000000000000000000000000000020000000000 >>> >>> * <Data END> >>> >>> >>> Looking at the log the last part of the file is gone when Property DATA >>> is >>> logged. >>> ----------------------------------------------------------------------------- >>> >>> 2009-05-12 15:23:50,825 [-] [vfs-Worker-5] INFO LogMediator DATA = >>> *000000141973601M95202YBA000000014197 341316840 >>> >>> 010402030000010 >>> 000190000000000000000000000000000000000000000000000000000000000 >>> >>> >>> 2009-05-12 15:23:54,387 [-] [vfs-Worker-5] DEBUG LogMediator End : Log >>> mediator >>> >>> Looking at the XML created we get: >>> ---------------------------------- >>> >>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope >>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><data>*000000141973601M95202YBA000000014197 >>> >>> 000190000000000000000000000000000000000000000000000000000000000</data></soapenv:Body></soapenv:Envelope> >>> >>> >>> Clearly the last part of the file has gone. I thought the error could be >>> in the Javascript but it appears the data is already lost at the >>> property >>> when extracted from the VFS payload. Any suggestions on whats happening >>> here ? >>> >>> Could this be an issue with the XPATH property setting code? >>> >>> Version of Synapse is one of recent Stable Snapshots. >>> >>> Thanks >>> Kim >>> >>> <definitions xmlns="http://ws.apache.org/ns/synapse";> >>> <proxy name="FileCheckProxyPRS" transports="vfs"> >>> <parameter >>> name="transport.vfs.FileURI">file:///C:/work/sftpagent/testdata/prs/ndc</parameter> >>> <parameter name="transport.vfs.ContentType">text/plain</parameter> >>> <parameter name="transport.vfs.FileNamePattern">.*req</parameter> >>> <parameter name="transport.PollInterval">15</parameter> >>> <parameter >>> name="transport.vfs.MoveAfterProcess">file:///C:/work/sftpagent/testdata/prs/ndc/archive</parameter> >>> <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter> >>> <target inSequence="inSequencePRS"/> >>> </proxy> >>> <sequence name="inSequencePRS"> >>> <log level="custom"> >>> <property name="MSG" value=">>>>>>> IN"/> >>> </log> >>> <log level="custom"> >>> <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"; >>> name="DATA" expression="//axis2ns:text"/> >>> </log> >>> <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"; >>> name="claimData" expression="//axis2ns:text"/> >>> <log level="full"/> >>> <script language="js"><![CDATA[ var claimData = >>> mc.getProperty("claimData").toString(); >>> mc.setPayloadXML(<data>{claimData}</data>); >>> ]]></script> >>> <send> >>> <endpoint> >>> <address uri="vfs:file:///C:/work/sftpagent/testdata/prs"/> >>> </endpoint> >>> </send> >>> </sequence> >>> </definitions> >>> >>> >>> >>> >>> >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23496604.html >> Sent from the Synapse - Dev mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > > > > -- > Paul Fremantle > Co-Founder and CTO, WSO2 > Apache Synapse PMC Chair > OASIS WS-RX TC Co-chair > > blog: http://pzf.fremantle.org > [email protected] > > "Oxygenating the Web Service Platform", www.wso2.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > > -- View this message in context: http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23497440.html Sent from the Synapse - Dev mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
