Kim, I did a quick test, but I'm unable to reproduce this. To investigate this, we will definitely need an example file that reproduces this problem.
Andreas On Tue, May 12, 2009 at 09:21, kimhorn <[email protected]> wrote: > > The synapse.xml is in the original post. We have tried synapse with very > large files and other types of data but have never succeeded with this data > type. The data contains many spaces, carriage returns, line feeds etc. I > must admit the results I am getting are weird, but I can't see what I am > doing wrong. I gave up on this a month ago and now have some time to look > into it again. > > Currently our existing systems (and our clients) must use CData to wrap this > data for sends and receives but unfortunately I cannot even get this far to > test this with Synapse. Hence why I added the "*" to see where the > whitespace are disappearing in the data. > > I will also attach a data file, when I can, but can't do that at the moment > as all the data has private info in it and it is not easy to sanitise these > types of files. > > Thanks > Kim > > > > pzfreo wrote: >> >> Kim >> >> Can you please post a simple Synapse.xml and a file that demonstrates >> this. I'm quite surprised because I have had successful use of Synapse >> with similar record oriented data to yours. >> >> Paul >> >> On Tue, May 12, 2009 at 6:53 AM, kimhorn <[email protected]> wrote: >>> >>> I should also mention that not only the data at the end of the file is >>> missing but data in the middle is also gone in a number of places. >>> >>> >>> kimhorn wrote: >>>> >>>> In testing the CDATA issues (another stream) we have come across a more >>>> serious problem. >>>> We have found that synapse is loosing parts of files that have imbedded >>>> CR >>>> and spaces. These are fixed format files. The script at the bottom is >>>> used >>>> to reproduce the problem. Logging the property extracted shows the loss >>>> of >>>> data. As the files are not small (96KB) and are full of spaces it is >>>> hard >>>> to demonstrate or shows the full logs. Here is the start and end of the >>>> data file only; it has 100's of records in between. The data has a "*" >>>> at >>>> the beginning and at the end. >>>> >>>> <Data Start> >>>> *000000141973601M95202YBA000000014197 341316840 >>>> >>>> 010402030000010 >>>> 000190000000000000000000000000000000000000000000000000000000000 >>>> >>>> 006770000000000000000000000000000000000000000000000020000000000 >>>> >>>> * <Data END> >>>> >>>> >>>> Looking at the log the last part of the file is gone when Property DATA >>>> is >>>> logged. >>>> ----------------------------------------------------------------------------- >>>> >>>> 2009-05-12 15:23:50,825 [-] [vfs-Worker-5] INFO LogMediator DATA = >>>> *000000141973601M95202YBA000000014197 341316840 >>>> >>>> 010402030000010 >>>> 000190000000000000000000000000000000000000000000000000000000000 >>>> >>>> >>>> 2009-05-12 15:23:54,387 [-] [vfs-Worker-5] DEBUG LogMediator End : Log >>>> mediator >>>> >>>> Looking at the XML created we get: >>>> ---------------------------------- >>>> >>>> <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope >>>> xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><data>*000000141973601M95202YBA000000014197 >>>> >>>> 000190000000000000000000000000000000000000000000000000000000000</data></soapenv:Body></soapenv:Envelope> >>>> >>>> >>>> Clearly the last part of the file has gone. I thought the error could be >>>> in the Javascript but it appears the data is already lost at the >>>> property >>>> when extracted from the VFS payload. Any suggestions on whats happening >>>> here ? >>>> >>>> Could this be an issue with the XPATH property setting code? >>>> >>>> Version of Synapse is one of recent Stable Snapshots. >>>> >>>> Thanks >>>> Kim >>>> >>>> <definitions xmlns="http://ws.apache.org/ns/synapse";> >>>> <proxy name="FileCheckProxyPRS" transports="vfs"> >>>> <parameter >>>> name="transport.vfs.FileURI">file:///C:/work/sftpagent/testdata/prs/ndc</parameter> >>>> <parameter name="transport.vfs.ContentType">text/plain</parameter> >>>> <parameter name="transport.vfs.FileNamePattern">.*req</parameter> >>>> <parameter name="transport.PollInterval">15</parameter> >>>> <parameter >>>> name="transport.vfs.MoveAfterProcess">file:///C:/work/sftpagent/testdata/prs/ndc/archive</parameter> >>>> <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter> >>>> <target inSequence="inSequencePRS"/> >>>> </proxy> >>>> <sequence name="inSequencePRS"> >>>> <log level="custom"> >>>> <property name="MSG" value=">>>>>>> IN"/> >>>> </log> >>>> <log level="custom"> >>>> <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"; >>>> name="DATA" expression="//axis2ns:text"/> >>>> </log> >>>> <property xmlns:axis2ns="http://ws.apache.org/commons/ns/payload"; >>>> name="claimData" expression="//axis2ns:text"/> >>>> <log level="full"/> >>>> <script language="js"><![CDATA[ var claimData = >>>> mc.getProperty("claimData").toString(); >>>> mc.setPayloadXML(<data>{claimData}</data>); >>>> ]]></script> >>>> <send> >>>> <endpoint> >>>> <address uri="vfs:file:///C:/work/sftpagent/testdata/prs"/> >>>> </endpoint> >>>> </send> >>>> </sequence> >>>> </definitions> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23496604.html >>> Sent from the Synapse - Dev mailing list archive at Nabble.com. >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> >> >> >> -- >> Paul Fremantle >> Co-Founder and CTO, WSO2 >> Apache Synapse PMC Chair >> OASIS WS-RX TC Co-chair >> >> blog: http://pzf.fremantle.org >> [email protected] >> >> "Oxygenating the Web Service Platform", www.wso2.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> >> > > -- > View this message in context: > http://www.nabble.com/Data-Loss-Issue-VFS-tp23496565p23497440.html > Sent from the Synapse - Dev mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
