Hi Prabushi Mam, When I am trying to process the records using below SmooksConfig.xml it is giving* out of memory* errors for files greater than 600MB. Please guide.
* SmooksConfig.xml :* <?xml version="1.0" encoding="UTF-8"?> <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"; xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd";> <params> <param name="stream.filter.type">SAX</param> <!-- <param name="inputType">input.xml</param> <param name="input.xml" type="input.type.actived">File:/C:\Work\2016\09_ESB_Auto_Mail_rpts\TestFiles\TestFileXMLCSVMap.xml </param> --> <param name="default.serialization.on">true</param> </params> <resource-config selector="catalog,book"> <resource>org.milyn.delivery.DomModelCreator</resource> </resource-config> <ftl:freemarker applyOnElement="catalog"> <ftl:template><!--<#ftl > author,title,genre <?TEMPLATE-SPLIT-PI?> --> </ftl:template> </ftl:freemarker> <ftl:freemarker applyOnElement="book"> <ftl:template> <!-- <#ftl > ${book.author},${.vars["book"].title},${.vars["book"].genre} --> </ftl:template> <param name="quote">"</param> <param name="includeFieldNames">true</param> <param name="csvFields">author,title,genre</param> <param name="seperator">,</param> <param name="messageType">CSV</param> <param name="templateDataProvider">input</param> </ftl:freemarker> </smooks-resource-list> *Proxy Service " SmooksBigFile " :* <?xml version="1.0" encoding="UTF-8"?> <proxy xmlns="http://ws.apache.org/ns/synapse"; name="SmooksBigFile" startOnLoad="true" statistics="disable" trace="disable" transports="vfs"> <target> <inSequence> <log level="custom"> <property name="property_name" value=""Inside FlatFile ***************************************** ""/> </log> <property name="FORCE_SC_ACCEPTED" scope="axis2" type="STRING" value="true"/> <property name="REST_URL_POSTFIX" scope="axis2" type="STRING" value=""/> <property action="remove" name="ClientApiNonBlocking" scope="axis2"/> <smooks config-key="gov:custom/smooksConfig.xml"> <input type="xml"/> <output type="text"/> </smooks> <property expression="fn:concat(fn:substring-after(get-property('MessageID'), 'urn:uuid:'), '.csv')" name="transport.vfs.ReplyFileName" scope="transport" type="STRING"/> <property name="OUT_ONLY" scope="default" type="STRING" value="true"/> <property name="messageType" scope="axis2" type="STRING" value="application/csv"/> <property action="remove" name="LAST_MODIFIED" scope="transport"/> <send> <endpoint> <address uri="vfs:file:///C:/Flatfile/output"/> </endpoint> </send> </inSequence> <outSequence/> <faultSequence/> </target> <parameter name="transport.vfs.Streaming">true</parameter> <parameter name="transport.PollInterval">50ms</parameter> <parameter name="transport.vfs.ContentType">application/xml</parameter> <parameter name="transport.vfs.FileURI">file:///C:/Flatfile/input</parameter> <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter> <parameter name="transport.vfs.MoveAfterFailure">file:///C:/Flatfile/failure</parameter> <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter> <parameter name="transport.vfs.FileNamePattern">.*\.xml</parameter> <parameter name="transport.vfs.Locking">disable</parameter> <parameter name="transport.vfs.MoveAfterProcess">file:///C:/Flatfile/orgFilesProcessedSuccessfully</parameter> <description/> </proxy> With Regards, Aditya On Tue, Mar 20, 2018 at 9:55 AM, Prabushi Samarakoon <[email protected]> wrote: > Hi Aditya, > > It seems the error is coming from the smooks configuration. When I removed > the xml namespace from the smooksConfig as follows, I was able to > retrieve the csv content as a text output. "catalog" tag remains in the > output, since only the "book" tag is transformed according to the template. > > *smooksConfig.xml* > > <?xml version="1.0" encoding="UTF-8"?> > <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"; > xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd";> > <params> > <param name="stream.filter.type">SAX</param> > <!-- <param name="inputType">input.xml</param> > <param name="input.xml" type="input.type.actived"> > File:/C:\Work\2016\09_ESB_Auto_Mail_rpts\TestFiles\TestFileXMLCSVMap.xml > </param> --> > <param name="default.serialization.on">true</param> > </params> > <resource-config selector="catalog,book"> > <resource>org.milyn.delivery.DomModelCreator</resource> > </resource-config> > <ftl:freemarker applyOnElement="book"> > <ftl:template> > <!-- > ${.vars["book"].author},${.vars["book"].title},${.vars["book"].genre} > --> > </ftl:template> > <param name="quote">"</param> > <param name="includeFieldNames">true</param> > <param name="csvFields">author,title,genre</param> > <param name="seperator">,</param> > <param name="messageType">CSV</param> > <param name="templateDataProvider">input</param> > </ftl:freemarker> > </smooks-resource-list> > > *Output* > > <?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv=" > http://schemas.xmlsoap.org/soap/envelope/";><soapenv:Body><ax:text > xmlns:ax="http://ws.apache.org/commons/ns/payload";><catalog> > > Gambardella Matthew,XML Developer's Guide,Computer > > > Ralls Kim,Midnight Rain,Fantasy > > Gambardella Matthew,XML Developer's Guide,Computer > > > Ralls Kim,Midnight Rain,Fantasy > > </catalog></ax:text></soapenv:Body></soapenv:Envelope> > > > Thanks, > Prabushi > > On Thu, Mar 15, 2018 at 3:09 PM, aditya shivankar < > [email protected]> wrote: > >> Subject : Smooks mediator XML TO CSV conversion. Please guide. >> >> Respected Sir, >> >> I am reading a xml file using vfs, converting it into csv, then writing >> it to another file. >> Trying to use smooks mediator for converting xml data to csv. >> Using Smooks mediator as I am trying to process very large files. >> >> Sample Input : >> >> <?xml version="1.0"?> >> <catalog> >> <book id="bk101"> >> <author>Gambardella Matthew</author> >> <title>XML Developer's Guide</title> >> <genre>Computer</genre> >> <price>44.95</price> >> <publish_date>2000-10-01</publish_date> >> <description>Epic1</description> >> </book> >> <book id="bk102"> >> <author>Ralls Kim</author> >> <title>Midnight Rain</title> >> <genre>Fantasy</genre> >> <price>5.95</price> >> <publish_date>2000-12-16</publish_date> >> <description>Epic2</description> >> </book> >> <book id="bk101"> >> <author>Gambardella Matthew</author> >> <title>XML Developer's Guide</title> >> <genre>Computer</genre> >> <price>44.95</price> >> <publish_date>2000-10-01</publish_date> >> <description>Epic1</description> >> </book> >> <book id="bk102"> >> <author>Ralls Kim</author> >> <title>Midnight Rain</title> >> <genre>Fantasy</genre> >> <price>5.95</price> >> <publish_date>2000-12-16</publish_date> >> <description>Epic2</description> >> </book> >> </catalog> >> >> Expected output : >> >> author,id,title,genre,price,publish_date,description >> Gambardella Matthew,bk101,XML Developer's Guide,Computer,44.95,2000-10-0 >> 1,Epic1 >> Ralls Kim,bk102,Midnight Rain,Fantasy,5.95,2000-12-16,Epic2 >> Gambardella Matthew,bk101,XML Developer's Guide,Computer,44.95,2000-10-0 >> 1,Epic1 >> Ralls Kim,bk102,Midnight Rain,Fantasy,5.95,2000-12-16,Epic2 >> Number of Records :4, TotalPrice : 101.8 >> >> >> >> smooksConfig I am trying : >> >> <?xml version="1.0" encoding="UTF-8"?> >> <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"; >> xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd";> >> <params> >> <param name="stream.filter.type">SAX</param> >> <!-- <param name="inputType">input.xml</param> >> <param name="input.xml" type="input.type.actived">File >> :/C:\Work\2016\09_ESB_Auto_Mail_rpts\TestFiles\TestFileXMLCSVMap.xml >> </param> --> >> <param name="default.serialization.on">true</param> >> </params> >> <resource-config selector="catalog,book"> >> <resource>org.milyn.delivery.DomModelCreator</resource> >> </resource-config> >> <ftl:freemarker applyOnElement="book"> >> <ftl:template> >> <!-- <#ftl ns_prefixes={"D":"http://ws.apache.org/ns/synapse"}> >> ${.vars["book"].author},${.vars["book"].title},${.vars["book"].genre} --> >> </ftl:template> >> <param name="quote">"</param> >> <param name="includeFieldNames">true</param> >> <param name="csvFields">author,title,genre</param> >> <param name="seperator">,</param> >> <param name="messageType">CSV</param> >> <param name="templateDataProvider">input</param> >> </ftl:freemarker> >> </smooks-resource-list> >> >> Error I am getting : >> >> [2018-03-15 13:24:49,417] [EI-Core] INFO - LogMediator property_name = >> "Inside FlatFile ***************************************** " >> [2018-03-15 13:24:50,225] [EI-Core] ERROR - runtime >> >> Error on line 2, column 75 in free-marker-template >> Expecting a string, date or number here, Expression .vars["book"].author >> is instead a freemarker.ext.dom.NodeListModel >> The problematic instruction: >> ---------- >> ==> ${.vars["book"].author} [on line 2, column 73 in free-marker-template] >> ---------- >> >> Java backtrace for programmers: >> ---------- >> freemarker.core.NonStringException: Error on line 2, column 75 in >> free-marker-template >> Expecting a string, date or number here, Expression .vars["book"].author >> is instead a freemarker.ext.dom.NodeListModel >> >> With Regards, >> Aditya >> >> >> _______________________________________________ >> Dev mailing list >> [email protected] >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > *Prabushi Samarakoon* > Software Engineer > Mobile: +94715434580 > Email: [email protected] >
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
