I just tripped on that idea while looking over my code more and searching the net. Had a duh moment…been having a few of those the past couple of days. lol.
Switched to the CSV Reader and the message went away. The only problem now is my CSV field names don't match the names of the member variables on my MyBatis beans. Been reading to see if I can map these with out having to build a new set of Pojo's that match the CSV field names. I don't want to do a copy only over field name differences if I can help it. Can I plug a CSV reader into a <jb:bean> tag instead of using the <csv:reader> tag? Is there a way to tell the binding to map my_field_name to myFieldName? It mentions setting a decoder on the binding config, but I don't need to decode the actual data type (I don't think). I just want to map the field's name to one that matches what's defined on the object. "Bean setter method [setSis_Dat_id(java.lang.String)] not found on type [com.mycompany.Dat]. You may need to set a 'decoder' on the binding config." I apologize if this is slipping more into the smooks side of things than camel (seeing as this is the camel users list). Since I'm using both technologies the line is a little blurry. If this is the wrong forum for this level of smooks questions, please let me know. Cool, thanks for the tip on beanio Claus. I'll read up on that and check it out. At this point, Camel is a definite, but I'm not locked into Smooks by any means. If it's easier to code beanio and performs well I'm good with that option. On 5/22/14 12:29 PM, "Claus Ibsen" <claus.ib...@gmail.com> wrote: >Hi > >Are you sure smooks is reading it as a CSV as the code failure is in >sax which is usually about XML. > >Btw an alternative to smooks is beanio which I find good >http://camel.apache.org/beanio >http://beanio.org/ > >On Thu, May 22, 2014 at 5:34 PM, Wilkerson, Daniel ><dwilker...@fullsail.com> wrote: >> Thank you for the quick reply Claus. I sincerely appreciate it. >> >> I placed the <convertBodyTo type="String" /> ahead of the Smooks as you >> mentioned but get a strange error that I suspect is BOM characters at >>the >> beginning of the CSV file. Is there a quick, clever way to strip BOM >> characters in a route before passing along to the Smooks endpoint? >> >> aused by: org.xml.sax.SAXParseException: Content is not allowed in >>prolog. >> at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown >>Source) >> at org.milyn.delivery.sax.SAXParser.parse(SAXParser.java:76) >> at >> org.milyn.delivery.sax.SmooksSAXFilter.doFilter(SmooksSAXFilter.java:86) >> ... 28 more >> >> >> >> On 5/22/14 11:01 AM, "Claus Ibsen" <claus.ib...@gmail.com> wrote: >> >>>Hi >>> >>>Add a <convertBodyTo type="String"/> before the smooks so the data is >>>in a format that smooks can read. >>> >>>On Thu, May 22, 2014 at 4:55 PM, Wilkerson, Daniel >>><dwilker...@fullsail.com> wrote: >>>> Hi everyone. I working on a data integration >>>>prototype/proof-of-concept >>>>using Camel 2.12, Fuse 7.1, Smooks 1.5.2, and ActiveMQ 5.9. I'm stuck >>>>currently in a unit test I'm writing to test binding an CSV file read >>>>from the file system using the file:// component and passed to smooks >>>>for transforming directly to some MyBatis POJO's I've already coded >>>>previously. While running my unit test (without any ActiveMQ being used >>>>yet) I get the output/stacktrace below. I am stuck and not sure how to >>>>get this to work properly. I've been through the smooks and camel >>>>documentation but am hitting a wall seeing what the issue might be. >>>> >>>> I have spent some time rummaging through the component unit tests in >>>>the camel source code base, but haven't been able to find ones that are >>>>similar to what I'm doing e.g. Using CamelBluprintTestSupport with >>>>similar components fashioned using declarative XML/DSL. >>>> >>>> What I'm look for my unit test to do is confirm the binding has >>>>occurred by dumping the contents of the bound POJO to verify it >>>>contains >>>>the records from the CSV. If I can get that far success fully, then I >>>>can move on to using my Queues, the mybatis:// component, and trying >>>>pushing into my demo postgres database. I have, thankfully, been able >>>>to >>>>get a sample to work that transforms the CSV to XML and been able to >>>>route that along, however, that's not the use case I'm looking for. >>>> >>>> Any help getting unstuck would be greatly appreciated. I posted my >>>>output, smooks config, unit test bean, and my blueprint.xml file. I >>>>have >>>>structured this project using the Camel Blueprint Maven archetype. It's >>>>seems like it trying to use SAX to parse the file despite the fact that >>>>I'm trying to bind directly to a POJOŠ? >>>> >>>> Thanks you all! >>>> Daniel >>>> >>>> >>>> Test Output: >>>> >>>> Message History >>>> >>>> >>>>----------------------------------------------------------------------- >>>>-- >>>>-------------------------------------------------------------- >>>> >>>> RouteId ProcessorId Processor >>>> Elapsed (ms) >>>> >>>> [route1 ] [route1 ] >>>>[file://data_files/data?delay=360000&delete=false&fileName=dat] [ >>>>61] >>>> >>>> [route1 ] [to1 ] [smooks://smooks-config.xml >>>> ] [ 59] >>>> >>>> >>>> Exchange >>>> >>>> >>>>----------------------------------------------------------------------- >>>>-- >>>>-------------------------------------------------------------- >>>> >>>> Exchange[ >>>> >>>> Id >>>>ID-ada0481cdb71c7-fullsail-com-49652-1400704749961-0-4 >>>> >>>> ExchangePattern InOnly >>>> >>>> Headers >>>>{breadcrumbId=ID-ada0481cdb71c7-fullsail-com-49652-1400704749961-0-3, >>>>CamelFileAbsolute=false, >>>>CamelFileAbsolutePath=/Users/dwilkerson/jbdevworkspace/fs-camel-bluepri >>>>nt >>>>/data_files/data/dat.csv, CamelFileLastModified=1400704750000, >>>>CamelFileLength=6, CamelFileName=dat.csv, >>>>CamelFileNameConsumed=dat.csv, >>>>CamelFileNameOnly=dat.csv, CamelFileParent=data_files/data, >>>>CamelFilePath=data_files/data/dat.csv, CamelFileRelativePath=dat.csv, >>>>CamelRedelivered=false, CamelRedeliveryCounter=0, >>>>CamelSmooksExecutionContext={interface >>>>org.apache.camel.Exchange=Exchange[org.apache.camel.component.file.Gene >>>>ri >>>>cFileMessage@5b2f9ee1]}} >>>> >>>> BodyType org.apache.camel.component.file.GenericFile >>>> >>>> Body [Body is file based: GenericFile[dat.csv]] >>>> >>>> ] >>>> >>>> >>>> Stacktrace >>>> >>>> >>>>----------------------------------------------------------------------- >>>>-- >>>>-------------------------------------------------------------- >>>> >>>> org.apache.camel.TypeConversionException: Error during type conversion >>>>from type: org.apache.camel.component.file.GenericFile to the required >>>>type: javax.xml.transform.Source with value GenericFile[dat.csv] due >>>>java.lang.AbstractMethodError: >>>>javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z >>>>)V >>>> >>>> at >>>>org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(Bas >>>>eT >>>>ypeConverterRegistry.java:130) >>>> >>>> at >>>>org.apache.camel.core.osgi.OsgiTypeConverter.convertTo(OsgiTypeConverte >>>>r. >>>>java:108) >>>> >>>> at >>>>org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:72) >>>> >>>> at >>>>org.apache.camel.impl.MessageSupport.getBody(MessageSupport.java:47) >>>> >>>> at >>>>org.milyn.smooks.camel.processor.SmooksProcessor.getSource(SmooksProces >>>>so >>>>r.java:172) >>>> >>>> at >>>>org.milyn.smooks.camel.processor.SmooksProcessor.process(SmooksProcesso >>>>r. >>>>java:112) >>>> >>>> at >>>>org.apache.camel.impl.ProcessorEndpoint.onExchange(ProcessorEndpoint.ja >>>>va >>>>:103) >>>> >>>> at >>>>org.apache.camel.impl.ProcessorEndpoint$1.process(ProcessorEndpoint.jav >>>>a: >>>>71) >>>> >>>> at >>>>org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncPro >>>>ce >>>>ssorBridge.process(AsyncProcessorConverterHelper.java:61) >>>> >>>> at >>>>org.apache.camel.processor.SendProcessor.process(SendProcessor.java:110 >>>>) >>>> >>>> at >>>>org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceIn >>>>te >>>>rceptor.java:163) >>>> >>>> at >>>>org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErr >>>>or >>>>Handler.java:398) >>>> >>>> at >>>>org.apache.camel.processor.CamelInternalProcessor.process(CamelInternal >>>>Pr >>>>ocessor.java:191) >>>> >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:118) >>>> >>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:80) >>>> >>>> at >>>>org.apache.camel.processor.CamelInternalProcessor.process(CamelInternal >>>>Pr >>>>ocessor.java:191) >>>> >>>> at >>>>org.apache.camel.component.file.GenericFileConsumer.processExchange(Gen >>>>er >>>>icFileConsumer.java:401) >>>> >>>> at >>>>org.apache.camel.component.file.GenericFileConsumer.processBatch(Generi >>>>cF >>>>ileConsumer.java:201) >>>> >>>> at >>>>org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileCon >>>>su >>>>mer.java:165) >>>> >>>> at >>>>org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer >>>>.j >>>>ava:187) >>>> >>>> at >>>>org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.j >>>>av >>>>a:114) >>>> >>>> at >>>>java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>> >>>> at >>>>java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:3 >>>>17 >>>>) >>>> >>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>> >>>> at >>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ac >>>>ce >>>>ss$101(ScheduledThreadPoolExecutor.java:98) >>>> >>>> at >>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru >>>>nP >>>>eriodic(ScheduledThreadPoolExecutor.java:180) >>>> >>>> at >>>>java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.ru >>>>n( >>>>ScheduledThreadPoolExecutor.java:204) >>>> >>>> at >>>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecut >>>>or >>>>.java:895) >>>> >>>> at >>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j >>>>av >>>>a:918) >>>> >>>> at java.lang.Thread.run(Thread.java:695) >>>> >>>> Caused by: org.apache.camel.RuntimeCamelException: >>>>java.lang.AbstractMethodError: >>>>javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z >>>>)V >>>> >>>> at >>>>org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelp >>>>er >>>>.java:1363) >>>> >>>> at >>>>org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1005) >>>> >>>> at >>>>org.apache.camel.impl.converter.InstanceMethodTypeConverter.convertTo(I >>>>ns >>>>tanceMethodTypeConverter.java:78) >>>> >>>> at >>>>org.apache.camel.component.file.GenericFileConverter.convertTo(GenericF >>>>il >>>>eConverter.java:97) >>>> >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> >>>> at >>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja >>>>va >>>>:39) >>>> >>>> at >>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso >>>>rI >>>>mpl.java:25) >>>> >>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>> >>>> at >>>>org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1001) >>>> >>>> at >>>>org.apache.camel.impl.converter.StaticMethodFallbackTypeConverter.conve >>>>rt >>>>To(StaticMethodFallbackTypeConverter.java:62) >>>> >>>> at >>>>org.apache.camel.impl.converter.BaseTypeConverterRegistry.doConvertTo(B >>>>as >>>>eTypeConverterRegistry.java:315) >>>> >>>> at >>>>org.apache.camel.impl.converter.BaseTypeConverterRegistry.convertTo(Bas >>>>eT >>>>ypeConverterRegistry.java:113) >>>> >>>> ... 29 more >>>> >>>> Caused by: java.lang.AbstractMethodError: >>>>javax.xml.parsers.DocumentBuilderFactory.setFeature(Ljava/lang/String;Z >>>>)V >>>> >>>> at >>>>org.apache.camel.converter.jaxp.XmlConverter.createDocumentBuilderFacto >>>>ry >>>>(XmlConverter.java:957) >>>> >>>> at >>>>org.apache.camel.converter.jaxp.XmlConverter.getDocumentBuilderFactory( >>>>Xm >>>>lConverter.java:897) >>>> >>>> at >>>>org.apache.camel.converter.jaxp.XmlConverter.createDocumentBuilder(XmlC >>>>on >>>>verter.java:968) >>>> >>>> at >>>>org.apache.camel.converter.jaxp.XmlConverter.toDOMSource(XmlConverter.j >>>>av >>>>a:596) >>>> >>>> at >>>>org.apache.camel.converter.jaxp.XmlConverter.toDOMSource(XmlConverter.j >>>>av >>>>a:604) >>>> >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> >>>> at >>>>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja >>>>va >>>>:39) >>>> >>>> at >>>>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso >>>>rI >>>>mpl.java:25) >>>> >>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>> >>>> at >>>>org.apache.camel.util.ObjectHelper.invokeMethod(ObjectHelper.java:1001) >>>> >>>> ... 39 more >>>> >>>> [data_files/dat] GenericFileOnCompletion DEBUG Done processing >>>>file: GenericFile >>>> >>>> Unit Test Class: >>>> public class RouteTest extends CamelBlueprintTestSupport { >>>> >>>> @Override >>>> protected String getBlueprintDescriptor() { >>>> return "/OSGI-INF/blueprint/blueprint.xml"; >>>> } >>>> >>>> @Override >>>> public void setUp() throws Exception{ >>>> super.setUp(); >>>> // send a message >>>> template.sendBodyAndHeader("file:data_files/data", "Dat", >>>>Exchange.FILE_NAME, "dat.csv"); >>>> } >>>> >>>> @Test >>>> public void testRoute() throws Exception { >>>> // set mock expectations >>>> MockEndpoint me = getMockEndpoint("mock:results"); >>>> me.expectedBodiesReceivedInAnyOrder("Dat"); >>>> >>>> // assert mocks >>>> assertMockEndpointsSatisfied(); >>>> >>>> // assert on the debugBefore/debugAfter methods below being >>>>called as we've enabled the debugger >>>> } >>>> } >>>> >>>> Bluprint.xml: >>>> >>>> <?xml version="1.0" encoding="UTF-8"?> >>>> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"; >>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>>> xmlns:camel="http://camel.apache.org/schema/blueprint"; >>>> xsi:schemaLocation=" >>>> http://www.osgi.org/xmlns/blueprint/v1.0.0 >>>>http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd >>>> http://camel.apache.org/schema/blueprint >>>>http://camel.apache.org/schema/blueprint/camel-blueprint.xsd";> >>>> >>>> <bean id="smooks" >>>>class="org.milyn.smooks.camel.dataformat.SmooksDataFormat"> >>>> <argument value="classpath:/smooks-config.xml" /> >>>> </bean> >>>> >>>> <camelContext trace="true" id="blueprintContext" >>>>xmlns="http://camel.apache.org/schema/blueprint";> >>>> <route> >>>> <from uri="direct:in"/> >>>> <from >>>>uri="file:data_files/data?fileName=dat.csv&delete=false&delay=3 >>>>60 >>>>000" /> >>>> >>>> <to uri="smooks://smooks-config.xml"/> >>>> <to uri="mock:result"/> >>>> </route> >>>> >>>> </camelContext> >>>> </blueprint> >>>> >>>> >>>> Smooks Config: >>>> >>>> <?xml version="1.0"?> >>>> <smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"; >>>> xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd"; >>>> xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.3.xsd"; >>>> xmlns:camel="http://www.milyn.org/xsd/smooks/camel-1.4.xsd"; >>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>>> xsi:schemaLocation="http://www.milyn.org/xsd/smooks-1.1.xsd >>>>http://www.milyn.org/xsd/smooks-1.1.xsd >>>> http://www.milyn.org/xsd/smooks/javabean-1.3.xsd >>>>http://www.milyn.org/xsd/smooks/javabean-1.3.xsd >>>> http://www.milyn.org/xsd/smooks/camel-1.4.xsd >>>>http://www.milyn.org/xsd/smooks/camel-1.4.xsd";> >>>> >>>> <jb:bean beanId="datBean" class="com.mycompany.Dat" >>>>createOnElement="#document"> >>>> <!-- jb:value property="assetsCount" data=""/> >>>> <jb:value property="blocker" data=""/> >>>> <jb:value property="code" data="code"/--> >>>> <jb:value property="creditHours" data="credit_hours"/> >>>> <!-- jb:value property="ctBatchAction" data=""/--> >>>> <jb:value property="description" data="description"/> >>>> <!-- jb:value property="id" data=""/--> >>>> <jb:value property="lmsTitle" data="title"/> >>>> <!-- jb:value property="minimumBlockReleaseScore" data=""/--> >>>> <jb:value property="sisCourseId" data="sis_course_id"/> >>>> <jb:value property="sisGradeScaleId" data="sis_grade_scale_id"/> >>>> <jb:value property="sisTitle" data="title"/> >>>> <!-- jb:value property="viewable" data=""/--> >>>> </jb:bean> >>>> >>>> <camel:route beanId="datBean"> >>>> <camel:to endpoint="direct:in" /> >>>> </camel:route> >>>> </smooks-resource-list> >>> >>> >>> >>>-- >>>Claus Ibsen >>>----------------- >>>Red Hat, Inc. >>>Email: cib...@redhat.com >>>Twitter: davsclaus >>>Blog: http://davsclaus.com >>>Author of Camel in Action: http://www.manning.com/ibsen >>>hawtio: http://hawt.io/ >>>fabric8: http://fabric8.io/ >> > > > >-- >Claus Ibsen >----------------- >Red Hat, Inc. >Email: cib...@redhat.com >Twitter: davsclaus >Blog: http://davsclaus.com >Author of Camel in Action: http://www.manning.com/ibsen >hawtio: http://hawt.io/ >fabric8: http://fabric8.io/
