Thanks for explanation. I updated the wiki ;) Jan
> -----Ursprüngliche Nachricht----- > Von: Babak Vahdat [mailto:[email protected]] > Gesendet: Freitag, 13. Juni 2014 13:15 > An: [email protected] > Betreff: Re: AW: Problem with ETL example > > No, the file consumer makes use of the memory based Idempotent Consumer > here. > No matter if you make use of noop true/false it will consume each file > *only* once. Change noop to false or remove it and you will not see any > change in the behavior of the second route. The first route will > *always* consume *only* 2 times (row1.xml & row2.xml) no matter what > the noop option value is. > > To verify this one can also make use of hawtio, so just run: > > ~/dev/workspace/camel/examples/camel-example-etl>mvn > io.hawt:hawtio-maven-plugin:1.4.4:camel > > Then refresh you browser and jump to the following URL: > > http://localhost:8080/hawtio/#/jmx/attributes?tab=camel&nid=root- > org.apache.camel-camel-routes > > And you should see something like Screen_Shot_2014-06-13_at_13.png > <http://camel.465427.n5.nabble.com/file/n5752263/Screen_Shot_2014-06- > 13_at_13.png> > > As you see the the first route has the “Completed #” column count equal > to 2 and the number doesn’t increase anymore, however the second route > counts up continuously. > > Now change consumerDelete option by the second route and run hawtio > maven plugin again as above and verify the change in the behavior of > the second route. > > Babak > > > Jan Matèrne (jhm) wrote > > from("file:src/data?noop=true") > > .convertBodyTo(PersonDocument.class) > > .to("jpa:org.apache.camel.example.etl.CustomerEntity"); > > > > // the following will dump the database to files > > > > > from("jpa:org.apache.camel.example.etl.CustomerEntity?consumeDelete=fa > > lse&de > > lay=3000&consumeLockEntity=false") > > .setHeader(Exchange.FILE_NAME, > el("${in.body.userName}.xml")) > > .to("file:target/customers"); > > > > I think it is because the first route doesnt move the processed files > > to somewhere else ... > > http://camel.apache.org/file2.html > > > > > > Jan > > > > > >> -----Ursprüngliche Nachricht----- > >> Von: Babak Vahdat [mailto: > > > babak.vahdat@ > > > ] > >> Gesendet: Freitag, 13. Juni 2014 12:18 > >> An: > > > [email protected] > > >> Betreff: Re: Problem with ETL example > >> > >> Hi > >> > >> The reason for the same logs you see is because of the consumeDelete > >> option being set to false: > >> > >> http://camel.apache.org/jpa.html#JPA-Options > >> > >> So that the JPA consumer polls the *same* 2 entities again and > again. > >> This is just for the demonstration purpose so that you can > >> continuously see something is running. > >> > >> You can set that option to true or remove it (default is true) and > >> check the change in the behavior here: > >> > >> https://github.com/apache/camel/blob/master/examples/camel-example- > >> etl/src/main/java/org/apache/camel/example/etl/EtlRoutes.java#L36 > >> > >> And if not already done, maybe you want to go through the > >> documentation of this example as well: > >> > >> http://camel.apache.org/etl-example.html > >> > >> Babak > >> > >> Licia wrote > >> > Hi everyone, > >> > > >> > I'm all new to Camel and after getting documentation, I went > >> > directly to the ETL example because it really looks like what I'm > >> > trying to > >> do. > >> > > >> > So I just did mvn compile, which worked (everything did in the > >> > directory where the POM is). > >> > And then I tried mvn camel:run and I get a problem : it looks like > >> the > >> > same message get displayed over and over again. > >> > > >> > > >> > > >> > > >> > > >> > " > >> * > >> > 2014-06-13 11:07:52,094 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2) > >> > from(jpa://org.apache.camel > >> > > >> > .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal > >> s > >> > e&delay=30 > >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly, > >> > Headers:{CamelEntityManager > >> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b, > >> > breadcrumbId=ID-ITEM > >> > -93866-58463-1402650414339-0-77}, > >> > BodyType:org.apache.camel.example.etl.Customer > >> > Entity, Body:<?xml version="1.0" encoding="UTF-8" > >> > standalone="yes"?> > >> > > > <customer id="1"> > >> > > >> > > > <userName> > >> > james > >> > > > </userName> > >> > > >> > > > <firstName> > >> > James > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Strachan > >> > > > </surname> > >> > > >> > > > <city> > >> > London > >> > > > </city> > >> > > > </customer> > >> * > >> > > >> > 2014-06-13 11:07:52,096 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-78 >>> (route2) > >> > setHeader[CamelFileName] -- > >> >> file://target/customers <<< Pattern:InOnly, > >> >> Headers:{breadcrumbId=ID-ITEM-9386 > >> > 6-58463-1402650414339-0-77, CamelFileName=james.xml, > >> > CamelEntityManager=org.apac > >> > he.openjpa.persistence.EntityManagerImpl@5d84d74b}, > >> > BodyType:org.apache.camel.ex > >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF- > 8" > >> > standalone=" > >> > yes"?> > >> > > > <customer id="1"> > >> > > >> > > > <userName> > >> > james > >> > > > </userName> > >> > > >> > > > <firstName> > >> > James > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Strachan > >> > > > </surname> > >> > > >> > > > <city> > >> > London > >> > > > </city> > >> > > > </customer> > >> > 2014-06-13 11:07:52,136 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2) > >> > from(jpa://org.apache.camel > >> > > >> > .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal > >> s > >> > e&delay=30 > >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly, > >> > Headers:{CamelEntityManager > >> > =org.apache.openjpa.persistence.EntityManagerImpl@5d84d74b, > >> > breadcrumbId=ID-ITEM > >> > -93866-58463-1402650414339-0-79}, > >> > BodyType:org.apache.camel.example.etl.Customer > >> > Entity, Body:<?xml version="1.0" encoding="UTF-8" > >> > standalone="yes"?> > >> > > > <customer id="2"> > >> > > >> > > > <userName> > >> > hiram > >> > > > </userName> > >> > > >> > > > <firstName> > >> > Hiram > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Chirino > >> > > > </surname> > >> > > >> > > > <city> > >> > Tampa > >> > > > </city> > >> > > > </customer> > >> > 2014-06-13 11:07:52,137 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-80 >>> (route2) > >> > setHeader[CamelFileName] -- > >> >> file://target/customers <<< Pattern:InOnly, > >> >> Headers:{CamelEntityManager=org.ap > >> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b, > >> > breadcrumbId=ID-ITEM-93866- > >> > 58463-1402650414339-0-79, CamelFileName=hiram.xml}, > >> > BodyType:org.apache.camel.ex ample.etl.CustomerEntity, Body:<?xml > >> > version="1.0" encoding="UTF-8" > >> > standalone=" > >> > yes"?> > >> > > > <customer id="2"> > >> > > >> > > > <userName> > >> > hiram > >> > > > </userName> > >> > > >> > > > <firstName> > >> > Hiram > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Chirino > >> > > > </surname> > >> > > >> > > > <city> > >> > Tampa > >> > > > </city> > >> > > > </customer> > >> > 60513 camel TRACE [Camel (camel) thread #1 - > >> > jpa://org.apache.camel.example.e tl.CustomerEntity] > >> > openjpa.jdbc.SQL > >> - > >> > > > <t 53213167, conn 227180630> > > executing pre pstmnt 882159474 SELECT > >> > t0.id, t0.city, t0.firstName, t0.phone, t0.street, t0.surname, > >> > t0.userName, t0.zip > >> > FROM customer t0 > >> > > >> > 60514 camel TRACE [Camel (camel) thread #1 - > >> > jpa://org.apache.camel.example.e tl.CustomerEntity] > >> > openjpa.jdbc.SQL > >> - > >> > > > <t 53213167, conn 227180630> > > [0 ms] spent > >> * > >> > 2014-06-13 11:07:55,165 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2) > >> > from(jpa://org.apache.camel > >> > > >> > .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal > >> s > >> > e&delay=30 > >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly, > >> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-81, > >> > CamelEntityManager=org.apache.openjpa.persist > >> > ence.EntityManagerImpl@5d84d74b}, > >> > BodyType:org.apache.camel.example.etl.Customer > >> > Entity, Body:<?xml version="1.0" encoding="UTF-8" > >> > standalone="yes"?> > >> > > > <customer id="1"> > >> > > >> > > > <userName> > >> > james > >> > > > </userName> > >> > > >> > > > <firstName> > >> > James > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Strachan > >> > > > </surname> > >> > > >> > > > <city> > >> > London > >> > > > </city> > >> > > > </customer> > >> * > >> > > >> > 2014-06-13 11:07:55,167 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-82 >>> (route2) > >> > setHeader[CamelFileName] -- > >> >> file://target/customers <<< Pattern:InOnly, > >> >> Headers:{breadcrumbId=ID-ITEM-9386 > >> > 6-58463-1402650414339-0-81, CamelFileName=james.xml, > >> > CamelEntityManager=org.apac > >> > he.openjpa.persistence.EntityManagerImpl@5d84d74b}, > >> > BodyType:org.apache.camel.ex > >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF- > 8" > >> > standalone=" > >> > yes"?> > >> > > > <customer id="1"> > >> > > >> > > > <userName> > >> > james > >> > > > </userName> > >> > > >> > > > <firstName> > >> > James > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Strachan > >> > > > </surname> > >> > > >> > > > <city> > >> > London > >> > > > </city> > >> > > > </customer> > >> > 2014-06-13 11:07:55,196 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2) > >> > from(jpa://org.apache.camel > >> > > >> > .example.etl.CustomerEntity?consumeDelete=false&consumeLockEntity=fal > >> s > >> > e&delay=30 > >> > 00) --> setHeader[CamelFileName] <<< Pattern:InOnly, > >> > Headers:{breadcrumbId=ID-IT EM-93866-58463-1402650414339-0-83, > >> > CamelEntityManager=org.apache.openjpa.persist > >> > ence.EntityManagerImpl@5d84d74b}, > >> > BodyType:org.apache.camel.example.etl.Customer > >> > Entity, Body:<?xml version="1.0" encoding="UTF-8" > >> > standalone="yes"?> > >> > > > <customer id="2"> > >> > > >> > > > <userName> > >> > hiram > >> > > > </userName> > >> > > >> > > > <firstName> > >> > Hiram > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Chirino > >> > > > </surname> > >> > > >> > > > <city> > >> > Tampa > >> > > > </city> > >> > > > </customer> > >> > 2014-06-13 11:07:55,198 [.CustomerEntity] INFO Tracer > >> > - > >> > ID-ITEM-93866-58463-1402650414339-0-84 >>> (route2) > >> > setHeader[CamelFileName] -- > >> >> file://target/customers <<< Pattern:InOnly, > >> >> Headers:{CamelEntityManager=org.ap > >> > ache.openjpa.persistence.EntityManagerImpl@5d84d74b, > >> > CamelFileName=hiram.xml, br > >> > eadcrumbId=ID-ITEM-93866-58463-1402650414339-0-83}, > >> > BodyType:org.apache.camel.ex > >> > ample.etl.CustomerEntity, Body:<?xml version="1.0" encoding="UTF- > 8" > >> > standalone=" > >> > yes"?> > >> > > > <customer id="2"> > >> > > >> > > > <userName> > >> > hiram > >> > > > </userName> > >> > > >> > > > <firstName> > >> > Hiram > >> > > > </firstName> > >> > > >> > > > <surname> > >> > Chirino > >> > > > </surname> > >> > > >> > > > <city> > >> > Tampa > >> > > > </city> > >> > > > </customer> > >> > " > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > The two bold parts are the exact same ones. > >> > > >> > Plus I'm really confused about what the example is supposed to do. > >> > Isn't it supposed to create the java classes ? Or am I completely > >> wrong ? > >> > Else, isn't it supposed to create the persons in my database ? How > >> > do I see if it has been done ? > >> > > >> > Thanks a lot for your answers. > >> > >> > >> > >> > >> > >> -- > >> View this message in context: > >> http://camel.465427.n5.nabble.com/Problem-with-ETL-example- > >> tp5752256p5752259.html > >> Sent from the Camel - Users mailing list archive at Nabble.com. > > > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Problem-with-ETL-example- > tp5752256p5752263.html > Sent from the Camel - Users mailing list archive at Nabble.com.
