On Wed, Nov 16, 2011 at 10:37 AM, Claus Ibsen <claus.ib...@gmail.com> wrote: > On Tue, Nov 15, 2011 at 4:52 PM, Preben.Asmussen <p...@dr.dk> wrote: >> @Claus >> >> +1 This would be really helpfull. >> >> In addition to the problem you laid out I can tell that consuming 100 rows >> -> processing the 70 building a xml payload -> put in on a jms queue and >> then on the 71 have an error that rolles everything back ultimately can fill >> up the redolog off the db and bring it to it's knees. >> Because of this we only consume one record at the time + have a onException >> handler with backoff. >> > > Yeah I would suggest to pickup only 1 row at the time if you want to > enforce transactional integrity, > when multiple TX resources is involved such as JPA + JMS. > > Or at least limit the number of messages to pickup, to a reasonable > size, to avoid the number of participating messages in the TX gets too > big, as well the memory consumption needed would be bigger etc. > > Of course if speed is important, then you may be able to find a > reasonable batch size. >
Oh if you have multiple TX resources participating, such as from JPA to bean A to JMS Where JPA + JMS participate, then you should consider using transacted=true on the JPA consumer, if you pickup 2+ or more rows in the same TX from the JPA consumer. Now imagine that there is 5 rows being picked up from the JPA consumer. Each row will get processed as a new Exchange in the Camel route, and thus send to a JMS destination. Now imagine the first 3 Exchange is processed successfully. However the 4th Exchange fails with an exception from the bean. The exception is propagated back the JPA consumer, which initiate a full rollback on the TX. So all 5 rows is kept in the database, as well the 3 messages sent to the JMS queue is not committed either. Now you could argue that any previous successfully messages should commit, and only the last message should rollback. So we can do this by setting consumer.transacted=false on the JPA consumer. The reason for this is that we regard all previous work before the exception as successful. So in this example if consumer.transacted=true, then the 3 rows will get commited as well the 3 JMS messages. The JPA consumer will when committing a row, delete the row by default (you can adjust this behavior). So on the next JPA poll, then the JpaConsumer will only find the 2 rows (as the first 3 was committed). However the game plan changes if you do work in the Camel route after sending to the JMS queue. from JPA to bean A to JMS to bean B Now imagine that we pickup again 5 rows, the first 3 rows are processed succesfully. Now the 4th row fails, but this time in the bean B step; *after* sending to JMS. So if we have consumer.transacted=false, then it would regard all previous work *before* the exception as successful, and issue a commit. And ignore the last exception. So in this case the commit will commit the 3 rows in the database, by deleting the 3 rows, and send the 3+1 messages to the JMS queue. Here is the problem, we end up sending that 4th message to the JMS queue. To remedy this you would need to use consumer.transacted=true, to enforce rolling back all together. And thus you may want to only pickup 1 message at a time from the JPA consumer, and only have 1 message per TX. Then you would be able to commit the 3 first rows. And only rollback on that 4th row. > > >> preben >> >> >> >> >> >> -- >> View this message in context: >> http://camel.465427.n5.nabble.com/Misleading-jmx-statistics-on-jpa-component-tp4960503p4994706.html >> Sent from the Camel - Users mailing list archive at Nabble.com. >> > > > > -- > Claus Ibsen > ----------------- > FuseSource > Email: cib...@fusesource.com > Web: http://fusesource.com > Twitter: davsclaus, fusenews > Blog: http://davsclaus.blogspot.com/ > Author of Camel in Action: http://www.manning.com/ibsen/ > -- Claus Ibsen ----------------- FuseSource Email: cib...@fusesource.com Web: http://fusesource.com Twitter: davsclaus, fusenews Blog: http://davsclaus.blogspot.com/ Author of Camel in Action: http://www.manning.com/ibsen/
