Indeed it can be stripped down to the question whether jms read transactions are committed as a batch or as individual reads.
On Wednesday, 19 October 2016, Roberto Coluccio <[email protected]> wrote: > Hi, > > When leveraging a JMS (Tibco implementation) source to consume messages > from a queue, are such messages immediately consumed , hence removed from > the queue, even if the flume transaction hasn't been closed yet? What > happens if the transaction is not committed (e.g. in case of errors)? > > For instance, in a topology made by: > > jms source -> memory or file channel -> hdfs sink > > with a configured batch size of 100, when 99 of them are read from JMS and > written to the channel, if an error occurs when trying to read the 100th, > to the best of my knowledge the whole batch of 100 is discarded, the > transaction canceled, and the messages tried to be re-processed. > > In such a scenario, are the first 99 messages already consumed from JMS to > be considered as lost, supposing the ones in the channel are lost as well > (because of other issues)? > > thanks a lot in advance. > > Best regards, > Roberto > -- *Attila Simon* Software Engineer Email: [email protected] [image: Cloudera Inc.]
