I think this is something called a customized operator implementation that is taking care of exactly once processing at output.
What if any previous operators fail ? How we can make sure they also recover using EXACTLY_ONCE processing mode ? Thanks Jaspal On Thursday, October 6, 2016, Thomas Weise <thomas.we...@gmail.com> wrote: > In that case please have a look at: > > https://github.com/apache/apex-malhar/blob/master/kafka/ > src/main/java/org/apache/apex/malhar/kafka/KafkaSinglePortExactlyOnceOutp > utOperator.java > > The operator will ensure that messages are not duplicated, under the > stated assumptions. > > > On Thu, Oct 6, 2016 at 3:37 PM, Jaspal Singh <jaspal.singh1...@gmail.com > <javascript:_e(%7B%7D,'cvml','jaspal.singh1...@gmail.com');>> wrote: > >> Hi Thomas, >> >> In our case we are writing the results back to maprstreams topic based >> on some validations. >> >> >> Thanks >> Jaspal >> >> >> On Thursday, October 6, 2016, Thomas Weise <t...@apache.org >> <javascript:_e(%7B%7D,'cvml','t...@apache.org');>> wrote: >> >>> Hi, >>> >>> which operators in your application are writing to external systems? >>> >>> When you look at the example from the blog ( >>> https://github.com/DataTorrent/examples/tree/master/tutoria >>> ls/exactly-once), there is Kafka input, which is configured to be >>> idempotent. The results are written to JDBC. That operator by itself >>> supports exactly-once through transactions (in conjunction with idempotent >>> input), hence there is no need to configure the processing mode at all. >>> >>> Thomas >>> >>>
