Re: Handling errors in IOs

2018-02-11 Thread Motty Gruda
runner: spark runner (1.6.3) beam: 2.2.0 activemq: 5.14.3 code: Pipeline p = Pipeline.create(PipelineOptionsFactory.fromArgs(args).withValidation().create()); ConnectionFactory factory_a = new ActiveMQConnectionFactory("tcp://activemq-a:61616"); ConnectionFactory factory_b = new ActiveMQConnecti

Re: Stateful processing with session window

2018-02-11 Thread Kenneth Knowles
Hi Maurizio, What runner are you using? Are you trying this out in the DirectRunner? As far as I know, no runner supports stateful processing in session windows yet. It is probably a bug that your pipeline was accepted by the runner when it includes features that the runner cannot execute. It woul

Re: KafkaIO reading from latest offset when pipeline fails on FlinkRunner

2018-02-11 Thread Sushil Ks
Thanks, Raghu. On Tue, Feb 6, 2018 at 6:41 AM, Raghu Angadi wrote: > Hi Sushil, > > That is expected behavior. If you don't have any saved checkpoint, the > pipeline would start from scratch. It does not have any connection to > previous run. > > On Thu, Feb 1, 2018 at 1:29 AM, Sushil Ks wrote:

Re: Handling errors in IOs

2018-02-11 Thread Jean-Baptiste Onofré
Hi, here you don't use split, but different JmsIO reading from different queues (not the same). The two reads are not related. If you kill connection from one, you have to reconnect. That can be done by configuration on the ConnectionFactory. Is it what you want ? Automatically reconnect ?