RE: Kafka as source for batch job

2018-02-08 Thread Marchant, Hayden
com> Subject: RE: Kafka as source for batch job Hi Marchant, Yes I agree. In general, the isEndOfStream method has a very ill-defined semantic, with actually different behaviors across different Kafka connector versions. This method will definitely need to be revisited in the future (we are th

RE: Kafka as source for batch job

2018-02-08 Thread Marchant, Hayden
? Thanks Hayden From: Tzu-Li (Gordon) Tai [mailto:tzuli...@apache.org] Sent: Thursday, February 08, 2018 8:05 PM To: user@flink.apache.org; Marchant, Hayden [ICG-IT] <hm97...@imceu.eu.ssmb.com> Subject: RE: Kafka as source for batch job Hi Marchant, Yes I agree. In general, the isEndOf

RE: Kafka as source for batch job

2018-02-08 Thread Tzu-Li (Gordon) Tai
@flink.apache.org; Marchant, Hayden [ICG-IT] <hm97...@imceu.eu.ssmb.com> Subject: Re: Kafka as source for batch job   Hi Hayden,   Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that could be what you are looking for. It signals the end of the stream when consumin

RE: Kafka as source for batch job

2018-02-08 Thread Marchant, Hayden
m97...@imceu.eu.ssmb.com> Subject: Re: Kafka as source for batch job Hi Hayden, Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that could be what you are looking for. It signals the end of the stream when consuming from Kafka. Cheers, Gordon On 8 February 2018 at

Re: Kafka as source for batch job

2018-02-08 Thread Tzu-Li (Gordon) Tai
Hi Hayden, Have you tried looking into `KeyedDeserializationSchema#isEndOfStream` [1]? I think that could be what you are looking for. It signals the end of the stream when consuming from Kafka. Cheers, Gordon On 8 February 2018 at 10:44:59 AM, Marchant, Hayden (hayden.march...@citi.com)