Re: [DISCUSS] Release testing procedures, Flink 1.3.2

By setting properties.setProperty("batch.size", "1024"); properties.setProperty("linger.ms", "1"); In the properties passed to FlinkKafkaProducer010 (to postpone automatic flushing) and killing (kill -9 PID) the YarnTaskManager process in the middle of executing a Flink job. Thus recor

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

Sweet (maybe?)! How did you reproduce data-loss? Best, Aljoscha > On 26. Jul 2017, at 11:13, Piotr Nowojski wrote: > > It took me longer then I expected but I was able reproduce data loss with > older Flink versions while running fling in 3 nodes cluster. I have also > validated that at-least

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

It took me longer then I expected but I was able reproduce data loss with older Flink versions while running fling in 3 nodes cluster. I have also validated that at-least-once semantic is fixed for Kafka 0.10 in Flink 1.3-SNAPSHOT. Piotrek > On Jul 20, 2017, at 4:52 PM, Stephan Ewen wrote: >

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

Thank you very much, for driving this! On Thu, Jul 20, 2017 at 9:09 AM, Piotr Nowojski wrote: > Hi, > > Regarding Kafka at-least-once bug. I could try to play with Flink 1.3.1 on > a real cluster to provoke this bug, by basically repeating > KafkaProducerTestBase#testOneToOneAtLeastOnce on a lar

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

Hi, Regarding Kafka at-least-once bug. I could try to play with Flink 1.3.1 on a real cluster to provoke this bug, by basically repeating KafkaProducerTestBase#testOneToOneAtLeastOnce on a larger scale. Piotrek > On Jul 19, 2017, at 5:26 PM, Aljoscha Krettek wrote: > > Hi, > > Yes! In my o

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

Hi, Yes! In my opinion, the most critical issues are these: - https://issues.apache.org/jira/browse/FLINK-6964: Fix recovery for incremental checkpoints in StandaloneCompletedCheckpointStore - https://issues.apache.org/jira/browse/FLINK-7041

Re: [DISCUSS] Release testing procedures, Flink 1.3.2

Hi Aljoscha, Glad to see that we have a more thorough testing procedure. Could you please share us what (the critical issues you mentioned) have been broken in 1.3.0 & 1.3.1, and how the new proposed "functional testing section and a combination of systems/configurations" can cover this. This will

[DISCUSS] Release testing procedures, Flink 1.3.2

Hi Everyone, We are on the verge of starting the release process for Flink 1.3.2 and there have been some critical issues in both Flink 1.3.0 and 1.3.1. For Flink 1.3.2 I want to make very sure that we test as much as possible. For this I’m proposing a slightly changed testing procedure [1]. Th