[jira] [Commented] (FLINK-18767) Streaming job stuck when disabling operator chaining
[ https://issues.apache.org/jira/browse/FLINK-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170037#comment-17170037 ] Nico Kruber commented on FLINK-18767: - Just to document it here: setting a buffer timeout of -1 means "no output flusher" and if the buffer isn't filled (like in my case), it won't be flushed. I meant to use an interval of 0 which flushes always. We also confirmed that this one is working, so nothing wrong here, except my code. > Streaming job stuck when disabling operator chaining > > > Key: FLINK-18767 > URL: https://issues.apache.org/jira/browse/FLINK-18767 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.9.3, 1.10.1, 1.11.1 >Reporter: Nico Kruber >Assignee: Zhijiang >Priority: Critical > > The following code is stuck sending data from the source to the map operator. > Two settings seem to have an influence here: {{env.setBufferTimeout(-1);}} > and {{env.disableOperatorChaining();}} - if I remove either of these, the job > works as expected. > (I pre-populated my Kafka topic with one element to reproduce easily) > {code} > StreamExecutionEnvironment env = > StreamExecutionEnvironment.getExecutionEnvironment(); > // comment either these two and the job works > env.setBufferTimeout(-1); > env.disableOperatorChaining(); > Properties properties = new Properties(); > properties.setProperty("bootstrap.servers", "localhost:9092"); > properties.setProperty("group.id", "test"); > FlinkKafkaConsumer consumer = new FlinkKafkaConsumer<>("topic", > new SimpleStringSchema(), properties); > consumer.setStartFromEarliest(); > DataStreamSource input = env.addSource(consumer); > input > .map((x) -> x) > .print(); > env.execute(); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-18767) Streaming job stuck when disabling operator chaining
[ https://issues.apache.org/jira/browse/FLINK-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169904#comment-17169904 ] Zhijiang commented on FLINK-18767: -- After confirming with [~NicoK] offline, it had some misleading before and not a real problem, so close it. > Streaming job stuck when disabling operator chaining > > > Key: FLINK-18767 > URL: https://issues.apache.org/jira/browse/FLINK-18767 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.9.3, 1.10.1, 1.11.1 >Reporter: Nico Kruber >Assignee: Zhijiang >Priority: Critical > > The following code is stuck sending data from the source to the map operator. > Two settings seem to have an influence here: {{env.setBufferTimeout(-1);}} > and {{env.disableOperatorChaining();}} - if I remove either of these, the job > works as expected. > (I pre-populated my Kafka topic with one element to reproduce easily) > {code} > StreamExecutionEnvironment env = > StreamExecutionEnvironment.getExecutionEnvironment(); > // comment either these two and the job works > env.setBufferTimeout(-1); > env.disableOperatorChaining(); > Properties properties = new Properties(); > properties.setProperty("bootstrap.servers", "localhost:9092"); > properties.setProperty("group.id", "test"); > FlinkKafkaConsumer consumer = new FlinkKafkaConsumer<>("topic", > new SimpleStringSchema(), properties); > consumer.setStartFromEarliest(); > DataStreamSource input = env.addSource(consumer); > input > .map((x) -> x) > .print(); > env.execute(); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-18767) Streaming job stuck when disabling operator chaining
[ https://issues.apache.org/jira/browse/FLINK-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169769#comment-17169769 ] Nico Kruber commented on FLINK-18767: - [~zjwang] I actually tried 1.8.2 and it showed the same behaviour (but didn't try any earlier version). Why did you remove that tag? > Streaming job stuck when disabling operator chaining > > > Key: FLINK-18767 > URL: https://issues.apache.org/jira/browse/FLINK-18767 > Project: Flink > Issue Type: Bug > Components: Runtime / Network >Affects Versions: 1.9.3, 1.10.1, 1.11.1 >Reporter: Nico Kruber >Assignee: Zhijiang >Priority: Critical > > The following code is stuck sending data from the source to the map operator. > Two settings seem to have an influence here: {{env.setBufferTimeout(-1);}} > and {{env.disableOperatorChaining();}} - if I remove either of these, the job > works as expected. > (I pre-populated my Kafka topic with one element to reproduce easily) > {code} > StreamExecutionEnvironment env = > StreamExecutionEnvironment.getExecutionEnvironment(); > // comment either these two and the job works > env.setBufferTimeout(-1); > env.disableOperatorChaining(); > Properties properties = new Properties(); > properties.setProperty("bootstrap.servers", "localhost:9092"); > properties.setProperty("group.id", "test"); > FlinkKafkaConsumer consumer = new FlinkKafkaConsumer<>("topic", > new SimpleStringSchema(), properties); > consumer.setStartFromEarliest(); > DataStreamSource input = env.addSource(consumer); > input > .map((x) -> x) > .print(); > env.execute(); > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)