Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-103530579
Manually merged
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-102320172
Oh, and when rebasing to the current master please pay attention to 4786b56.
---
If your project is set up for it, you can reply to this email and have your
reply
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-102319519
A couple of comments:
* As I see you do not introduce the PollingSourceFunction, this leads to
the following point :)
* I would have the Flume package
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-101663155
I rebased this PR on top of the latest work on the checkpointing. What are
the opinions on merging this?
---
If your project is set up for it, you can reply to this
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-101684152
I'll do another pass tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-101721428
I will also try to have a closer look later today or tomorrow.
First rough glance looks good!
---
If your project is set up for it, you can reply to this email
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-100953248
I think I addressed all the (reasonable) comments you made? The user facing
API does not change in any way from this and I tried to pick consistent names
for the
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-100144884
I think this would not solve our problems. I will start a discussion thread
on the dev list.
---
If your project is set up for it, you can reply to this email and have
Github user mbalassi commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-99984916
@aljoscha I think we have two options:
* Those sources with return constant false in reachedEnd() and killed
externally when the user desires to do so.
*
GitHub user aljoscha opened a pull request:
https://github.com/apache/flink/pull/659
[FLINK-1977] Rework Stream Operators to always be push based
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aljoscha/flink event-time
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29850458
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29850741
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851342
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851632
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java
---
@@ -36,9
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854075
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/StreamSource.java
---
@@ -17,33 +17,56
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854889
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OutputHandler.java
---
@@ -210,4 +210,36
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854824
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/TwoInputStreamOperator.java
---
@@ -0,0
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854565
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/flume/FlumeSource.java
---
@@ -1,149
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855036
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/windowing/StreamDiscretizer.java
---
@@
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855104
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/co/CoStreamWindow.java
---
@@ -174,21
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855250
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855265
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/test/java/org/apache/flink/streaming/api/operators/GroupedReduceTest.java
---
@@ -23,6
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29856594
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/windowing/StreamDiscretizer.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854456
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/windowing/StreamDiscretizer.java
---
@@
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854032
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/flume/FlumeSource.java
---
@@ -1,149
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854531
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/graph/JSONGenerator.java
---
@@ -165,11 +165,10 @@
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854598
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/Source.java
---
@@ -0,0 +1,87 @@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854617
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854969
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/SourceStreamTask.java
---
@@ -0,0 +1,75
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854975
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/SourceStreamTask.java
---
@@ -0,0 +1,75
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854951
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/windowing/GroupedActiveDiscretizer.java
---
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855201
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/test/java/org/apache/flink/streaming/api/SourceTest.java
---
@@ -53,18 +47,22 @@ public
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855135
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855133
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/TwoInputStreamTask.java
---
@@ -24,56
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29856484
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/StreamSource.java
---
@@ -17,33 +17,56
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29850553
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851037
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/Source.java
---
@@ -0,0 +1,87 @@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851488
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851736
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/Source.java
---
@@ -0,0 +1,87 @@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851871
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/Source.java
---
@@ -0,0 +1,87 @@
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855288
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29856097
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/OneInputStreamOperator.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29856563
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/windowing/GroupedActiveDiscretizer.java
---
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851217
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851924
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/environment/StreamExecutionEnvironment.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29853065
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/Source.java
---
@@ -0,0 +1,87 @@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854243
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/windowing/GroupedActiveDiscretizer.java
---
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854771
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/co/CoStreamWindow.java
---
@@ -174,21
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854768
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854734
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29850604
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29851250
--- Diff:
flink-staging/flink-streaming/flink-streaming-connectors/src/main/java/org/apache/flink/streaming/connectors/kafka/api/KafkaSource.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29852045
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/Source.java
---
@@ -0,0 +1,87 @@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29852972
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/FileSourceFunction.java
---
@@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29853002
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/functions/source/FileSourceFunction.java
---
@@
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854205
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/co/CoStreamWindow.java
---
@@ -174,21
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29853728
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/graph/JSONGenerator.java
---
@@ -165,11 +165,10 @@
Github user aljoscha commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29854686
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/StreamSource.java
---
@@ -17,33 +17,56
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855315
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855206
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/OneInputStreamTask.java
---
@@ -0,0
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855517
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/runtime/tasks/TwoInputStreamTask.java
---
@@ -24,56
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29855586
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/test/java/org/apache/flink/streaming/api/SourceTest.java
---
@@ -53,18 +47,22 @@ public
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29856695
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/OneInputStreamOperator.java
---
@@ -0,0
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-99889825
Yes, I will adapt the documentation for the sources.
I think the low watermark stuff I will take out, since it does not
technically belong here.
What to
Github user rmetzger commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-99888405
I agree with Marton's comments.
The change is quite massive. Maybe it would be a good idea to throw
everything onto a cluster and verify that everything is still
Github user mbalassi commented on a diff in the pull request:
https://github.com/apache/flink/pull/659#discussion_r29857654
--- Diff:
flink-staging/flink-streaming/flink-streaming-core/src/main/java/org/apache/flink/streaming/api/operators/OneInputStreamOperator.java
---
@@ -0,0
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/659#issuecomment-99901700
By the way, how should we deal with Sources that might not have new data at
the moment but still might emit new data in the future. The interface of
reachedEnd() next()
67 matches
Mail list logo
