[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-18 Thread uce
Github user uce closed the pull request at: https://github.com/apache/flink/pull/471 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-18 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-83049370 Thanks for the comments. I'll address the generateSequence comment and open an issue for the optimizer deadlock detection and pipeline breaking stuff after merging this. ---

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-17 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-82555635 Once this is in, we can start removing the deadlock detection in the optimizer and the pipeline breaking caches. --- If your project is set up for it, you can reply t

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-17 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-82554365 Look good, nice test coverage and fits very well with the recent execution mode changes. Only downside: This pull request does contains many cases where only

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-17 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-82479502 I've rebased this on the current master including the `ExecutionMode` changes introduced by Stephan. This allows you to set the type of produced runtime result partiti

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-13 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-79017193 I've rebased this on the latest master and set the default I/O mode to synchronous, i.e. we currently use the simpler synchronous spilled subpartition view when consuming inte

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-12 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-78481915 PS: where we wouldn't need the asynchronous implementations are for local reads. There it should be perfectly fine to just synchronously read the spilled partitions. --- If

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-12 Thread uce
Github user uce commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-78479167 The root "cause" of all asynchronous operations is that TCP connections are shared among multiple logical channels, which are handled by a fixed number of network I/O threads.

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-12 Thread StephanEwen
Github user StephanEwen commented on the pull request: https://github.com/apache/flink/pull/471#issuecomment-78474538 Concerning the questions: 1. I think deploying after all blocking producers are finished is what we should go for as a start. It is also what people would exp

[GitHub] flink pull request: [FLINK-1350] [runtime] Add blocking result par...

2015-03-10 Thread uce
GitHub user uce opened a pull request: https://github.com/apache/flink/pull/471 [FLINK-1350] [runtime] Add blocking result partition variant - **Renames** runtime intermediate result classes (after an offline discussion with @StephanEwen I realized that calling subpartitions *queue*