Hi everyone! I have started with a patch that introduces *"local collection-based execution"* of Flink programs.
Since Flink is a layered system, programs written against the APIs can be executed in a variety of ways. In this case, we just run the functions single-threaded directly on the Java collections, instead of firing up a memory management, IPC, parallel workers, data movement, etc. That gives programs a minimal execution footprint (like in the Java8 streams API) for small data. The idea is to enable users to use the same program in all sorts of different contexts. The collection execution sits below the common API, so both Java and Scala API can use it. I have implemented the basic framework in https://github.com/StephanEwen/incubator-flink/commits/collections I am looking for people who would be interested in implementing one operator collection based. This is very simple, compared to Flink's distributed runtime. An example how to do this is in the MapOperator ( https://github.com/StephanEwen/incubator-flink/blob/a806609bc85c56b87fea16a985c4df3152c3b955/flink-core/src/main/java/org/apache/flink/api/common/operators/base/MapOperatorBase.java ) Let me know what operators you would be interested in. I'll work on supporting Broadcast variables and Iterations. Just look where the code has compilation errors, that's where the operator implementations are still missing. Greetings, Stephan
