In fact, the use-cases of Spark and Flink overlap a bit. However, the technology used under the hood is quite different. Flink shares a lot of similarities with relational DBMS. Data is serialized in byte buffers and processed a lot in binary representation. This also allows for fine-grained memory control. Flink uses a pipelined processing model and it has a cost-based optimizer that selects execution strategies and avoids expensive partitioning and sorting steps. Moreover, Flink features a special kind of iterations (delta-iterations) that can significantly reduce the amount of computations as iterations go on (the vertex-centric computing model of Pregel / Giraph is a special kind of that).
Btw. The academic projects from which Spark and Flink originated started about the same time ;-) Best, Fabian 2014-10-19 20:33 GMT+02:00 Mohit Singh <[email protected]>: > Hi, > Good to see another interesting apache incubator project but I am just > curious on what is flink trying to solve which spark is not currently > addressing? I am sure you get this question a lot.. :) > Thanks > > -- > Mohit > > "When you want success as badly as you want the air, then you will get it. > There is no other secret of success." > -Socrates >
