Hi, I don't have a perfect list available but these are some of the things to keep in mind: 1) end2end Latency. Some systems (like spark) use microbatching which introduces a latency of seconds 2) Do you get "exactly once guarantees"? Storm can give you that but then the throughput goes really down. 3) Ease of programming. How 'nice' is the api you have to work with. 4) Resiliance of state. If you need some state over several events, Does the framework support this and has built in recovery of this stat in case of a faillure? 5) Tools. What kind of tools are " ready to run" available? I.e. kmeans, and things like that. 6) Deployment. How do you run it? Do you need a separate infrastructure or can you deploy it in an existing yarn/mesos/... 7) Security: Can it access kerberos secured resources (like hbase, hdfs or any other service) in a long running situation.
As a final note: I've been hacking at Storm for over a year now and last summer I found Flink. Today Storm is for me no longer an option and we are taking down what we already had running. Niels Basjes On 23 Jan 2016 20:59, "Vinaya M S" <vinay...@gmail.com> wrote: > Hi Flink user group, > > I am working on a project for the Insight Data Engineering Program in New > York to compare streaming tools. The program is designed for software > engineers and those straight from the university to transition to a data > engineering role. After completing the project, we present demos of the > project to several companies in NYC that we are interested in working for > (including top companies like NY Times, Capital One, Bloomberg, etc). > > I have decided to work on a project to compare streaming tools, namely > Flink and Storm. I already have Twitter data stored and would like to > design tests to benchmark the the two tools if possible. > > I wanted to be extra-careful in constructing a benchmark to work on and > present at companies here in NY. Do you have any recommendations to tests > to run with the Twitter data that I have that would showcase when to and > not use Flink compared to Storm? > > Thanks! > Vinaya >
