Hi Colleagues,I have implemented the Java version of the MIT's Linear Road
algorithm as a Beam app.I sanity tested it in a Flink Cluster (FlinkRunner).
Works fine.Receives tuples from Kafka, executes the LR algorithm, and produces
the correct results.I would like to repeat the same in a Spark cluster.I am
assuming that, other than changing the type of the Runner (Flink vs Spark) at
runtime, I should not make any code changes.Is that the right assumption based
on what Beam is promising regarding unifying of the underlying streaming
engines?
The real question is: What should I take into consideration if I want to
Benchmark Flink vs Spark by executing my same Beam LR app in both engines?How
would you approach the benchmarking process? What would you be looking for to
compare? etc.Thanks so much for your valuable time.Amir-
