Hi Eric, Max is right, the tool has been used for a different benchmark [1]. The throughput logger that should produce the right output is this one [2]. Very recently, I've opened a pull request for adding metric-measuring support into the engine [3]. Maybe that's helpful for your experiments.
[1] http://data-artisans.com/high-throughput-low-latency-and-exactly-once-stream-processing-with-apache-flink/ [2] https://github.com/dataArtisans/performance/blob/master/flink-jobs/src/main/java/com/github/projectflink/streaming/Throughput.java#L203 [3] https://github.com/apache/flink/pull/2386 On Wed, Aug 24, 2016 at 2:04 PM, Maximilian Michels <m...@apache.org> wrote: > I believe the AnaylzeTool is for processing logs of a different benchmark. > > CC Jamie and Robert who worked on the benchmark. > > On Wed, Aug 24, 2016 at 3:25 AM, Eric Fukuda <e.s.fuk...@gmail.com> wrote: > > Hi, > > > > I'm trying to benchmark Flink without Kafka as mentioned in this post > > (http://data-artisans.com/extending-the-yahoo-streaming-benchmark/). > After > > running flink.benchmark.state.AdvertisingTopologyFlinkState with > > user.local.event.generator in localConf.yaml set to 1, I ran > > flink.benchmark.utils.AnalyzeTool giving > > flink-1.0.1/log/flink-[username]-jobmanager-0-[servername].log as a > > command-line argument. I got the following output and it does not have > the > > information about the latency. > > > > > > ================= Latency (0 reports ) ===================== > > ================= Throughput (1 reports ) ===================== > > ====== null (entries: 10150)======= > > Mean throughput 639078.5018497099 > > Exception in thread "main" java.lang.IndexOutOfBoundsException: toIndex > = 2 > > at java.util.ArrayList.subListRangeCheck(ArrayList.java:962) > > at java.util.ArrayList.subList(ArrayList.java:954) > > at flink.benchmark.utils.AnalyzeTool.main(AnalyzeTool.java:133) > > > > > > Reading the code in AnalyzeTool.java, I found that it's looking for lines > > that include "Latency" in the log file, but apparently it's not finding > any. > > I tried grepping the log file, and couldn't find any either. I have one > > server that runs both JobManager and Task Manager and another server that > > runs Redis, and they are connected through a network with each other. > > > > I think I have to do something to read the data stored in Redis before > > running AnalyzeTool, but can't figure out what. Does anyone know how to > get > > the latency information? > > > > Thanks, > > Eric >
