Re: Executing a controllable benchmark in Flink

2020-05-29 Thread Felipe Gutierrez
I was losing something when because I was reading the line of the GZIPInputStream outside of the busy while loop. I changed it and now I am having more throughput. It is also a good idea to use VisualVM to check if the throughput is correct and where I am losing more cycles. while

Re: Executing a controllable benchmark in Flink

2020-05-29 Thread Robert Metzger
Hi Felipe, the file is just 80 MBs. It is probably cached in the linux page cache, there should not be any disk IO involved. So you are saying is that you can not further increase the throughput for sleeps shorter than 2000 nanoseconds. Have you tried running this w/o any Sleep / nano.time

Executing a controllable benchmark in Flink

2020-05-27 Thread Felipe Gutierrez
Hi, I am trying to benchmark a stream application in Flink. So, I am using the source Function that reads events from the NYC Taxi Rides (http://training.ververica.com/trainingData/nycTaxiRides.gz) and I control the emission with System.nanoTime(). I am not using Thread.sleep because Java does