Re: Flink performance pre-packaged vs. self-compiled

2016-04-14 Thread Robert Schmidtke
You're obviously right, the configs were different. In the downloaded version I had set off heap memory to true, whereas in the version I compiled myself this one-time change to flink-conf.yaml was overwritten by recompiling. I have fixed it now and performance is the same. For the record, I had 3

Re: Flink performance pre-packaged vs. self-compiled

2016-04-14 Thread Ovidiu-Cristian MARCU
Hi, Your assumption may be incorrect related to the TeraSort use case for eastcirclek's implementation. How many time did you run your program? It would be helpful to give more details about your experiment, in terms of configuration, dataset size. Best, Ovidiu > On 14 Apr 2016, at 17:14, Rob

Re: Flink performance pre-packaged vs. self-compiled

2016-04-14 Thread Robert Schmidtke
I have tried multiple Maven and Scala Versions, but to no avail. I can't seem to achieve performance of the downloaded archive. I am stumped by this and will need to do more experiments when I have more time. Robert On Thu, Apr 14, 2016 at 1:13 PM, Robert Schmidtke wrote: > Hi Robert, > > thank

Re: Flink performance pre-packaged vs. self-compiled

2016-04-14 Thread Robert Schmidtke
Hi Robert, thanks for the hint! Looks like something I could have figured out myself -.-" I'll let you know if I find something. Robert On Thu, Apr 14, 2016 at 1:06 PM, Robert Metzger wrote: > Hi Robert, > > check out the tools/create_release_files.sh file in the source tree. There > you can s

Re: Flink performance pre-packaged vs. self-compiled

2016-04-14 Thread Robert Metzger
Hi Robert, check out the tools/create_release_files.sh file in the source tree. There you can see how we are building the release binaries. It would be quite interesting to find out what caused the performance difference. On Wed, Apr 13, 2016 at 5:03 PM, Robert Schmidtke wrote: > Hi everyone, >

Flink performance pre-packaged vs. self-compiled

2016-04-13 Thread Robert Schmidtke
Hi everyone, I'm using Flink 0.10.2 for some benchmarks and had to add some small changes to Flink, which led me to compiling and running it myself. This is when I noticed a performance difference in the pre-packaged Flink version that I downloaded from the web ( http://archive.apache.org/dist/fli