Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
lots  exceptions.  So I am going incremental with that value. Have it set at 8192 (twice as much as before 4096).Thanks From: Greg Hogan <c...@greghogan.com> To: dev@flink.apache.org; amir bahmanyari <amirto...@yahoo.com> Sent: Monday, September 19, 2016 1:28 PM Subject: Re: Perf

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
; From: Greg Hogan <c...@greghogan.com> > To: dev@flink.apache.org; amir bahmanyari <amirto...@yahoo.com> > Sent: Monday, September 19, 2016 12:43 PM > Subject: Re: Performance and Latency Chart for Flink > > You will need to add the configuration parameters to

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
gt; Apache Flink 1.2-SNAPSHOT Documentation: Configuration >>> | | >>> >>>| >>> >>>| >>> >>> >>> >>> 4096 = (16x16)x4x4 where 16 is number of tasks per TM, 4 is # of TMs & 4 >>> is there in

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
he formula.What would you set it to? Once I have that number, > I will set "taskmanager.memory.preallocate" to true & will give it > another shot.Thanks Greg > > From: Greg Hogan <c...@greghogan.com> > To: dev@flink.apache.org; amir bahmanyari <amirto..

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Chesnay Schepler
e & will give it another shot.Thanks Greg From: Greg Hogan <c...@greghogan.com> To: dev@flink.apache.org; amir bahmanyari <amirto...@yahoo.com> Sent: Monday, September 19, 2016 8:29 AM Subject: Re: Performance and Latency Chart for Flink Hi Amir, You may see improved p

Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
com> To: "dev@flink.apache.org" <dev@flink.apache.org> Sent: Monday, September 19, 2016 10:34 AM Subject: Re: Performance and Latency Chart for Flink Hi Greg,I used this guideline to calculate  "taskmanager.network.numberOfBuffers&q

Re: Performance and Latency Chart for Flink

2016-09-19 Thread amir bahmanyari
t; To: dev@flink.apache.org; amir bahmanyari <amirto...@yahoo.com> Sent: Monday, September 19, 2016 8:29 AM Subject: Re: Performance and Latency Chart for Flink Hi Amir, You may see improved performance setting "taskmanager.memory.preallocate: true" in order to use off-heap mem

Re: Performance and Latency Chart for Flink

2016-09-19 Thread Greg Hogan
t anyways.If you have > suggestions to make improvements in the above case, I am all ears & greatly > appreciate it.Cheers,Amir- > > From: "Chawla,Sumit" <sumitkcha...@gmail.com> > To: dev@flink.apache.org; amir bahmanyari <amirto...@yahoo.com> >

Re: Performance and Latency Chart for Flink

2016-09-18 Thread amir bahmanyari
o.com> Sent: Sunday, September 18, 2016 2:07 PM Subject: Re: Performance and Latency Chart for Flink Has anyone else run these kind of benchmarks?  Would love to hear more people'e experience and details about those benchmarks. Regards Sumit Chawla On Sun, Sep 18, 2016 at 2:01 PM, Chawl

Re: Performance and Latency Chart for Flink

2016-09-18 Thread Chawla,Sumit
og/blog-implementing-linear-ro >> ad-benchmark-in-apex/ >> >> >> From: Fabian Hueske <fhue...@gmail.com> >> To: "dev@flink.apache.org" <dev@flink.apache.org> >> Sent: Friday, September 16, 2016 12:31 AM >> Subject: Re: Performance

Re: Performance and Latency Chart for Flink

2016-09-18 Thread Chawla,Sumit
ng-linear- > road-benchmark-in-apex/ > > > From: Fabian Hueske <fhue...@gmail.com> > To: "dev@flink.apache.org" <dev@flink.apache.org> > Sent: Friday, September 16, 2016 12:31 AM > Subject: Re: Performance and Latency Chart for Flink > > Hi, >

Re: Performance and Latency Chart for Flink

2016-09-16 Thread amir bahmanyari
://www.datatorrent.com/blog/blog-implementing-linear-road-benchmark-in-apex/ From: Fabian Hueske <fhue...@gmail.com> To: "dev@flink.apache.org" <dev@flink.apache.org> Sent: Friday, September 16, 2016 12:31 AM Subject: Re: Performance and Latency Chart for Flink Hi, I am

Re: Performance and Latency Chart for Flink

2016-09-16 Thread Timo Walther
ev@flink.apache.org> Sent: Friday, September 16, 2016 12:31 AM Subject: Re: Performance and Latency Chart for Flink Hi, I am not aware of periodic performance runs for the Flink releases. I know a few benchmarks which have been published at different points in time like [1], [2], and

Re: Performance and Latency Chart for Flink

2016-09-16 Thread Fabian Hueske
Hi, I am not aware of periodic performance runs for the Flink releases. I know a few benchmarks which have been published at different points in time like [1], [2], and [3] (you'll probably find more). In general, fair benchmarks that compare different systems (if there is such thing) are very