Re: [FEEDBACK REQUEST] Re: [ANNOUNCEMENT] Nexmark included to the CI

No problem, always glad to help Etienne Le jeudi 29 novembre 2018 à 09:20 -0800, Alex Amato a écrit : > Thanks Etienne, appreciate the info. This will help me a lot :) > On Wed, Nov 28, 2018 at 1:02 AM Etienne Chauchot wrote: > > Hi Alex, > > Exporting results to the dashboards is as easy as

Re: [FEEDBACK REQUEST] Re: [ANNOUNCEMENT] Nexmark included to the CI

Thanks Etienne, appreciate the info. This will help me a lot :) On Wed, Nov 28, 2018 at 1:02 AM Etienne Chauchot wrote: > Hi Alex, > Exporting results to the dashboards is as easy as writing to a BigQuery > table and then configure the dashboard SQL request to display it. Here is > an example:

Re: [FEEDBACK REQUEST] Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Alex,Exporting results to the dashboards is as easy as writing to a BigQuery table and then configure the dashboard SQL request to display it. Here is an example:- exporting:

Re: [FEEDBACK REQUEST] Re: [ANNOUNCEMENT] Nexmark included to the CI

It would be great to add some lower level benchmark tests for the java SDK. I was thinking of using open census for collecting benchmarks, which looks easy to use should be license compatible. I'm just not sure about how to export the results so that we can display them on the perfkit dashboard

Re: [FEEDBACK REQUEST] Re: [ANNOUNCEMENT] Nexmark included to the CI

The doc changes look good to me, I'll add Dataflow once it is ready. Thanks for opening the issue on the DirectRunner. I'll try to get some progress on a dedicated perf node while you are gone, we can talk about increasing the size of the nexmark input collection for the runs once we know what the

[FEEDBACK REQUEST] Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi guys,As suggested by Anton bellow, I opened a PR on the website to reference the Nexmark dashboards. As I did not want users to take them for proper neutral benchmarks of the runners / engines, but more for a CI piece of software, I added a disclaimer. Please:- tell if you agree on the

Re: [ANNOUNCEMENT] Nexmark included to the CI

Andrew,here is the ticket about query3:https://issues.apache.org/jira/browse/BEAM-4825 I added details with pseudo code of query3 in the ticket and a link to the dashboards to see how flaky this query is on DR. Etienne Le jeudi 19 juillet 2018 à 12:22 +0200, Etienne Chauchot a écrit : > Hi

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Anton, Yes, good idea, I'll update nexmark website page Etienne Le mercredi 18 juillet 2018 à 10:17 -0700, Anton Kedin a écrit : > These dashboards look great! > > Can publish the links to the dashboards somewhere, for better visibility? > E.g. in the jenkins website / emails, or the > wiki.

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Andrew, Le mercredi 18 juillet 2018 à 10:08 -0700, Andrew Pilloud a écrit : > Hi Etienne, > I've been asking around and it sounds like we should be able to get a > dedicated Jenkins node for performance tests. Cool ! > Another thing that might help is making the runs a few times longer. They

Re: [ANNOUNCEMENT] Nexmark included to the CI

These dashboards look great! Can publish the links to the dashboards somewhere, for better visibility? E.g. in the jenkins website / emails, or the wiki. Regards, Anton On Wed, Jul 18, 2018 at 10:08 AM Andrew Pilloud wrote: > Hi Etienne, > > I've been asking around and it sounds like we

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Etienne, I've been asking around and it sounds like we should be able to get a dedicated Jenkins node for performance tests. Another thing that might help is making the runs a few times longer. They are currently running around 2 seconds each, so the total time of the build probably exceeds

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Andrew, Yes I saw that, except dedicating jenkins nodes to nexmark, I see no other way. Also, did you see query 3 output size on direct runner? Should be a straight line and it is not, I'm wondering if there is a problem with sate and timers impl in direct runner. Etienne Le mardi 17 juillet

Re: [ANNOUNCEMENT] Nexmark included to the CI

@Etienne: Nice to see the graphs! :) @Ismael: Good idea, there's no document yet. I think we could create a small google doc with instructions on how to do this. pt., 13 lip 2018 o 10:46 Etienne Chauchot napisał(a): > Hi, > > @Andrew, this is because I did not find a way to set 2 scales on the

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi, @Andrew, this is because I did not find a way to set 2 scales on the Y axis on the perfkit graphs. Indeed numResults varies from 1 to 100 000 and runtimeSec is usually bellow 10s. Etienne Le jeudi 12 juillet 2018 à 12:04 -0700, Andrew Pilloud a écrit : > This is great, should make

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Pablo, yes they run at each commit on master (post-commit jenkins script) Etienne Le jeudi 12 juillet 2018 à 11:40 -0700, Pablo Estrada a écrit : > This is really cool Etienne : ) thanks for working on this.Our of curiosity, > do you know how often the tests run on > each runner? > > Best >

Re: [ANNOUNCEMENT] Nexmark included to the CI

Thank you Etienne! This looks great. I hope we can get other languages to have benchmarks at this level soon enough. Ahmet On Thu, Jul 12, 2018 at 1:45 PM, Ismaël Mejía wrote: > That’s great to see in action, great work Etienne! > > Is there any document on how to integrate ‘stuff’ into the

Re: [ANNOUNCEMENT] Nexmark included to the CI

That’s great to see in action, great work Etienne! Is there any document on how to integrate ‘stuff’ into the dashboards? I think this is worth having for people willing to do so like Kai or Andrew. Are there any docs on this? or maybe Lukasz Gajowy know ? On Thu, Jul 12, 2018 at 9:04 PM Andrew

Re: [ANNOUNCEMENT] Nexmark included to the CI

This is great, should make performance work much easier! I'm going to get the Beam SQL Nexmark jobs publishing as well. (Opened https://issues.apache.org/jira/browse/BEAM-4774 to track.) I might take on the Dataflow runner as well if no one else volunteers. I am curious as to why you have two

Re: [ANNOUNCEMENT] Nexmark included to the CI

This is really cool Etienne : ) thanks for working on this. Our of curiosity, do you know how often the tests run on each runner? Best -P. On Thu, Jul 12, 2018 at 2:15 AM Romain Manni-Bucau wrote: > Awesome Etienne, this is really important for the (user) community to have > that visibility

Re: [ANNOUNCEMENT] Nexmark included to the CI

Awesome Etienne, this is really important for the (user) community to have that visibility since it is one of the most important aspect of the Beam's quality, kudo! Romain Manni-Bucau @rmannibucau | Blog | Old Blog

Re: [ANNOUNCEMENT] Nexmark included to the CI

It's really great to have these dashboards and integration in Jenkins ! Thanks Etienne for driving this ! Regards JB On 11/07/2018 15:13, Etienne Chauchot wrote: > > Hi guys, > > I'm glad to announce that the CI of Beam has much improved ! Indeed > Nexmark is now included in the perfkit

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Kai, Cool for TPC-H, it will be complementary to Nexmark.Regarding Dataflow we can run Nexmark on dataflow, note that it was the original target of Nexmark port by Mark. We have not done it because we have no DF environment available. Is someone from google willing to run nexmark on dataflow

Re: [ANNOUNCEMENT] Nexmark included to the CI

Hi Etienne, It's awesome for working on these useful dashboards. I am getting TPC-H benchmark running on Flink and Dataflow Runner. I could work on similar dashboards for TPC benchmark after code merged. Also, it's great to have a dashboards for Dataflow. Best, Kai ᐧ On Wed, Jul 11, 2018 at

Re: [ANNOUNCEMENT] Nexmark included to the CI

First catch of the nexmark-CI:It seems that there was a change in the direct runner. Query3 (exercise state and timers) - output size should be constant but has increased today => Was there a change in state and timer related code?- the output size of this query is different between batch and

Re: [ANNOUNCEMENT] Nexmark included to the CI

Is someone interested in creating the scripts and dashboards for the other runners? They can be created by copying the existing scripts and dashboards and changing one gradle parameter in the scripts and the table name in the dashboards. I have created the