Hi JB, Le samedi 10 mars 2018 à 06:59 +0100, Jean-Baptiste Onofré a écrit : > Good ideas ! > > Validates runner tests and Integration tests should be nightly executed. > > For the Performance tests, it's a great idea, but not sure daily basis > is required. Maybe two times per week ? As these tests could be long, we > should avoid to block executors that could impact our PR build and > master build. Yes, I think once or twice a week is ok. > Maybe we can add Jenkins executors dedicated to PerfTest. >
It would be awesome ! If you have some contacts in the apache infra team, maybe you can get us some executors ? :) > Regards > JB > > On 09/03/2018 12:08, Etienne Chauchot wrote: > > > > > Hi guys, > > > > I was looking at the various jenkins jobs and I wanted to submit a > > proposition: > > > > - Validates runner tests: currently run at PostCommit for all the runners. > > I think it is the quickest way to see > > regressions. So keep it that way > > > > - Integration tests: AFAIK we only run the ones in examples module and only > > on demand. What about running all the IT (in > > particular IO IT) as a cron job on a daily basis with direct runner? Please > > note that it will require some always up > > backend infrastructure. > > > > - Performance tests: what about running Nexmark SMOKE test suite in batch > > and streaming modes with all the runners on a > > daily basis and store the running times in a RRD database (to see > > performance regressions)? Please note that not all the > > queries run in all the runners in all the modes right now. Also, we have > > some streaming pipelines termination issues > > (see https://issues.apache.org/jira/browse/BEAM-2847) > > ) > > > > I know that Stephen Sisk use to work on these topics. I also talked to guys > > from Polidea. But As I understood, they > > launch mainly integration tests on Dataflow runner. > > > > WDYT? > > > > Etienne > > > > > >
