Le vendredi 09 mars 2018 à 20:57 +0000, Kenneth Knowles a écrit : > On Fri, Mar 9, 2018 at 3:08 AM Etienne Chauchot <echauc...@apache.org> wrote: > > Hi guys, > > > > I was looking at the various jenkins jobs and I wanted to submit a > > proposition: > > > > - Validates runner tests: currently run at PostCommit for all the runners. > > I think it is the quickest way to see > > regressions. So keep it that way > We've also toyed with precommit for runners where it is fast. > > > - Integration tests: AFAIK we only run the ones in examples module and only > > on demand. What about running all the IT > > (in > > particular IO IT) as a cron job on a daily basis with direct runner? Please > > note that it will require some always up > > backend infrastructure. > I like this idea. We actually run more, but in postcommit. You can see the > goal here: https://github.com/apache/beam/b > lob/master/.test-infra/jenkins/job_beam_PostCommit_Java_MavenInstall.groovy#L47 > > There's no infrastructure set up that I see. It is only DirectRunner and > DataflowRunner currently, as they are "always > up". But so could be local Flink and Spark. Do the ITs spin up local versions > of what they are connecting to? No, currently IO IT expect the backend middlewares to be set up and running. > > If we have adequate resources, I also think ValidatesRunner on a real > > cluster would add value, once we have the cluster set up / tear down or > > "always up". big +1 > > > > > - Performance tests: what about running Nexmark SMOKE test suite in batch > > and streaming modes with all the runners on a > > > > daily basis and store the running times in a RRD database (to see > > performance regressions)?> > I like this idea, too. I think we could do DirectRunner (and probably local > Flink) as postcommit without being too expensive. You mean run nexmark manually as a postCommit for DirectRunner and Flink in addition to scheduling nexmark runs? Etienne > > Kenn > > > > Please note that not all the > > > > queries run in all the runners in all the modes right now. Also, we have > > some streaming pipelines termination issues > > > > (see https://issues.apache.org/jira/browse/BEAM-2847) > >
> > > > I know that Stephen Sisk use to work on these topics. I also talked to guys > > from Polidea. But As I understood, they > > > > launch mainly integration tests on Dataflow runner. > > > > > > WDYT? > > > > > > Etienne > > > > > > >
