Le vendredi 09 mars 2018 à 20:57 +0000, Kenneth Knowles a écrit :
> On Fri, Mar 9, 2018 at 3:08 AM Etienne Chauchot <echauc...@apache.org> wrote:
> > Hi guys,
> > 
> > I was looking at the various jenkins jobs and I wanted to submit a 
> > proposition:
> > 
> > - Validates runner tests: currently run at PostCommit for all the runners. 
> > I think it is the quickest way to see
> > regressions. So keep it that way
> We've also toyed with precommit for runners where it is fast.
>  
> > - Integration tests: AFAIK we only run the ones in examples module and only 
> > on demand. What about running all the IT
> > (in
> > particular IO IT) as a cron job on a daily basis with direct runner? Please 
> > note that it will require some always up
> > backend infrastructure.
> I like this idea. We actually run more, but in postcommit. You can see the 
> goal here: https://github.com/apache/beam/b
> lob/master/.test-infra/jenkins/job_beam_PostCommit_Java_MavenInstall.groovy#L47
> 
> There's no infrastructure set up that I see. It is only DirectRunner and 
> DataflowRunner currently, as they are "always
> up". But so could be local Flink and Spark. Do the ITs spin up local versions 
> of what they are connecting to?
No, currently IO IT expect the backend middlewares to be set up and running.
> > If we have adequate resources, I also think ValidatesRunner on a real 
> > cluster would add value, once we have the cluster set up / tear down or 
> > "always up".
big +1
>  > > 
> > - Performance tests: what about running Nexmark SMOKE test suite in batch 
> > and streaming modes with all the runners on a
> > 
> > daily basis and store the running times in a RRD database (to see 
> > performance regressions)?> 
> I like this idea, too. I think we could do DirectRunner (and probably local 
> Flink) as postcommit without being too expensive.
You mean run nexmark manually as a postCommit for DirectRunner and Flink in 
addition to scheduling nexmark runs?
Etienne
> > Kenn
> >  
> >  Please note that not all the
> > 
> > queries run in all the runners in all the modes right now. Also, we have 
> > some streaming pipelines termination issues
> > 
> > (see https://issues.apache.org/jira/browse/BEAM-2847)
> > 

> > 
> > I know that Stephen Sisk use to work on these topics. I also talked to guys 
> > from Polidea. But As I understood, they
> > 
> > launch mainly integration tests on Dataflow runner.
> > 

> > 
> > WDYT?
> > 

> > 
> > Etienne
> > 

> > 

> > 

> 

Reply via email to