Hi Sergio - as I mentioned in my email about integration testing sources[1], we'll also need a cluster hosting mechanism for that and I'm checking out some options.
In addition to mesos+marathon, I'm checking out kubernetes and docker swarm. I started out excited about mesos, but so far mesos has been more complicated to set up on GCE (24 detailed steps you have to get right) than kubernetes (1 step), but I'm curious to see if mesos+marathon is easier to use with flink/spark etc.. [1] https://lists.apache.org/thread.html/518d78478ae9b6a56d6a690033071aa6e3b817546499c4f0f18d247d@%3Cdev.beam.apache.org%3E S On Sun, Oct 23, 2016 at 11:35 PM Jean-Baptiste Onofré <[email protected]> wrote: > Yes. I also have a mesos/marathon to bootstrap a cluster a run test > pipelines on it. We are discussing with Jason to use this with our Jenkins. > > Regards > JB > > > > On Oct 24, 2016, 08:33, at 08:33, "Sergio Fernández" <[email protected]> > wrote: > >On Mon, Oct 24, 2016 at 8:21 AM, Jean-Baptiste Onofré <[email protected]> > >wrote: > > > >> Yes. I also blogged about this. > >> > > > >This article > > > http://blog.nanthrax.net/2016/08/apache-beam-in-action-same-code-several-execution-engines/ > >right? > > > > > >> Basically I used one Maven profile per runner. Same code, different > >> runners. > >> > > > >Exactly that was the idea :-) > > > >Have you tried more complex setups? For instance, deploy to clusters > >with > >the different runners. > > > >Very interesting stuff, JB! > > > >Cheers, > > > > > > > >On Oct 24, 2016, 08:12, at 08:12, "Sergio Fernández" > ><[email protected]> > >> wrote: > >> >Related, but a bit off-topic: has anybody setup a Bean Pipeline CI > >> >against > >> >the different runners? > >> > > >> >I normally do this against a custom Docker-based setup, but just > >> >against > >> >Spark. Probably it'd be very interesting to run a pipeline against > >> >different runners to get broader conclusions. > >> > > >> >If anybody has worked on a similar idea, I'd appreciate to discuss > >it. > >> > > >> >Thanks in advance. > >> > > >> >On Mon, Oct 24, 2016 at 2:37 AM, Frances Perry > ><[email protected]> > >> >wrote: > >> > > >> >> Jason recently started documenting the project's testing setup. > >See > >> >> http://beam.incubator.apache.org/contribute/testing/ > >> >> > >> >> On Sat, Oct 22, 2016 at 11:18 PM, Jean-Baptiste Onofré > >> ><[email protected]> > >> >> wrote: > >> >> > >> >> > Hi > >> >> > > >> >> > We use both Jenkins and Travis. > >> >> > > >> >> > Jenkins is on build.apache.org and you can find Travis yml in > >> >source. > >> >> > > >> >> > Regards > >> >> > JB > >> >> > > >> >> > > >> >> > > >> >> > On Oct 22, 2016, 21:56, at 21:56, Neelesh Salian > >> ><[email protected]> > >> >> > wrote: > >> >> > >Hello, > >> >> > > > >> >> > >I wanted to check to see if there is continuous integration > >> >service > >> >> > >used to > >> >> > >build and test Beam similar to Travis. > >> >> > >Would certainly help speed up adding code to the project. > >> >> > > > >> >> > >Thank you in advance. > >> >> > > > >> >> > >-- > >> >> > >Neelesh Srinivas Salian > >> >> > >Engineer > >> >> > > >> >> > >> > > >> > > >> > > >> >-- > >> >Sergio Fernández > >> >Partner Technology Manager > >> >Redlink GmbH > >> >m: +43 6602747925 <+43%20660%202747925> > >> >e: [email protected] > >> >w: http://redlink.co > >> > > > > > > > >-- > >Sergio Fernández > >Partner Technology Manager > >Redlink GmbH > >m: +43 6602747925 <+43%20660%202747925> > >e: [email protected] > >w: http://redlink.co >
