Correct, there is no requirement from ASF about where to run CI or even to run CI. I am all for the best tools and stable (and repeatable) environments. I would be open to seeing how Concourse could work for Geode.
Once the Concourse environment is setup, probably would be best to update the Jenkins job to a pipeline job type as well. There a few questions I have, which can be discussed as we go: - What does 'donate' mean for Pivotal? Is there a, for lack of a better term, contract or is this a 'good faith' effort? I am hesitant to go to any corporately controlled infrastructure, especially since the entity could decide to stop funding. - What types of activities will require admin assistance? - Who will be watching the dev list for requests? --Mark On Fri, Oct 6, 2017 at 10:05 AM, Kenneth Howe <kh...@pivotal.io> wrote: > +1 - Reduce the noise level for analyzing CI results > > > On Oct 6, 2017, at 9:49 AM, Udo Kohlmeyer <ukohlme...@pivotal.io> wrote: > > > > +1 Switch... parallel runs would be safest > > > > On Fri, Oct 6, 2017 at 9:42 AM, Jianxia Chen <jche...@apache.org> wrote: > > > >> +1 to switch to Concourse > >> +1 As a first step we could run both CI systems side-by-side and see how > >> the Concourse approach works for our project. > >> > >> On Fri, Oct 6, 2017 at 7:08 AM, Anthony Baker <aba...@pivotal.io> > wrote: > >> > >>> Hi all, > >>> > >>> I’d like to propose the following that we switch our continuous > >>> integration (CI) system from Jenkins [1] to Concourse [2]. I suggest > >>> this because we continue to experience a significant number of > >>> environmental-related test failures. > >>> > >>> These issues include CPU interference from other Jenkins jobs on the > >>> same host, running out of disk space, port conflicts, and other > >>> gremlins. The net effect is that we are only getting 1-2 successful > >>> builds per month. Certainly not all test failures can be traced back > >>> to environmental issues. However, internal testing on isolated VM’s > >>> shows a combined success rate of about 3X higher compared to ASF > >>> Jenkins for the same tests. This is still definitely NotAwesome, but > >>> removing environmental factors will let us focus on stabilizing flaky > >>> tests. > >>> > >>> Concourse is an Apache-licensed open source CI system based on > >>> pipelines. The pipelines are defined in a YML file containing job > >>> definitions—inputs, outputs, resources, and tasks. A task is simply a > >>> bash script that returns 0/1 for success/failure. A web UI displays > >>> build status. Importantly, each job runs inside an isolated > >>> container. The containers are load-balanced across a pool of workers. > >>> For an example of a build pipeline, see [3] for the pipeline used to > >>> build concourse itself. > >>> > >>> A Concourse environment is deployed and managed in cloud environments > >>> through bosh [4]. Pivotal has agreed to donate AWS and/or GCP compute > >>> and storage resources as well as manage the infrastructure. These > >>> project resources would be available for use by all committers and > >>> community members regardless of corporate affiliations. Note that > >>> AFAIK there is no explicit requirement to host CI on ASF > >>> infrastructure—unlike for critical project resources such as source > >>> code, mailing lists, and issue tracking. > >>> > >>> The source for the pipeline and job scripts would reside within the > >>> geode-* repos. Geode committers would be able to modify those, same > >>> as with our .travis.yml scripts. All test results and build artifacts > >>> would be publicly viewable just like with our Jenkins build output > >>> today. Requests for admin assistance would go through the dev@geode > >>> mailing list. > >>> > >>> Thoughts? As a first step we could run both CI systems side-by-side > >>> and see how the Concourse approach works for our project. > >>> > >>> Thanks, > >>> Anthony > >>> > >>> > >>> [1] https://builds.apache.org/job/Geode-nightly/ > >>> [2] https://concourse.ci > >>> [3] https://ci.concourse.ci > >>> [4] https://bosh.io > >>> > >> > > > > > > > > -- > > Kindest Regards > > ----------------------------- > > *Udo Kohlmeyer* | *Snr Solutions Architect* |*Pivotal* > > *Mobile:* +61 409-279-160 | ukohlme...@pivotal.io > > <http://www.gopivotal.com/> > > www.pivotal.io > >
