Re: managing transitions
On Oct 06, 2015, at 03:12 PM, Fred Drake wrote: >What CI tools are you using with GitLab CE? We don't run CE; we use the hosted EE at gitlab.com. But anyway, we have a custom VM on which we run the GitLab runner software in a docker image. This runs our test suite in all supported Python 3s against both SQLite and PostgreSQL. At the time we set things up they didn't have shared runners, but now they do so I don't think you even need to provision a VM. http://doc.gitlab.com/ci/ Cheers, -Barry pgp1V5NfV0r9R.pgp Description: OpenPGP digital signature
Re: managing transitions
On Tue, Oct 6, 2015 at 1:11 PM, Barry Warsaw wrote: > On Oct 06, 2015, at 07:05 PM, Thomas Goirand wrote: > >>Interesting. It's the first time I hear about it, I thought it was just >>closed source. > > The instance at gitlab.com is the non-free Enterprise Edition (EE). EE has > features we probably don't care about ayway. The Community Edition (CE) is > MIT/Expat. > >>Does it have test capabilities as well, so that we could >>run tests on each new patch, like Gerrit does? Can we plug zuul and >>nodepool to it? > > I don't know about plugging in zuul and nodepool, but the CE does have CI > integration, which we use in the GNU Mailman project, and seems to work > great. It definitely can integrate with its own CI (which I believe is also licensed similarly to GitLab CE) as well as Jenkins. Currently the Flake8 project is hosted and developed on GitLab and uses integration with vanilla Jenkins.
Re: managing transitions
On Tue, Oct 6, 2015 at 2:11 PM, Barry Warsaw wrote: > I don't know about plugging in zuul and nodepool, but the CE does have CI > integration, which we use in the GNU Mailman project, and seems to work > great. What CI tools are you using with GitLab CE? -Fred -- Fred L. Drake, Jr. "A storm broke loose in my mind." --Albert Einstein
Re: managing transitions
On Oct 06, 2015, at 07:05 PM, Thomas Goirand wrote: >Interesting. It's the first time I hear about it, I thought it was just >closed source. The instance at gitlab.com is the non-free Enterprise Edition (EE). EE has features we probably don't care about ayway. The Community Edition (CE) is MIT/Expat. >Does it have test capabilities as well, so that we could >run tests on each new patch, like Gerrit does? Can we plug zuul and >nodepool to it? I don't know about plugging in zuul and nodepool, but the CE does have CI integration, which we use in the GNU Mailman project, and seems to work great. Cheers, -Barry
Re: managing transitions
On 10/06/2015 05:42 PM, Barry Warsaw wrote: > On Oct 06, 2015, at 08:39 AM, Brian May wrote: > >> On Tue, 6 Oct 2015 at 18:46 Thomas Goirand wrote: >> >>> This IMO is the same topic as having a Gerrit review system (and not >>> just Git) which could do tests on each change of a package even before >>> having them committed to our git. >>> >> >> Sounds like an interesting thing to discuss/test after we move to git... > > I don't intend to bikeshed this, nor do I have time to do the work, so in our > do-it-ocracy any online review system would be a fantastic addition. I'll > just point to GitLab community edition as a nice open source option in this > space. Interesting. It's the first time I hear about it, I thought it was just closed source. Does it have test capabilities as well, so that we could run tests on each new patch, like Gerrit does? Can we plug zuul and nodepool to it? FYI, I have Zuul and Nodepool nearly done. I am just waiting for the support of statsd 3.x to land upstream (as it'd be broken in Sid otherwise). Cheers, Thomas Goirand (zigo)
Re: managing transitions (was: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental)
On Oct 06, 2015, at 08:39 AM, Brian May wrote: >On Tue, 6 Oct 2015 at 18:46 Thomas Goirand wrote: > >> This IMO is the same topic as having a Gerrit review system (and not >> just Git) which could do tests on each change of a package even before >> having them committed to our git. >> > >Sounds like an interesting thing to discuss/test after we move to git... I don't intend to bikeshed this, nor do I have time to do the work, so in our do-it-ocracy any online review system would be a fantastic addition. I'll just point to GitLab community edition as a nice open source option in this space. Cheers, -Barry
Re: managing transitions
On Tue, Oct 06, 2015 at 04:03:54PM +0200, Thomas Goirand wrote: > On 10/06/2015 01:02 PM, Mattia Rizzolo wrote: > > On Tue, Oct 06, 2015 at 08:39:48AM +, Brian May wrote: > >> On Tue, 6 Oct 2015 at 18:46 Thomas Goirand wrote: > >> > >>> This IMO is the same topic as having a Gerrit review system (and not > >>> just Git) which could do tests on each change of a package even before > >>> having them committed to our git. > >>> > >> > >> Sounds like an interesting thing to discuss/test after we move to git... > > > > Moreover, jenkins.debian.org is happening right about now, guess it > > would help quite something (and I'd be happy to host such tests there). > > I don't think a single machine is enough for this kind of workload. > Imagine the upload of a new setuptools and rebuilding all packages > build-depending on it. That's a lot of packages to rebuild in > jenkins.d.o. I have offers from some cloud providers which we could use > instead for this kind of job. We "only" need to do the work... jenkins.d.o won't build anything but only keep the master node (without builders, or maybe one or two to do maintenance stuff). If what you would like to do is to "just" rebuild all rdeps, that's really one of the thing I'd really like to do, it just takes time do all the work (and atm I'm busy doing other stuff, if that wasn't clear) :) To be precise, what I plan is to really improve the current reproducible.d.n scripts and base everything on them. I'd like to support both rebuilds on a modified toolchain on demand and on all r-deps of a particular packages which uploads would act as a trigger. It just takes a shitload of time I don't currently have, as usual. Currently there are means to ask for rebuild of stuff using a particular packages (through the AWS credit, managed by lucas & I-don't-remember-who), but to me it looks like it's not really used if not for big packages (like, before gcc uploads), and the results are somewhat difficult to access and understand and often used by a single person, which sucks. -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: http://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Re: managing transitions
On 10/06/2015 01:02 PM, Mattia Rizzolo wrote: > On Tue, Oct 06, 2015 at 08:39:48AM +, Brian May wrote: >> On Tue, 6 Oct 2015 at 18:46 Thomas Goirand wrote: >> >>> This IMO is the same topic as having a Gerrit review system (and not >>> just Git) which could do tests on each change of a package even before >>> having them committed to our git. >>> >> >> Sounds like an interesting thing to discuss/test after we move to git... > > Moreover, jenkins.debian.org is happening right about now, guess it > would help quite something (and I'd be happy to host such tests there). I don't think a single machine is enough for this kind of workload. Imagine the upload of a new setuptools and rebuilding all packages build-depending on it. That's a lot of packages to rebuild in jenkins.d.o. I have offers from some cloud providers which we could use instead for this kind of job. We "only" need to do the work... Cheers, Thomas
Re: managing transitions (was: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental)
On Tue, Oct 06, 2015 at 08:39:48AM +, Brian May wrote: > On Tue, 6 Oct 2015 at 18:46 Thomas Goirand wrote: > > > This IMO is the same topic as having a Gerrit review system (and not > > just Git) which could do tests on each change of a package even before > > having them committed to our git. > > > > Sounds like an interesting thing to discuss/test after we move to git... Moreover, jenkins.debian.org is happening right about now, guess it would help quite something (and I'd be happy to host such tests there). -- regards, Mattia Rizzolo GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`. more about me: http://mapreri.org : :' : Launchpad user: https://launchpad.net/~mapreri `. `'` Debian QA page: https://qa.debian.org/developer.php?login=mattia `- signature.asc Description: PGP signature
Re: managing transitions (was: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental)
On Tue, 6 Oct 2015 at 18:46 Thomas Goirand wrote: > This IMO is the same topic as having a Gerrit review system (and not > just Git) which could do tests on each change of a package even before > having them committed to our git. > Sounds like an interesting thing to discuss/test after we move to git...
Re: managing transitions (was: python-networkx_1.10-1_amd64.changes ACCEPTED into experimental)
On 10/05/2015 11:11 PM, Barry Warsaw wrote: > On Oct 05, 2015, at 02:51 PM, Thomas Goirand wrote: > >> In other distributions (Red Hat and Ubuntu), everyone is aware of this >> kind of issue before uploading, and this kinds of things don't happen. > > Ubuntu at least does have a technical solution that helps ameliorate > archive-wide breakages, and that is -proposed migration. When you upload > e.g. to wily, it gets diverted to wily-proposed and to get promoted it has to > pass a number of tests. The package and their reverses have to build. DEP-8 > tests have to pass, etc. You can get a nice report about which -proposed > promotions are failing: > > http://people.canonical.com/~ubuntu-archive/proposed-migration/update_excuses.html Oh, nice! We do need something like this too. > The downside is that you should probably be proactively checking this list > (poll vs ping) and it can sometimes be difficult to figure out why a promotion > fails or how to fix it. It's a super nice tool, though in some cases, I do see that we may want to ignore it. For example, dozens of packages passing, and just a single leaf one with some issues. > But this does mean that the archive itself is very rarely broken, and it can > be a convenient way to stage package updates that may have effects in parts of > the archive you might not be aware of. If we need the compute power to do it, I have a few proposal for that. I'm all for having a CI / CD also for packages. This IMO is the same topic as having a Gerrit review system (and not just Git) which could do tests on each change of a package even before having them committed to our git. Thomas