Re: [OpenStack-Infra] [zuul] third-party CI for zuul-jobs
On November 28, 2017 7:37 pm, James E. Blair wrote: Jens Harbott writes: 2017-11-23 5:28 GMT+00:00 Tristan Cacqueray : ... TL;DR; Is it alright if we re-enable this CI and report those tests on zuul-jobs patchsets? I like the general idea, but please wait for more feedback until doing so. I am in favor of the idea in general, thanks! Also, IMHO it would be better if you could change the "recheck-sf" trigger to something that does not also rerun upstream checks. What seems to work well for other projects is "run ci-name", where ci-name is the name of the Gerrit account. Actually, I'd prefer that we do the opposite. I'd like the recheck command for both to just be "recheck". There's no harm in both systems re-running tests for a change in this case, and it keeps things simpler for developers. The requirements in https://docs.openstack.org/infra/system-config/third_party.html#requirements state that all systems should honor "recheck". I'd like to go beyond that in zuul-jobs and say that third-party ci systems on that repo should *only* honor "recheck". In the meeting today we agreed that we should start by reporting without voting, gain some confidence, then enable +1/-1 voting. Now that zuul-jobs correctly run on CentOS I enabled the patchset-created and recheck comment event filters. Thanks, -Tristan pgpPkDMwjdMyb.pgp Description: PGP signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [zuul] third-party CI for zuul-jobs
On November 28, 2017 7:37 pm, James E. Blair wrote: Jens Harbott writes: 2017-11-23 5:28 GMT+00:00 Tristan Cacqueray : ... TL;DR; Is it alright if we re-enable this CI and report those tests on zuul-jobs patchsets? I like the general idea, but please wait for more feedback until doing so. I am in favor of the idea in general, thanks! Also, IMHO it would be better if you could change the "recheck-sf" trigger to something that does not also rerun upstream checks. What seems to work well for other projects is "run ci-name", where ci-name is the name of the Gerrit account. Actually, I'd prefer that we do the opposite. I'd like the recheck command for both to just be "recheck". There's no harm in both systems re-running tests for a change in this case, and it keeps things simpler for developers. The requirements in https://docs.openstack.org/infra/system-config/third_party.html#requirements state that all systems should honor "recheck". I'd like to go beyond that in zuul-jobs and say that third-party ci systems on that repo should *only* honor "recheck". How about supporting both comments, like that we can still recheck a single CI system? When doing tests, there is no need to consume other CI resources... In the meeting today we agreed that we should start by reporting without voting, gain some confidence, then enable +1/-1 voting. -Jim Thanks for the feedback. I've increased coverage by creating more jobs, though I didn't enable the patchset-created event yet because zuul-jobs isn't usable as-is on our systems, mostly because zuul isn't sudoer. Here is a collection of fix to cover the bare minimal roles (prepare-workspace, emit-ara) and to run the tox job on CentOS: https://review.openstack.org/#/q/topic:fix-centos-7 CI is success on https://review.openstack.org/525510 And a negative test is working on https://review.openstack.org/522438 Regards, -Tristan pgpw_UVmPQyr4.pgp Description: PGP signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [zuul] third-party CI for zuul-jobs
Jens Harbott writes: > 2017-11-23 5:28 GMT+00:00 Tristan Cacqueray : > ... >> TL;DR; Is it alright if we re-enable this CI and report those tests on >> zuul-jobs patchsets? > > I like the general idea, but please wait for more feedback until doing so. I am in favor of the idea in general, thanks! > Also, IMHO it would be better if you could change the "recheck-sf" > trigger to something that does not also rerun upstream checks. What > seems to work well for other projects is "run ci-name", where ci-name > is the name of the Gerrit account. Actually, I'd prefer that we do the opposite. I'd like the recheck command for both to just be "recheck". There's no harm in both systems re-running tests for a change in this case, and it keeps things simpler for developers. The requirements in https://docs.openstack.org/infra/system-config/third_party.html#requirements state that all systems should honor "recheck". I'd like to go beyond that in zuul-jobs and say that third-party ci systems on that repo should *only* honor "recheck". In the meeting today we agreed that we should start by reporting without voting, gain some confidence, then enable +1/-1 voting. -Jim ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [zuul] third-party CI for zuul-jobs
On November 23, 2017 10:21 am, Jens Harbott wrote: 2017-11-23 5:28 GMT+00:00 Tristan Cacqueray : ... TL;DR; Is it alright if we re-enable this CI and report those tests on zuul-jobs patchsets? I like the general idea, but please wait for more feedback until doing so. Thank you for the feedback. I've removed the patchset-created and change-restored trigger event. Also, IMHO it would be better if you could change the "recheck-sf" trigger to something that does not also rerun upstream checks. What seems to work well for other projects is "run ci-name", where ci-name is the name of the Gerrit account. Ok, it's now triggering on "run sf-project-io" It would also be nice to see an entry for your system at https://wiki.openstack.org/wiki/ThirdPartySystems Alright, I've added https://wiki.openstack.org/wiki/ThirdPartySystems/Software_Factory_CI Regards, -Tristan pgpf0f0OsnOep.pgp Description: PGP signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
Re: [OpenStack-Infra] [zuul] third-party CI for zuul-jobs
2017-11-23 5:28 GMT+00:00 Tristan Cacqueray : ... > TL;DR; Is it alright if we re-enable this CI and report those tests on > zuul-jobs patchsets? I like the general idea, but please wait for more feedback until doing so. Also, IMHO it would be better if you could change the "recheck-sf" trigger to something that does not also rerun upstream checks. What seems to work well for other projects is "run ci-name", where ci-name is the name of the Gerrit account. It would also be nice to see an entry for your system at https://wiki.openstack.org/wiki/ThirdPartySystems ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
[OpenStack-Infra] [zuul] third-party CI for zuul-jobs
Greetings, We have discussed it in the past and I'd like to share my experience trying to run third-party tests on zuul-jobs. My goal is to ensure zuul-jobs can run on CentOS system outside of the openstack-infra resources. Please find bellow a step by step documentation of my setup. # Create a new tenant to prevent conflicts with openstack-infra/zuul-jobs ```yaml - tenant: name: openstack.org source: softwarefactory-project.io: config-projects: - third-party-ci-config untrusted-projects: - third-party-ci-jobs openstack.org: untrusted-projects: - openstack-infra/zuul-jobs: exclude: - project - project-template - pipeline ``` # Define base jobs and projects in third-party-ci-config ```yaml - job: name: base parent: null description: The base job not using openstack-infra/zuul-jobs roles pre-run: playbooks/base/pre.yaml post-run: - playbooks/base/post.yaml - job: name: upstream-base parent: null description: The upstream-base job using zuul-jobs roles pre-run: playbooks/upstream-base/pre.yaml post-run: - playbooks/upstream-base/post-logs.yaml roles: - zuul: openstack-infra/zuul-jobs - project: name: openstack-infra/zuul-jobs third-party-check: jobs: - test-basic-roles - test-job-unittests third-party-post: jobs: - test-base ``` # Define jobs in third-party-ci-jobs ```yaml - job: name: test-base description: A job to test base playbook parent: upstream-base run: playbooks/test-base.yaml - job: name: test-basic-roles description: A job to test prepare-workspace and emit-ara role parent: base run: playbooks/test-basic-roles.yaml - job: name: test-job-unittests description: A job to test the unittests job parent: unittests run: playbooks/noop.yaml ``` Here is how the results looks like: https://review.openstack.org/522261 And here is how it fail when there is an error in prepare-workspace: https://review.openstack.org/522438 This CI configuration is defined by these repositories: (which you are welcome to contribute using git review) * https://softwarefactory-project.io/r/config for the tenant * https://softwarefactory-project.io/r/third-party-ci-config for config * https://softwarefactory-project.io/r/third-party-ci-jobs for the jobs Note that the CI is currently disabled because the unittests job is failing without: https://review.openstack.org/522255 and https://review.openstack.org/522261 TL;DR; Is it alright if we re-enable this CI and report those tests on zuul-jobs patchsets? Cheers, -Tristan pgp0Tf5qqM9qT.pgp Description: PGP signature ___ OpenStack-Infra mailing list OpenStack-Infra@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
