Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 08/14/2014 12:35 AM, Russell Bryant wrote: On 08/13/2014 06:23 PM, Mark McLoughlin wrote: On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Beautiful! Thank you so much to everyone involved. +1! Love this. Yes, finally. One of the things I really wanted. Thanks guys! -- @flaper87 Flavio Percoco ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Wed, 13 Aug 2014 18:52:05 -0400 Jay Pipes jaypi...@gmail.com wrote: On 08/13/2014 06:35 PM, Russell Bryant wrote: On 08/13/2014 06:23 PM, Mark McLoughlin wrote: On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Beautiful! Thank you so much to everyone involved. +1! Love this. Indeed. Amazeballs. Agreed! This is a really nice improvement Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Wed, Aug 13, 2014 at 12:05:27PM -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Thanks again to Radoslav Gerganov for writing the original change. These are both a great step forward in usability of the Gerrit web UI. Thanks for the effort everyone put into these changes. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) The table only includes CI systems that leave their results in the same format that we use for Jenkins; we will update the recommendations for third party CI systems to encourage the use of that format. This is all still fairly brittle, based mostly on javascript-powered screen scraping. However, I'm hoping we can get something like it in a Gerrit plugin for a more long-term solution to the problem. Thanks again to Radoslav Gerganov for writing the original change. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Friggin. Awesome. Thanks again to Radoslav Gerganov for writing the original change. Thanks to all involved, as this is a major improvement for everyone! One thing that we discussed at the nova meetup was having a space for each CI we *expect* to vote. I haven't looked at the implementation here, but I assume it just parses the comments to generate the table. How hard would it be to make the table show all the CI systems we expect so that it's very obvious that one has gone MIA (as they often do) before we merge a patch? I think we struggle right now with merging things that a CI system would have NAKed, but only because we didn't notice that it hadn't voted. --Dan signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Dan Smith d...@danplanet.com writes: You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Friggin. Awesome. Thanks again to Radoslav Gerganov for writing the original change. Thanks to all involved, as this is a major improvement for everyone! One thing that we discussed at the nova meetup was having a space for each CI we *expect* to vote. I haven't looked at the implementation here, but I assume it just parses the comments to generate the table. How hard would it be to make the table show all the CI systems we expect so that it's very obvious that one has gone MIA (as they often do) before we merge a patch? I think we struggle right now with merging things that a CI system would have NAKed, but only because we didn't notice that it hadn't voted. I think my preferred method of doing this would be to drive all third-party CI systems from OpenStack's Zuul. We are not far from being able to do that technically, though it's not clear to me that there is the will for third party CI systems to do that. However, if we really are expecting them to report back and are willing to hold up merging a change because of it, then we really should consider that. As we look into using a Gerrit plugin for test results, we could see about adding that as a feature. Something that we could technically do immediately would be to add third-party CI systems as reviewers to changes when they are uploaded. Then they would show up in the list of reviewers at the top of the change. This would require maintaining a list of which systems were expected to vote on which repositories. Rather than keeping that in git, we could use group membership for it (and implement something we have talked about off-and-on, which is using Gerrit group membership to grant voting privileges to each individual repository). That would also allow projects to self-manage both who is permitted to vote on their repos and who is expected to vote. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Wed, Aug 13, 2014 at 3:05 PM, James E. Blair cor...@inaugust.com wrote: You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Very cool!! this is really nice UI, super useful one litle suggestions for the folk that knows how to do that if that's possible to do is to sort between the voting and the non-voting so we can easily spot which one are worthwhile to look at or not. Chmouel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Beautiful! Thank you so much to everyone involved. Mark. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Chmouel Boudjnah chmo...@enovance.com writes: On Wed, Aug 13, 2014 at 3:05 PM, James E. Blair cor...@inaugust.com wrote: You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Very cool!! this is really nice UI, super useful one litle suggestions for the folk that knows how to do that if that's possible to do is to sort between the voting and the non-voting so we can easily spot which one are worthwhile to look at or not. If it is not worth looking at a job that is run by the OpenStack CI system, please propose a patch to openstack-infra/config to delete it from the Zuul config. We only want to run what's useful, and we have other methods (the silent and experimental queues) to develop new jobs without creating noise. If there is a third-party CI system reporting non-voting jobs, um, I don't know what that means. If it bothers you, you might ask the third-party CI system to disable them and if they don't, then ask us to disable the third-party CI system. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 08/13/2014 06:23 PM, Mark McLoughlin wrote: On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Beautiful! Thank you so much to everyone involved. +1! Love this. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Wed, Aug 13, 2014 at 6:27 PM, James E. Blair cor...@inaugust.com wrote: If it is not worth looking at a job that is run by the OpenStack CI system, please propose a patch to openstack-infra/config to delete it from the Zuul config. We only want to run what's useful, and we have other methods (the silent and experimental queues) to develop new jobs without creating noise. If there is a third-party CI system reporting non-voting jobs, um, I don't know what that means. If it bothers you, you might ask the third-party CI system to disable them and if they don't, then ask us to disable the third-party CI system. I didn't meant that they were not worthwhile to look at I was just thinking it could be useful to sort them so we can easily identify from a UI perspective which one was voting or not. Chmouel ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 08/13/2014 06:35 PM, Russell Bryant wrote: On 08/13/2014 06:23 PM, Mark McLoughlin wrote: On Wed, 2014-08-13 at 12:05 -0700, James E. Blair wrote: cor...@inaugust.com (James E. Blair) writes: Sean Dague s...@dague.net writes: This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. I should also mention that there is a pending change to do something similar via site-local Javascript in our Gerrit: https://review.openstack.org/#/c/95743/ I don't think it's an ideal long-term solution, but if it works, we may have some immediate relief without all having to install greasemonkey scripts. You may have noticed that this has merged, along with a further change that shows the latest results in a table format. (You may need to force-reload in your browser to see the change.) Beautiful! Thank you so much to everyone involved. +1! Love this. Indeed. Amazeballs. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Chmouel Boudjnah chmo...@enovance.com writes: On Wed, Aug 13, 2014 at 6:27 PM, James E. Blair cor...@inaugust.com wrote: If it is not worth looking at a job that is run by the OpenStack CI system, please propose a patch to openstack-infra/config to delete it from the Zuul config. We only want to run what's useful, and we have other methods (the silent and experimental queues) to develop new jobs without creating noise. If there is a third-party CI system reporting non-voting jobs, um, I don't know what that means. If it bothers you, you might ask the third-party CI system to disable them and if they don't, then ask us to disable the third-party CI system. I didn't meant that they were not worthwhile to look at I was just thinking it could be useful to sort them so we can easily identify from a UI perspective which one was voting or not. I think part of why I responded with that is because this is not the first time I have heard someone say they don't want to see the non-voting jobs. If that's a real desire, we should get to the bottom of it. We don't actually want the CI system to be annoying. :) From my perspective, anything red should represent something that someone needs to process and either correct or make an informed decision to ignore. That is, a failing non-voting job should either cause the submitter to fix a problem with their code (same as a failing voting job), or investigate the result and determine that in this case, the non-voting job can be safely ignored (ie, is an expected result of the change). If we have non-voting jobs that don't match that criteria, we should remove them. Or make them voting. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 2 July 2014 16:11, Anita Kuno ante...@anteaya.info wrote: Hmmm, my first response - given that long chew we had on the ml[0] about the use of the word certified as well as the short confirmation we had in the tc meeting[1] that the word certified would not be used, but rather some version of the word 'tested' - how long until edits can be made to the cinder wiki to comply with that agreement? It's a wiki - anybody who cares enough can go and re-word it... -- Duncan Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 07/04/2014 08:11 AM, Duncan Thomas wrote: On 2 July 2014 16:11, Anita Kuno ante...@anteaya.info wrote: Hmmm, my first response - given that long chew we had on the ml[0] about the use of the word certified as well as the short confirmation we had in the tc meeting[1] that the word certified would not be used, but rather some version of the word 'tested' - how long until edits can be made to the cinder wiki to comply with that agreement? It's a wiki - anybody who cares enough can go and re-word it... True, but it is a cinder wiki. I don't know what words you want to use and don't want to lose the intent of the page. Also given that some folks in other places in cinder are still using references to certified I think it shows a strong message if someone in a leadership position in cinder makes the edits. Thanks Duncan, Anita. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 07/01/2014 12:27 PM, Asselin, Ramy wrote:
Anita,
This line [1] is effectively a sub-set of tempest-dsm-full, and what we're
currently running manually now. I far as I understood, this is the current
minimum. The exact sub-set (or full set, or if additional tests are allowed)
is still under discussion.
Fair enough, I need to make some time to attend Cinder weekly meetings.
Wednesdays are tough for me, but I need to make more of an effort.
Thanks for making me aware.
I created a WIP reference patch [2] for the cinder team that mimics the above
script to run these tests based off the similar Jenkins job-template used by
Openstack-Jenkins {pipeline}-tempest-dsvm-*
Thanks for sharing the link to your patch, I have given it an initial
review for formatting. I'll be interested to see the discussion from
this, I'm not sure where a design like this fits but I appreciate you
putting up a patch so we can gather some input.
Thanks Ramy,
Anita.
Ramy
[1]
https://github.com/openstack-dev/devstack/blob/master/driver_certs/cinder_driver_cert.sh#L97
[2]
https://review.openstack.org/#/c/93141/1/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml
-Original Message-
From: Duncan Thomas [mailto:duncan.tho...@gmail.com]
Sent: Tuesday, July 01, 2014 7:03 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 1 July 2014 14:44, Anita Kuno ante...@anteaya.info wrote:
On 07/01/2014 05:56 AM, Duncan Thomas wrote:
For the record, cinder gave a very clear definition of success in our
3rd party guidelines: Passes every test in tempest-dsm-full. If that
needs documenting somewhere else, please let me know. It may of
course change as we learn more about how 3rd party CI works out, so
the fewer places it is duplicated the better, maybe?
Thanks Duncan, I wasn't aware of this. Can we start with a url for
those guidelines in your reply to this post and then go from there?
https://wiki.openstack.org/wiki/Cinder/certified-drivers should make it clear
but doesn't, I'll get that cleared up.
https://etherpad.openstack.org/p/juno-cinder-3rd-party-cert-and-verification
mentions it, and various weekly meeting minutes also mention it.
--
Duncan Thomas
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 07/01/2014 10:03 AM, Duncan Thomas wrote: On 1 July 2014 14:44, Anita Kuno ante...@anteaya.info wrote: On 07/01/2014 05:56 AM, Duncan Thomas wrote: For the record, cinder gave a very clear definition of success in our 3rd party guidelines: Passes every test in tempest-dsm-full. If that needs documenting somewhere else, please let me know. It may of course change as we learn more about how 3rd party CI works out, so the fewer places it is duplicated the better, maybe? Thanks Duncan, I wasn't aware of this. Can we start with a url for those guidelines in your reply to this post and then go from there? https://wiki.openstack.org/wiki/Cinder/certified-drivers should make it clear but doesn't, I'll get that cleared up. https://etherpad.openstack.org/p/juno-cinder-3rd-party-cert-and-verification mentions it, and various weekly meeting minutes also mention it. Thanks for sharing those links, Duncan. Hmmm, my first response - given that long chew we had on the ml[0] about the use of the word certified as well as the short confirmation we had in the tc meeting[1] that the word certified would not be used, but rather some version of the word 'tested' - how long until edits can be made to the cinder wiki to comply with that agreement? Thanks Duncan, Anita. [0] http://lists.openstack.org/pipermail/openstack-dev/2014-June/036933.html [1] http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-06-17-20.03.html 2.b ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 30 June 2014 16:49, Anita Kuno ante...@anteaya.info wrote: Right now that dashboard introduces more confusion than it alleviates since the definition of success in regards to third party ci systems has yet to be defined by the community. For the record, cinder gave a very clear definition of success in our 3rd party guidelines: Passes every test in tempest-dsm-full. If that needs documenting somewhere else, please let me know. It may of course change as we learn more about how 3rd party CI works out, so the fewer places it is duplicated the better, maybe? -- Duncan Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 07/01/2014 05:56 AM, Duncan Thomas wrote: On 30 June 2014 16:49, Anita Kuno ante...@anteaya.info wrote: Right now that dashboard introduces more confusion than it alleviates since the definition of success in regards to third party ci systems has yet to be defined by the community. For the record, cinder gave a very clear definition of success in our 3rd party guidelines: Passes every test in tempest-dsm-full. If that needs documenting somewhere else, please let me know. It may of course change as we learn more about how 3rd party CI works out, so the fewer places it is duplicated the better, maybe? Thanks Duncan, I wasn't aware of this. Can we start with a url for those guidelines in your reply to this post and then go from there? Thanks, Anita. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 1 July 2014 14:44, Anita Kuno ante...@anteaya.info wrote: On 07/01/2014 05:56 AM, Duncan Thomas wrote: For the record, cinder gave a very clear definition of success in our 3rd party guidelines: Passes every test in tempest-dsm-full. If that needs documenting somewhere else, please let me know. It may of course change as we learn more about how 3rd party CI works out, so the fewer places it is duplicated the better, maybe? Thanks Duncan, I wasn't aware of this. Can we start with a url for those guidelines in your reply to this post and then go from there? https://wiki.openstack.org/wiki/Cinder/certified-drivers should make it clear but doesn't, I'll get that cleared up. https://etherpad.openstack.org/p/juno-cinder-3rd-party-cert-and-verification mentions it, and various weekly meeting minutes also mention it. -- Duncan Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Anita,
This line [1] is effectively a sub-set of tempest-dsm-full, and what we're
currently running manually now. I far as I understood, this is the current
minimum. The exact sub-set (or full set, or if additional tests are allowed) is
still under discussion.
I created a WIP reference patch [2] for the cinder team that mimics the above
script to run these tests based off the similar Jenkins job-template used by
Openstack-Jenkins {pipeline}-tempest-dsvm-*
Ramy
[1]
https://github.com/openstack-dev/devstack/blob/master/driver_certs/cinder_driver_cert.sh#L97
[2]
https://review.openstack.org/#/c/93141/1/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml
-Original Message-
From: Duncan Thomas [mailto:duncan.tho...@gmail.com]
Sent: Tuesday, July 01, 2014 7:03 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 1 July 2014 14:44, Anita Kuno ante...@anteaya.info wrote:
On 07/01/2014 05:56 AM, Duncan Thomas wrote:
For the record, cinder gave a very clear definition of success in our
3rd party guidelines: Passes every test in tempest-dsm-full. If that
needs documenting somewhere else, please let me know. It may of
course change as we learn more about how 3rd party CI works out, so
the fewer places it is duplicated the better, maybe?
Thanks Duncan, I wasn't aware of this. Can we start with a url for
those guidelines in your reply to this post and then go from there?
https://wiki.openstack.org/wiki/Cinder/certified-drivers should make it clear
but doesn't, I'll get that cleared up.
https://etherpad.openstack.org/p/juno-cinder-3rd-party-cert-and-verification
mentions it, and various weekly meeting minutes also mention it.
--
Duncan Thomas
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Sat, Jun 28, 2014 at 08:26:44AM -0500, Matt Riedemann wrote: On 6/27/2014 7:35 AM, Daniel P. Berrange wrote: On Fri, Jun 27, 2014 at 07:40:51AM -0400, Sean Dague wrote: It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this. That's an understatement about the UI :-) It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. Agreed, it would be a great improvement in usability if we stopped all CI systems, including our default Jenkins, from ever commenting on reviews. At most gating CIs should +1/-1. Having a table of results displayed, pulling the data from an external result tracking system would be a great idea. Even better if this external system had a nice button you can press to trigger re-check, so we can stop using comments for that too. I would disagree with this idea since it's equivalent to 'recheck no bug' and that's naughty, because then we don't track race bugs as well. It could easily have a text field for entering a bug number. The point is to stop adding comments to gerrit that aren't related to code review. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 06/29/2014 09:39 AM, Joshua Hesketh wrote: On 6/28/14 10:40 AM, James E. Blair wrote: An alternate approach would be to have third-party CI systems register jobs with OpenStack's Zuul rather than using their own account. This would mean only a single report of all jobs (upstream and 3rd-party) per-patchset. It significantly reduces clutter and makes results more accessible -- but even with one system we've never actually wanted to have Jenkins results in comments, so I think one of the other options would be preferred. Nonetheless, this is possible with a little bit of work. I agree this isn't the preferred solution, but I disagree with the little bit of work. This would require CI systems registering with gearman which would mean security issues. The biggest problem with this though is that zuul would be stuck waiting from results from 3rd parties which often have very slow return times. Right, one of the other issues is the quality of the CI results varies as well. I think one of the test result burn out issues right now is based on the fact that they are too rolled up as it is. For instance, a docs only change gets Tempest results, which humans know are irrelevant, but Jenkins insists they aren't. I think that if we rolled up more information, and waited longer, we'd be in a worse state. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Joshua Hesketh joshua.hesk...@rackspace.com writes: On 6/28/14 10:40 AM, James E. Blair wrote: An alternate approach would be to have third-party CI systems register jobs with OpenStack's Zuul rather than using their own account. This would mean only a single report of all jobs (upstream and 3rd-party) per-patchset. It significantly reduces clutter and makes results more accessible -- but even with one system we've never actually wanted to have Jenkins results in comments, so I think one of the other options would be preferred. Nonetheless, this is possible with a little bit of work. I agree this isn't the preferred solution, but I disagree with the little bit of work. This would require CI systems registering with gearman which would mean security issues. The biggest problem with this though is that zuul would be stuck waiting from results from 3rd parties which often have very slow return times. Security issues is a bit vague. They already register with Gerrit; I'm only suggesting that the point of aggregation would change. I'm anticipating that they would use authenticated SSL, with ACLs scoped to the names of jobs each system is permitted to run. From the perspective of overall security as well as network topology (ie, firewalls), very little changes. The main differences are third party CI systems don't have to run Zuul anymore, and we go back to having a smaller number of votes/comments. Part of the little bit of work I was referring to was adding a timeout. That should truly be not much work, and work we're planning on doing anyway to help with the tripleo cloud. But anyway, it's not important to design this out if we prefer another solution (and I prefer the table of results separated from comments). -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Dan Smith d...@danplanet.com wrote on 06/27/2014 12:33:48 PM: What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. If it really does show up right in Gerrit as if it were integrated, then that would be fine with me. I think the biggest problem we have right now is that a lot of the CI systems are very inconsistent in their reporting and we often don't realize when one of them *hasn't* voted. If the new thing could fill out the chart based on everything we expect to see a vote from, so that it's very clear that one is missing, then that's a net win right there. There is a similar old bug for that, with a good suggestion for how it could possibly be done: https://bugs.launchpad.net/openstack-ci/+bug/1251758 Kurt Taylor (krtaylor) OpenStack Development Lead - PowerKVM CI IBM Linux Technology Center ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Sean Dague s...@dague.net wrote on 06/30/2014 06:03:50 AM: From: Sean Dague s...@dague.net To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org, Date: 06/30/2014 06:09 AM Subject: Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit On 06/29/2014 09:39 AM, Joshua Hesketh wrote: On 6/28/14 10:40 AM, James E. Blair wrote: An alternate approach would be to have third-party CI systems register jobs with OpenStack's Zuul rather than using their own account. This would mean only a single report of all jobs (upstream and 3rd-party) per-patchset. It significantly reduces clutter and makes results more accessible -- but even with one system we've never actually wanted to have Jenkins results in comments, so I think one of the other options would be preferred. Nonetheless, this is possible with a little bit of work. I agree this isn't the preferred solution, but I disagree with the little bit of work. This would require CI systems registering with gearman which would mean security issues. The biggest problem with this though is that zuul would be stuck waiting from results from 3rd parties which often have very slow return times. Right, one of the other issues is the quality of the CI results varies as well. Agreed. After last summit, Anita, Jay and I decided to start gathering a team of 3rd party testers that have the goal of improving the quality of the third party systems. We are starting with gathering global unwritten requirements, improving documentation and reaching out to new projects that will have third party testing needs. We are still in the early stages but now have weekly meetings to discuss what needs to be done and track progress. https://wiki.openstack.org/wiki/Meetings/ThirdParty I think one of the test result burn out issues right now is based on the fact that they are too rolled up as it is. For instance, a docs only change gets Tempest results, which humans know are irrelevant, but Jenkins insists they aren't. I think that if we rolled up more information, and waited longer, we'd be in a worse state. Maybe it could promptly timeout and then report the system that did not complete? That would also have the benefit of enforcing a time limit on reporting results. Kurt Taylor (krtaylor) OpenStack Development Lead - PowerKVM CI IBM Linux Technology Center ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
2014-06-30 19:17 GMT+04:00 Kurt Taylor krtay...@us.ibm.com: Dan Smith d...@danplanet.com wrote on 06/27/2014 12:33:48 PM: If it really does show up right in Gerrit as if it were integrated, then that would be fine with me. I think the biggest problem we have right now is that a lot of the CI systems are very inconsistent in their reporting and we often don't realize when one of them *hasn't* voted. If the new thing could fill out the chart based on everything we expect to see a vote from, so that it's very clear that one is missing, then that's a net win right there. There is a similar old bug for that, with a good suggestion for how it could possibly be done: https://bugs.launchpad.net/openstack-ci/+bug/1251758 What about to have report like this: http://stackalytics.com/report/ci/neutron/7 ? Thanks, Ilya ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There is a similar old bug for that, with a good suggestion for how it could possibly be done: https://bugs.launchpad.net/openstack-ci/+bug/1251758 This isn't what I'm talking about. What we need is, for each new patchset on a given change, an empty table listing all the answers we expect to see (i.e. one for each of the usual suspect nova CI systems). The above bug (AFAICT) is simply for tracking last-status of each, so that if one stops reporting entirely (as minesweeper often does), we get some indication that the *system* is broken. - --Dan -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTsYc+AAoJEBeZxaMESjNVcQYH+wayg4T9QFe4tTvGn24PisCf 5cEaeSkwXl+Adiae5cCfCGSTjlErK4lpFtzFapKukcM0+eEp464toskl7vNC0izp UWCpcg2gbON6Ef/AMa1+PT8uXR9OYAo+/eU8NUJNM01ajeZqqe3H3jnltgoUau0O fq3O3+Wa2PxBTAVVGi3HXJl4SWpVdEuYDZYOBOtkDXwhIS/hvBdIRuJwt0CygxHx 78WatFsQ09tBHaQCJbs2E+Oar0rD4sF93qjG8jAFiVB/0SJ6wV7AsLColVId2hbe Qfua3Q6CufJBO2WHV7JORX2fBSOTUmcPcOM1IE4/lgXGiyu3aw5ataL9e4qxudQ= =VoHH -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 6/28/14 10:40 AM, James E. Blair wrote: An alternate approach would be to have third-party CI systems register jobs with OpenStack's Zuul rather than using their own account. This would mean only a single report of all jobs (upstream and 3rd-party) per-patchset. It significantly reduces clutter and makes results more accessible -- but even with one system we've never actually wanted to have Jenkins results in comments, so I think one of the other options would be preferred. Nonetheless, this is possible with a little bit of work. I agree this isn't the preferred solution, but I disagree with the little bit of work. This would require CI systems registering with gearman which would mean security issues. The biggest problem with this though is that zuul would be stuck waiting from results from 3rd parties which often have very slow return times. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Matt Riedemann mrie...@linux.vnet.ibm.com writes: I would be good with Jenkins not reporting on a successful run, or if rather than a comment from Jenkins the vote in the table had a link to the test results, so if you get a -1 from Jenkins you can follow the link from the -1 in the table rather than the comment (to avoid cluttering up the review comments, especially if it's a +1). The problem with that is it makes non-voting jobs very difficult to see, not to mention ancillary information like job runtime. Plus it adds extra clicks to get to jobs whose output are frequently reviewed even with positive votes, such as docs jobs (on docs-draft.o.o). I think a new table on the page, separate from the comment stream, with only the latest results of each job is the ideal presentation. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all] 3rd Party CI vs. Gerrit
It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this. A couple of things have fallen out of this. 3rd Party CI bots outnumber Human comments on changes on some projects (Nova / Neutron). That has an impact on the readability of the votes section (on a neutron change the files in the change are rarely above the fold), the readability of the comments. 3rd Party CI systems haven't become all that reliable. They fall into the same problems that Jenkins hits with cloud networking, race bugs in OpenStack, but also new bugs around site configs. It's kind of a testament to how much we've learned about how to keep the machine running that the upstream CI system, even with all it's faults, still trends more reliable than most of the 3rd Party systems. Commenting in Gerrit was to eventually get towards voting in Gerrit. But my experience at this point is reviewers are at CI fatigue and are mostly not paying attention to the votes. Heck, when we're dealing with a bunch of bugs in the gate most reviewers want to ignore the Jenkins votes too, which is why you get the recheck grinding behavior. This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. I'm not signing up for this particular mission, but I wanted to stick it out there to figure out if the idea had merrit, and if it did, if it excited anyone to enough to dive on it. -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 27/06/14 07:40 -0400, Sean Dague wrote: It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this. A couple of things have fallen out of this. 3rd Party CI bots outnumber Human comments on changes on some projects (Nova / Neutron). That has an impact on the readability of the votes section (on a neutron change the files in the change are rarely above the fold), the readability of the comments. 3rd Party CI systems haven't become all that reliable. They fall into the same problems that Jenkins hits with cloud networking, race bugs in OpenStack, but also new bugs around site configs. It's kind of a testament to how much we've learned about how to keep the machine running that the upstream CI system, even with all it's faults, still trends more reliable than most of the 3rd Party systems. Commenting in Gerrit was to eventually get towards voting in Gerrit. But my experience at this point is reviewers are at CI fatigue and are mostly not paying attention to the votes. Heck, when we're dealing with a bunch of bugs in the gate most reviewers want to ignore the Jenkins votes too, which is why you get the recheck grinding behavior. This has all gone far enough that someone actually wrote a Grease Monkey script to purge all the 3rd Party CI content out of Jenkins UI. People are writing mail filters to dump all the notifications. Dan Berange filters all them out of his gerrit query tools. It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. I'm not signing up for this particular mission, but I wanted to stick it out there to figure out if the idea had merrit, and if it did, if it excited anyone to enough to dive on it. I agree all those comments cause way to much noise. The most important bit of the comment is the link that takes you to the CI logs so, it may be probably enough to just have 1 link that will take someone to the builds of the 3rd party CI for that specific change. Along the lines of what you just proposed, what if each 3rd Party CI username in the review's list just links to this builds-list resources that a developer can go to to check the logs. Would that be too much of a hack for gerrit? I'd like to avoid us to create yet-another-monitoring-resource that we'll have to go to in order to figure out what's going on. FWIW, the above could also be done for our Jenkins. Overall, +1 for the email and idea. Flavio -- @flaper87 Flavio Percoco pgpwpRJnlxUen.pgp Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Fri, Jun 27, 2014 at 07:40:51AM -0400, Sean Dague wrote: It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this. That's an understatement about the UI :-) It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. Agreed, it would be a great improvement in usability if we stopped all CI systems, including our default Jenkins, from ever commenting on reviews. At most gating CIs should +1/-1. Having a table of results displayed, pulling the data from an external result tracking system would be a great idea. Even better if this external system had a nice button you can press to trigger re-check, so we can stop using comments for that too. To me the ideal world is where the only things adding comments to reviews are human and their comments are actually about the code in the patch :-) Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On Fri, Jun 27, 2014 at 6:35 AM, Daniel P. Berrange berra...@redhat.com wrote: On Fri, Jun 27, 2014 at 07:40:51AM -0400, Sean Dague wrote: It's clear that lots of projects want 3rd Party CI information on patches. But it's also clear that 6 months into this experiment with a lot of 3rd Party CI systems, the Gerrit UI is really not great for this. That's an understatement about the UI :-) It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. Yes please, and no I'm not signing up for it either :) What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. Agreed, it would be a great improvement in usability if we stopped all CI systems, including our default Jenkins, from ever commenting on reviews. At most gating CIs should +1/-1. Having a table of results displayed, pulling the data from an external result tracking system would be a great idea. Even better if this external system had a nice button you can press to trigger re-check, so we can stop using comments for that too. To me the ideal world is where the only things adding comments to reviews are human and their comments are actually about the code in the patch :-) Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 What if 3rd Party CI didn't vote in Gerrit? What if it instead published to some 3rd party test reporting site (a thing that doesn't yet exist). Gerrit has the facility so that we could inject the dashboard content for this in Gerrit in a little table somewhere, but the data would fundamentally live outside of Gerrit. It would also mean that all the aggregate reporting of 3rd Party CI that's being done in custom gerrit scripts, could be integrated directly into such a thing. If it really does show up right in Gerrit as if it were integrated, then that would be fine with me. I think the biggest problem we have right now is that a lot of the CI systems are very inconsistent in their reporting and we often don't realize when one of them *hasn't* voted. If the new thing could fill out the chart based on everything we expect to see a vote from, so that it's very clear that one is missing, then that's a net win right there. But at the end of the day, we need to make sure we're improving visibility and not hiding all the noise they generate just because it's annoying. As long as something like what was suggested here is injected right into the gerrit API, then I'm in favor of something like this. - --Dan -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJTrar8AAoJEBeZxaMESjNVizsH/RfjoJOlX80N+jviDHG4yMUv n9dRV2vFfitKmapN8HIWdo6aFjRDHb5t3BWcO+9xP7+w/nZYpC88Y6Eq8c8HeZPz WIeVVY8/AmL/VVGXNojeIoCdszR8vdlaEaOrlKCjQIr8zqZJrz6LgPXIfoVCVrMg hHN2+TM4rn9scFHlKSnf8vq7rLj/aTfzG+ITSSksslM2YxWkqDFWWBTrTnqrWHx5 Uk91JSFmWTC2UnbyVzPI88D0a7HGkpyxgFnWb8Y63RBr/JygFisoBLaqbHA9o9ML ajI+st8ySBEwWGyVkckzX5Go1KWgTP9pH1zwhpZz+qvEZCJA6Pbp16+r+kTFBew= =NkTB -END PGP SIGNATURE- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Sean, there is a proposal[1] in upstream gerrit to fix this issue. David's proposal is to make Gerrit handle multiple notifications channels per project which would allow us to treat bot notifications differently than human notifications. He has the same problem as we do, most of his builds are done using 'third party' CI so he has many more third party systems reporting to his gerrit instance than we do. Please take a look and provide feedback. [1] *https://code.google.com/p/gerrit/issues/detail?id=2465 https://code.google.com/p/gerrit/issues/detail?id=2465* ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
On 06/27/2014 02:19 PM, Zaro wrote: Sean, there is a proposal[1] in upstream gerrit to fix this issue. David's proposal is to make Gerrit handle multiple notifications channels per project which would allow us to treat bot notifications differently than human notifications. He has the same problem as we do, most of his builds are done using 'third party' CI so he has many more third party systems reporting to his gerrit instance than we do. Please take a look and provide feedback. [1] _https://code.google.com/p/gerrit/issues/detail?id=2465_ That seems to require the new Change Screen? -Sean -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
David did suggest adding REST api endpoints to get data for each channel so it doesn't necessarily require you to even use the gerrit web ui. However the web UI's old screen is pretty much dead so I assume the presentation would only be available in new change screen. I know Openstackers have had reservations about the new change screen but please don't judge it by trying the gerrit 2.8.x implementation. It has improved quite a bit. I have been using it on the https://gerrit-review.googlesource.com and do prefer it over the old change screen :) -Khai On Fri, Jun 27, 2014 at 6:38 PM, Sean Dague s...@dague.net wrote: On 06/27/2014 02:19 PM, Zaro wrote: Sean, there is a proposal[1] in upstream gerrit to fix this issue. David's proposal is to make Gerrit handle multiple notifications channels per project which would allow us to treat bot notifications differently than human notifications. He has the same problem as we do, most of his builds are done using 'third party' CI so he has many more third party systems reporting to his gerrit instance than we do. Please take a look and provide feedback. [1] _https://code.google.com/p/gerrit/issues/detail?id=2465_ That seems to require the new Change Screen? -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
I do quite like the conflicts with part, I was thinking about that the other day. That would be hugely useful. -Sean On 06/27/2014 02:56 PM, Zaro wrote: David did suggest adding REST api endpoints to get data for each channel so it doesn't necessarily require you to even use the gerrit web ui. However the web UI's old screen is pretty much dead so I assume the presentation would only be available in new change screen. I know Openstackers have had reservations about the new change screen but please don't judge it by trying the gerrit 2.8.x implementation. It has improved quite a bit. I have been using it on the https://gerrit-review.googlesource.com and do prefer it over the old change screen :) -Khai On Fri, Jun 27, 2014 at 6:38 PM, Sean Dague s...@dague.net mailto:s...@dague.net wrote: On 06/27/2014 02:19 PM, Zaro wrote: Sean, there is a proposal[1] in upstream gerrit to fix this issue. David's proposal is to make Gerrit handle multiple notifications channels per project which would allow us to treat bot notifications differently than human notifications. He has the same problem as we do, most of his builds are done using 'third party' CI so he has many more third party systems reporting to his gerrit instance than we do. Please take a look and provide feedback. [1] _https://code.google.com/p/gerrit/issues/detail?id=2465_ That seems to require the new Change Screen? -Sean -- Sean Dague http://dague.net ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org mailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net signature.asc Description: OpenPGP digital signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] 3rd Party CI vs. Gerrit
Sean Dague s...@dague.net writes: It seems what we actually want is a dashboard of these results. We want them available when we go to Gerrit, but we don't want them in Gerrit itself. I agree with most of what you wrote, particularly that we want them available in Gerrit and with a sensible presentation. Though I don't think that necessarily excludes storing the data in Gerrit, as Khai points out. I think one ideal interface is a table in Gerrit of all the jobs run on a change (from all systems) and their results (with links). That sounds like the project David is working on (that Khai pointed out), which isn't surprising -- we sketched out the initial design with him about three years ago; our needs are very similar. David's proposal is a table like: Job/Category | started on | ended on | href | status foo [...] bar [...] bat […] I think that's a great approach because it provides the needed information to reviewers in an appropriate interface in Gerrit. I suspect that whether the data are stored in Gerrit or a different system, the (future) vinz web ui could retrieve it from either. An alternate approach would be to have third-party CI systems register jobs with OpenStack's Zuul rather than using their own account. This would mean only a single report of all jobs (upstream and 3rd-party) per-patchset. It significantly reduces clutter and makes results more accessible -- but even with one system we've never actually wanted to have Jenkins results in comments, so I think one of the other options would be preferred. Nonetheless, this is possible with a little bit of work. If anyone is seriously interested in working on Sean's proposal or something similar, please write up a spec in the openstack-infra/infra-specs repo. If you'd like to help with David's proposal, the upstream Gerrit project is the best place to collaborate. -Jim ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
