Hi Anita, et. all, I understand if the -1 is being voted because of testing framework failures, it is a real pain.
Assuming the framework is working fine, I have noticed that there could be genuine failures because of devstack failing to fully stack up or the test failures. In my specific case, I have observed that test_network_basic_ops() have 70% success rate and devstack fails once in a while as well - and these failures are not because of the patch being reviewed. Therefore, being sensitive to patch submitters, and not causing -1 votes, I am not running basic ops test. However, I would eventually like to start running this test. This means we will start to vote -1 once in while (or lots of time - depending upon the test results). My question is as long as a -1 vote is casted with logs of the results, is it OK? - even if the failures are because of known bugs/issues? regards.. -Sukhdev On Thu, Jan 2, 2014 at 6:11 AM, Anita Kuno <ante...@anteaya.info> wrote: > On 12/30/2013 09:32 PM, Anita Kuno wrote: > > Please. > > > > When your third party testing structure votes on patches and your > > testing structure is not stable, it will vote with a -1 on patches. > > > > This results in three consequences: > > 1. The patch it votes on starts a countdown for abandonment, this is > > frustrating for developers. > > 2. Reviewers who use -Verified-1 as a filter criteria will not review a > > patch with a -1 in the verification column in the Gerrit dashboard. This > > prevents developers from progressing in their work, and this also > > prevents reviewers from reviewing patches that need to be assessed. > > 3. Third party testing that does not provide publicly accessible logs > > leave developers with no way to diagnose the issue, which makes it very > > difficult for a developer to fix, leaving the patch in a state of limbo. > > > > You can post messages to the patches, including a stable working url to > > the logs for your tests, as you continue to work on your third party > tests. > > > > You are also welcome to post a success of failure message, just please > > refrain from allowing your testing infrastructure to vote on patches > > until your testing infrastructure is working, reliably, and has logs > > that developers can use to fix problems. > > > > The list of third party testing accounts are found here.[0] > > > > Right now there are three plugins that need to remove voting until they > > are stable. > > > > Please be active in #openstack-neutron, #openstack-qa, and the mailing > > list so that if there is an issue with your testing structure, people > > can come and talk to you. > > > > Thank you, > > Anita. > > > > [0]https://review.openstack.org/#/admin/groups/91,members > > > Keep in mind that the email provided in this list is expected to be an > email people can use to contact you with questions and concerns about > your testing interactions. [0] > > The point of the exercise is to provide useful and helpful information > to developers and reviewers so that we all improve our code and create a > better, more integrated product than we have now. > > Please check the email inboxes of the email addresses you have provided > and please respond to inquires in a timely fashion. > > Please also remember people will look for you on irc, so having a > representative available on irc for discussion will give you some useful > feedback on ensuring your third party testing structure is behaving as > efficiently as possible. > > We have a certain level of tolerance for unproductive noise while you > are getting the bugs knocked out of your system. If developers are > trying to contact you for more information and there is no response, > third party testing structures that fail to comply with the expectation > that they will respond to questions, will be addressed on a case by case > basis. > > Thank you in advance for your kind attention, > Anita. > > [0] https://review.openstack.org/#/admin/groups/91,members > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev