On Fri, Aug 22, 2014, at 05:55 AM, Ihar Hrachyshka wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hi all, > > this week is quite bumpy for unit testing in gate. First, it was > upgrade to new 'tox' version that broke quite some branches. > I did a ton of work to make the tox upgrade go smoothly because we knew it would be somewhat painful. About a month ago I sent mail to this list  describing the problem. This thread included a pointer to the bug filed to track this  and example work around changes  which I wrote and proposed for as many projects and branches as I had time to test at that point.
Updating tox to 1.7.2 is important for a couple reasons. We get a lot of confused developers wondering why using tox doesn't work to run their tests when all of our documentation says just run tox. Well you needed a special version (1.6.1). Communicating that to everyone that tries to run tox is hard. It is also important because tox adds new features like the hashseed randomization. This is the cause of our problems but it is exposing real bugs in openstack . We should be fixing these issues and hopefully my proposed workarounds are only temporary. I decided to push ahead  and upgrade tox a couple days ago for a couple reasons. This is an important change as illustrated above and feature freeze and stabilization are rapidly approaching and this probably needed to get in soon to have a chance at getting in at all. I felt this was appropriate because I had done a ton of work prior to make things go as smoothly as possible. Where things did not go smoothly was on the reviews for my workaround. Some changes were basically ignored  others ran into procedural paperwork associated with stable branches that are not quite appropriate for changes of this type . I get that generally we only want to backport things from master and that we have some specific way to cherry pick things, but this type of change is to address issues with stable/foo directly and has nothing to do with master. I did eventually go through the "backport" dance for most of these changes despite this not actually being a true backport.  http://lists.openstack.org/pipermail/openstack-dev/2014-July/041283.html  https://bugs.launchpad.net/cinder/+bug/1348818  https://review.openstack.org/#/c/109700/  http://lists.openstack.org/pipermail/openstack-dev/2014-July/041496.html  http://lists.openstack.org/pipermail/openstack-dev/2014-August/042010.html  https://review.openstack.org/#/c/109749/  https://review.openstack.org/#/c/109759/  https://review.openstack.org/#/c/109750/ With all of that out of the way are there suggestions for how we can do this better next time? Do we need more time (I gave us about 4 weeks which seemed like plenty to me)? Perhaps I should send more reminder emails? Feedback is very welcome. Thanks, Clark > > And today new testtools 0.9.36 were released and were caught by gate, > which resulted in the following unit test failures in multiple projects: > > TestCase.setUp was already called. Do not explicitly call setUp from > your tests. In your own setUp, use super to call the base setUp. > > All branches are affected: havana, icehouse, and master. > > This is because the following check was released with the new version > of the library: > https://github.com/testing-cabal/testtools/commit/5c3b92d90a64efaecdc4010a98002bfe8b888517 > > And the temporary fix is to merge the version pin patch in global > requirements, backport it to stable branches, and merge the updates > from Openstack Proposal Bot to all affected projects. The patch for > master requirements is: https://review.openstack.org/#/c/116267/ > > In the meantime, projects will need to fix their tests not to call > setUp() and tearDown() twice. This will be the requirement to unpin > the version of the library. > > So, please review, backport, and make sure it lands in project > requirements files. > > /Ihar > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.22 (Darwin) > > iQEcBAEBCgAGBQJT9z3cAAoJEC5aWaUY1u57DtsIAOFtK2i4zkMcC79nOrc5w9DW > oO2b064eyLwwbQEaWeeIL2JBSLBxqNV5zeN0eZB3Sq7LQLv0oPaUNTMFG2+gvask > JHCTAGKz776Rt7ptcfmpHURwcT9L//+1HXvd+ADtO0sYKwgmvaBF7aA4WFa4TseG > JCnAsi5OiOZZgTo/6U1B55srHkZr0DWxqTkKKysZJbR2Pr/ZT9io8yu9uucaz9VH > uNLfggtCcjGgccl7IqSUtVRf3lsSGuvBAxVqMszSFJQmFCjy2E26GfsTApp9KXtQ > gbCpEns8QCnt6KF9rygjHLMbYikjbITuUfSL2okZelX9VpKNx0CS29K/tRg5/BA= > =YavB > -----END PGP SIGNATURE----- > > _______________________________________________ > OpenStack-dev mailing list > OpenStackfirstname.lastname@example.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStackemail@example.com http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev