Excerpts from Lance Bragstad's message of 2016-08-03 11:26:56 -0500: > Sending a follow-up because I think we ended up finding something relevant > to this discussion. > > As keystone moves towards making fernet the default, one of our work items > was to mock the system clock in tests. This allows us to advance the clock > by one second where we need to avoid sub-second race conditions. To do this > we used freezegun [0]. We recently landed a bunch of fixes to do this. > > It turns out that there is a possible race between when freezegun patches > it's modules and when the test runs. This turned up in a patch I was > working on locally and I noticed certain clock operations weren't using the > fake time object from freezegun. As a work-around, we can leverage the > set_time_override() method from oslo_utils.timeutils to make sure we are > using the fake time from within the frozen time context. In my testing > locally this worked.
Supporting mocking time operations in tests is the primary reason some of those functions exist at all. Doug > > If keystone requires a hybrid approach to patching > (oslo_utils.timeutils.set_time_override() + freezegun), we should build it > into a well documented hybrid context manager so that's its more apparent > why we need it. > > Sean, I can start working on this to see if it starts mitigating the races > you're seeing. > > [0] https://pypi.python.org/pypi/freezegun > > On Tue, Aug 2, 2016 at 9:21 AM, Lance Bragstad <[email protected]> wrote: > > > Hi Sean, > > > > Thanks for the information. This obviously looks Fernet-related and I > > would be happy to spend some cycles on it. We recently landed a bunch of > > refactors in keystone to improve Fernet test coverage. This could be > > related to those refactors. Just double checking - but you haven't opened a > > bug in launchpad for this yet have you? > > > > Thanks for the heads up! > > > > On Tue, Aug 2, 2016 at 5:32 AM, Sean Dague <[email protected]> wrote: > > > >> One of my concerns about stacking up project unit tests in the > >> requirements jobs, is the unit tests aren't as free of races as you > >> would imagine. Because they only previously impacted the one project > >> team, those teams are often just fast to recheck instead of get to the > >> bottom of it. Cross testing with them in a voting way changes their > >> impact. > >> > >> The keystone unit tests have a existing race condition in them, which > >> recently failed an unrelated requirements bump - > >> > >> http://logs.openstack.org/50/348250/6/check/gate-cross-keystone-python27-db-ubuntu-xenial/962327d/console.html#_2016-08-02_03_52_14_537923 > >> > >> I'm not fully sure where to go from here. But wanted to make sure that > >> data is out there. Any keystone folks who can dive into and sort it out > >> would be highly appreciated. > >> > >> -Sean > >> > >> -- > >> Sean Dague > >> http://dague.net > >> > >> __________________________________________________________________________ > >> OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > >> [email protected]?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >> > > > > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
