Re: [openstack-dev] [keystone] gate-keystoneclient-dsvm-functional-ubuntu-xenial is broken
After some investigation I've found the possible issue: the functional tests run in parallel, some of them create and delete roles and others use tokens to perform the creation/update/delete of other types of fixtures. The problem is that when we delete a role, we also revoke *all* tokens from a user that has any assignment containing that role - so? Race condition: if we are executing a not related operation and another test deletes a role, the user tokens will be revoked resulting in a request error. The strange part is that reverting this commit [1], the tests seem to work fine most of the times - what makes think that commit actually *fixes* a big issue in our revoke events (since before it, we would not revoke such types of tokens). I can see a couple of options: - Create brand new users and role_assignments to be responsible to handle operations in the fixtures for each test - Change the "framework" of the tests and rely on tempest plugins What to think? Makes sense? On Tue, Sep 20, 2016 at 11:03 AM, Steve Martinelli wrote: > Since September 14th the keystoneclient functional test job has been > broken. Let's be mindful of infra resources and stop rechecking the patches > there. Anyone have time to investigate this? > > See patches https://review.openstack.org/#/c/369469/ or > https://review.openstack.org/#/c/371324/ > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Rodrigo Duarte Sousa Senior Quality Engineer @ Red Hat MSc in Computer Science http://rodrigods.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] gate-keystoneclient-dsvm-functional-ubuntu-xenial is broken
On Wed, Sep 21, 2016 at 8:59 AM, Rodrigo Duarte wrote: > After some investigation I've found the possible issue: the functional > tests run in parallel, some of them create and delete roles and others use > tokens to perform the creation/update/delete of other types of fixtures. > The problem is that when we delete a role, we also revoke *all* tokens > from a user that has any assignment containing that role - so? Race > condition: if we are executing a not related operation and another test > deletes a role, the user tokens will be revoked resulting in a request > error. > > The strange part is that reverting this commit [1], the tests seem to work > fine most of the times - what makes think that commit actually *fixes* a > big issue in our revoke events (since before it, we would not revoke such > types of tokens). > > I can see a couple of options: > - Create brand new users and role_assignments to be responsible to handle > operations in the fixtures for each test > This makes sense to me. There was a bug in the tests and this corrects it. > - Change the "framework" of the tests and rely on tempest plugins > > I don't know why this was suggested. - Brant > What to think? Makes sense? > > On Tue, Sep 20, 2016 at 11:03 AM, Steve Martinelli > wrote: > >> Since September 14th the keystoneclient functional test job has been >> broken. Let's be mindful of infra resources and stop rechecking the patches >> there. Anyone have time to investigate this? >> >> See patches https://review.openstack.org/#/c/369469/ or >> https://review.openstack.org/#/c/371324/ >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Rodrigo Duarte Sousa > Senior Quality Engineer @ Red Hat > MSc in Computer Science > http://rodrigods.com > > _ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] gate-keystoneclient-dsvm-functional-ubuntu-xenial is broken
Forgot to add the commit reference :) [1] https://review.openstack.org/#/c/368244/ On Wed, Sep 21, 2016 at 10:59 AM, Rodrigo Duarte wrote: > After some investigation I've found the possible issue: the functional > tests run in parallel, some of them create and delete roles and others use > tokens to perform the creation/update/delete of other types of fixtures. > The problem is that when we delete a role, we also revoke *all* tokens > from a user that has any assignment containing that role - so? Race > condition: if we are executing a not related operation and another test > deletes a role, the user tokens will be revoked resulting in a request > error. > > The strange part is that reverting this commit [1], the tests seem to work > fine most of the times - what makes think that commit actually *fixes* a > big issue in our revoke events (since before it, we would not revoke such > types of tokens). > > I can see a couple of options: > - Create brand new users and role_assignments to be responsible to handle > operations in the fixtures for each test > - Change the "framework" of the tests and rely on tempest plugins > > What to think? Makes sense? > > On Tue, Sep 20, 2016 at 11:03 AM, Steve Martinelli > wrote: > >> Since September 14th the keystoneclient functional test job has been >> broken. Let's be mindful of infra resources and stop rechecking the patches >> there. Anyone have time to investigate this? >> >> See patches https://review.openstack.org/#/c/369469/ or >> https://review.openstack.org/#/c/371324/ >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > -- > Rodrigo Duarte Sousa > Senior Quality Engineer @ Red Hat > MSc in Computer Science > http://rodrigods.com > -- Rodrigo Duarte Sousa Senior Quality Engineer @ Red Hat MSc in Computer Science http://rodrigods.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev