Thanks Matt. Agreed on 1, 2, 3, and 4. The reason for #5 is that there are other changes (for which we have FFE's) which will be merged and a new version of the client requested.
Specifically, this change https://review.openstack.org/290177 Thanks, -amrith > -----Original Message----- > From: Matt Riedemann [mailto:[email protected]] > Sent: Wednesday, March 09, 2016 2:53 PM > To: [email protected] > Subject: Re: [openstack-dev] [tc] [trove][stable] proboscis tests randomly > failing in stable branches; bug 1538506 > > > > On 3/9/2016 1:34 PM, Amrith Kumar wrote: > > Matt, > > > > We discussed this at the Trove meeting. Here's my current understanding > of the situation: > > > > 1. Your change https://review.openstack.org/#/c/290048/ which reverts > the trove client side of the change should be merged. > > > > 2. This change (the Trove API side) > https://review.openstack.org/#/c/245845/ should also be reverted. I'm > assuming that you'll submit a change for this as well for completeness. > > I can submit a change for this. > > > > > 3. We need to blacklist python-troveclient 2.1.0 on Liberty, this is > > your change https://review.openstack.org/#/c/290021/ > > > > 4. We need to blacklist python-troveclient 2.1.0 on master, this is > currently *NOT* what your change https://review.openstack.org/290168 does. > I disagree with the approach of just increasing the minimum requirement > from 1.2.0 to >=2.1.0. > > I'm OK with blacklisting 2.1.0 on master and can make that change in the > same patch. > > > > > 5. We need to request a new python-troveclient. Whether it should be > called 2.1.1 or 2.2.0, I'm not positive. My thought is 2.1.1. But out of > an abundance of caution, I'm going to look to #openstack- > release/#openstack-infra for guidance on this. > > There are no other changes to python-troveclient since 2.1.0 was released: > > user@xubuntu:~/git/python-troveclient$ git log --oneline --no-merges > 2.1.0.. > user@xubuntu:~/git/python-troveclient$ > > > So 2.1.1 is fine. > > > > > Thanks, > > > > -amrith > > > >> -----Original Message----- > >> From: Matt Riedemann [mailto:[email protected]] > >> Sent: Wednesday, March 09, 2016 12:42 PM > >> To: [email protected] > >> Subject: Re: [openstack-dev] [trove][stable] proboscis tests randomly > >> failing in stable branches; bug 1538506 > >> > >> > >> > >> On 3/9/2016 8:54 AM, Amrith Kumar wrote: > >>> Matt, > >>> > >>> As I understand it, you have 4 changes up for review. > >>> > >>> Change [1] seeks to revert the change [6] to python-troveclient > on > >>> master and reinstate the slave_of parameter. > >>> > >>> Change [2] is doing for stable/liberty, the same things that > were > >>> done for master in [9] on master, and [7] on Kilo earlier. > >>> > >>> Change [3] is looking to blacklist python-troveclient 2.1.0 on > >>> stable/liberty. > >>> > >>> Change [4] is looking to bump the minimum python-troveclient > >> version > >>> on master to 2.1.0. > >> > >> Right, and there was actually another for that beat me to this: > >> > >> https://review.openstack.org/#/c/290115/ > >> > >>> > >>> Here are three questions I have: > >>> > >>> (1) Wouldn't backward compatibility also require that you revert the > >>> other change that removed slave_of on the server [5] ? After all, > >>> just like a python client that calls create() with slave_of, a REST > >>> client could call the endpoint with slave_of. What is the policy for > >>> REST API backward compatibility? > >> > >> I guess it depends on the Trove team. In Nova, backward compatibility > >> in the API is serious business and that's why we have left all sorts > >> of warts in the v2.0 API and couldn't remove them. But with the v2.1 > >> API and microversions, we're able to move the API forward (Ironic > >> also has microversions, and I think cinder/neutron/keystone are > >> working on adding that support). > >> > >> So if maintaining backward compatibility in the trove API is > >> important to the trove team and it's users, then yes the API change > >> should probably be reverted. For anyone doing CD with Trove, they are > >> already broken, but people upgrading from liberty to mitaka could save > themselves the break. > >> > >>> > >>> (2) Wouldn't [4] just block 2.1.0 in global-requirements on master > >>> (mitaka)? > >> > >> I'm not sure I understand, [4] here bumps the minimum required > >> version of python-troveclient to be 2.1.0, not block it. As noted in > >> the review, I don't know that it's really necessary to bump to 2.1.0 > >> iff we land and release [1]. > >> > >>> > >>> (3) Something, potentially your patch set [2], should also fix the > >>> test that is invoking create() with slave_of, no? > >> > >> [2] is stable/liberty which still has slave_of in the create API, so > >> I don't think that needs to be fixed in stable/liberty. > >> > >>> > >>> -amrith > >>> > >>> [1] https://review.openstack.org/290048/ > >>> [2] https://review.openstack.org/290023/ > >>> [3] https://review.openstack.org/290021/ > >>> [4] https://review.openstack.org/290168/ > >>> [5] https://review.openstack.org/#/c/245845/ > >>> [6] https://review.openstack.org/#/c/245738/ > >>> [7] https://review.openstack.org/#/c/246735/ > >>> > >>> On 03/08/2016 08:24 PM, Matt Riedemann wrote: > >>>> > >>>> > >>>> On 3/8/2016 4:44 PM, Matt Riedemann wrote: > >>>>> > >>>>> > >>>>> On 3/8/2016 3:17 PM, Matt Riedemann wrote: > >>>>>> > >>>>>> > >>>>>> On 3/8/2016 1:18 PM, Amrith Kumar wrote: > >>>>>>> Matt, > >>>>>>> > >>>>>>> See inline below. > >>>>>>> > >>>>>>> -amrith > >>>>>>> > >>>>>>>> -----Original Message----- > >>>>>>>> From: Matt Riedemann [mailto:[email protected]] > >>>>>>>> Sent: Tuesday, March 08, 2016 2:00 PM > >>>>>>>> To: [email protected] > >>>>>>>> Subject: Re: [openstack-dev] [trove][stable] proboscis tests > >>>>>>>> randomly failing in stable branches; bug 1538506 > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On 3/8/2016 12:52 PM, Matt Riedemann wrote: > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On 3/8/2016 12:35 PM, Amrith Kumar wrote: > >>>>>>>>>> Matt, > >>>>>>>>>> > >>>>>>>>>> The correct solution for liberty is that we should fix the > tests. > >>>>>>>>>> Here's why I believe that this is the case. > >>>>>>>>>> > >>>>>>>>>> In pertinent part, the backtrace from your bug includes: > >>>>>>>>>> > >>>>>>>>>> 2016-01-27 07:02:06.777 | File > >>>>>>>>>> "/home/jenkins/workspace/periodic-trove-python27-liberty/trov > >>>>>>>>>> e/ > >>>>>>>>>> tests/ > >>>>>>>>>> > >>>>>>>>>> api/replication.py", > >>>>>>>>>> line 83, in create_slave > >>>>>>>>>> 2016-01-27 07:02:06.777 | slave_of=instance_info.id) > >>>>>>>>>> > >>>>>>>>>> This is in the tests, not the client. > >>>>>>>>>> > >>>>>>>>>> The test is now generating a parameter that the client no > >>>>>>>>>> longer understands. > >>>>>>>>>> > >>>>>>>>>> For a user, here are the various situations that could occur. > >>>>>>>>>> > >>>>>>>>>> 1. User running python-troveclient from the latest 2.1.0 and > >>>>>>>>>> a server from liberty. All is good. > >>>>>>>>>> 2. User running an old python-troveclient and a server from > >>>>>>>>>> liberty. > >>>>>>>>>> All is good. > >>>>>>>>> > >>>>>>>>> Will this work with python-troveclient 1.2.0 which is the > >>>>>>>>> minimum required troveclient for stable/liberty? > >>>>>>>>> > >>>>>>>>> https://github.com/openstack/requirements/blob/stable/liberty/ > >>>>>>>>> gl > >>>>>>>>> obal-r > >>>>>>>>> > >>>>>>>>> equirements.txt#L174 > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> 3. User running an old python-troveclient and a new server > >>>>>>>>>> from mitaka, this is not supported. > >>>>>>>>> > >>>>>>>>> How is this not supported? If it's not supported, the minimum > >>>>>>>>> required version of troveclient in master g-r is wrong, since > >>>>>>>>> it hasn't changed since liberty: > >>>>>>>>> > >>>>>>>>> https://github.com/openstack/requirements/blob/master/global-r > >>>>>>>>> eq > >>>>>>>>> uireme > >>>>>>>>> > >>>>>>>>> nts.txt#L202 > >>>>>>>> > >>>>>>>> I've confirmed that running master (mitaka) unit tests for > >>>>>>>> trove against python-troveclient 1.2.0 don't work: > >>>>>>>> > >>>>>>>> http://paste.openstack.org/show/489719/ > >>>>>>> > >>>>>>> [amrith] Just like the bug you listed, this appears to be a case > >>>>>>> of a broken test. Would you please LP it. > >>>>>>> > >>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>>> 4. User running a current python-troveclient from the latest > >>>>>>>>>> 2.1.0 and a server from mitaka, All is good. > >>>>>>>>>> > >>>>>>>>>> What you have here is a case where a test is generating an > >>>>>>>>>> argument > >>>>>>>>>> (slave_of) that the client (master, 2.1.0) no longer > understands. > >>>>>>>>>> There's nothing amiss there, except that the test needs to be > >>>>>>>>>> fixed. > >>>>>>>>>> > >>>>>>>>>> When you get back, let's continue the conversation either in > >>>>>>>>>> email or IRC #openstack-dev. Looking forward to hearing your > >>>>>>>>>> feedback on this. > >>>>>>>>>> > >>>>>>>>>> Thanks, > >>>>>>>>>> > >>>>>>>>>> -amrith > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>>> -----Original Message----- > >>>>>>>>>>> From: Matt Riedemann [mailto:[email protected]] > >>>>>>>>>>> Sent: Tuesday, March 08, 2016 12:11 PM > >>>>>>>>>>> To: [email protected] > >>>>>>>>>>> Subject: Re: [openstack-dev] [trove][stable] proboscis tests > >>>>>>>>>>> randomly failing in stable branches; bug 1538506 > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> On 3/8/2016 10:17 AM, Matt Riedemann wrote: > >>>>>>>>>>>> This is a call for help on resolving bug 1538506 [1] where > >>>>>>>>>>>> the proboscis tests randomly fail on the stable branches > >>>>>>>>>>>> with something > >>>>>>>>>>> like: > >>>>>>>>>>>> > >>>>>>>>>>>> TypeError: create() got an unexpected keyword argument > >> 'slave_of' > >>>>>>>>>>>> > >>>>>>>>>>>> Craig Vyvial has a proposed stable/kilo change [2] but it > >>>>>>>>>>>> has some issues, at least from me as a reviewer of that > change. > >>>>>>>>>>>> > >>>>>>>>>>>> The tests are failing the periodic-stable job daily [3]. > >>>>>>>>>>>> > >>>>>>>>>>>> Can we get someone to help out with this? Otherwise I'm > >>>>>>>>>>>> inclined to say the tests should be disabled on the stable > >>>>>>>>>>>> branches, but that's pretty drastic. > >>>>>>>>>>>> > >>>>>>>>>>>> Note that this is the kind of thing that breaks stable > >>>>>>>>>>>> branch policy, which is going to be part of a review when > >>>>>>>>>>>> applying for the stable:follows-policy tag. [4] And the > >>>>>>>>>>>> stable:follows-policy tag may be used to determine when a > >>>>>>>>>>>> stable branch for a project goes end of life if it's not > >>>>>>>>>>>> being maintained. > >>>>>>>>>>>> > >>>>>>>>>>>> [1] https://bugs.launchpad.net/trove/+bug/1538506 > >>>>>>>>>>>> [2] https://review.openstack.org/#/c/276934/ > >>>>>>>>>>>> [3] http://goo.gl/fqf11U > >>>>>>>>>>>> [4] > >>>>>>>>>>>> http://governance.openstack.org/reference/tags/stable_follo > >>>>>>>>>>>> ws > >>>>>>>>>>>> -polic > >>>>>>>>>>>> > >>>>>>>>>>>> y.h > >>>>>>>>>>>> tml > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> This is my solution for liberty, cap > >>>>>>>>>>> python-troveclient<2.1.0 in > >>>>>>>>>>> global- requirements on stable/liberty [1]. > >>>>>>>>>>> > >>>>>>>>>>> Then there is a change to trove on stable/liberty to use the > >>>>>>>>>>> g-r version range for python-troveclient for unit tests [2]. > >>>>>>>>>>> > >>>>>>>>>>> Alternatively, we could avoid the cap in stable/liberty by: > >>>>>>>>>>> > >>>>>>>>>>> 1. Reverting https://review.openstack.org/#/c/245738/ > >>>>>>>>>>> 2. Blacklisting python-troveclient 2.1.0 in g-r 3. Release > >>>>>>>>>>> python- troveclient 2.2.0. > >>>>>>>>>>> > >>>>>>>>>>> It's getting late in the mitaka release to be messing with > >>>>>>>>>>> this though since we're already past client freeze. > >>>>>>>>>>> > >>>>>>>>>>> [1] https://review.openstack.org/#/c/290021/ > >>>>>>>>>>> [2] https://review.openstack.org/#/c/290023/ > >>>>>>>>>>> > >>>>>>>>>>> -- > >>>>>>>>>>> > >>>>>>>>>>> Thanks, > >>>>>>>>>>> > >>>>>>>>>>> Matt Riedemann > >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> ____________________________________________________________ > >>>>>>>>>>> __ > >>>>>>>>>>> ______ > >>>>>>>>>>> > >>>>>>>>>>> ______ > >>>>>>>>>>> > >>>>>>>>>>> OpenStack Development Mailing List (not for usage questions) > >>>>>>>>>>> Unsubscribe: > >>>>>>>>>>> [email protected]?subject:unsubscrib > >>>>>>>>>>> e > >>>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac > >>>>>>>>>>> k- > >>>>>>>>>>> dev > >>>>>>>>>> > >>>>>>>>>> _____________________________________________________________ > >>>>>>>>>> __ > >>>>>>>>>> ______ > >>>>>>>>>> > >>>>>>>>>> _____ > >>>>>>>>>> > >>>>>>>>>> OpenStack Development Mailing List (not for usage questions) > >>>>>>>>>> Unsubscribe: > >>>>>>>>>> [email protected]?subject:unsubscribe > >>>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack > >>>>>>>>>> -d > >>>>>>>>>> ev > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> > >>>>>>>> Thanks, > >>>>>>>> > >>>>>>>> Matt Riedemann > >>>>>>>> > >>>>>>>> > >>>>>>>> _______________________________________________________________ > >>>>>>>> __ > >>>>>>>> _________ > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> OpenStack Development Mailing List (not for usage questions) > >>>>>>>> Unsubscribe: > >>>>>>>> [email protected]?subject:unsubscribe > >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-d > >>>>>>>> ev > >>>>>>> > >>>>>>> ________________________________________________________________ > >>>>>>> __ > >>>>>>> ________ > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> OpenStack Development Mailing List (not for usage questions) > >>>>>>> Unsubscribe: > >>>>>>> [email protected]?subject:unsubscribe > >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-de > >>>>>>> v > >>>>>>> > >>>>>> > >>>>>> Well, the point I'm trying to get to is, I don't think > >>>>>> troveclient < > >>>>>> 2.1.0 will work with the trove-api in mitaka. > >>>>>> > >>>>>> For example, in my change to revert the slave_of removal in the > >> client: > >>>>>> > >>>>>> https://review.openstack.org/#/c/290048/ > >>>>>> > >>>>>> That fails in the API: > >>>>>> > >>>>>> http://logs.openstack.org/48/290048/2/check/gate-trove-functional > >>>>>> -d > >>>>>> svm-mysql/ddd3089/logs/screen-tr-api.txt.gz#_2016-03-08_19_30_32_ > >>>>>> 79 > >>>>>> 0 > >>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> "Validation error: instance Additional properties are not allowed > >>>>>> (u'slave_of' was unexpected)" > >>>>>> > >>>>>> So if my client application that I wrote in liberty was using > >>>>>> slave_of, it's happily working until the cloud that my > >>>>>> application is running on upgrades to mitaka, then things break. > >>>>>> > >>>>>> Part of the issue here is the backward incompatible change in the > >>>>>> trove API. The other part is, the trove API in mitaka requires > >>>>>> troveclient>=2.1.0 because that's what removed slave_of. > >>>>>> > >>>>>> Right? > >>>>>> > >>>>>> I'm trying to sort out what a valid troveclient is for master, > >>>>>> because I don't really trust what's in global-requirements since > >>>>>> trove hasn't been testing with those versions, at least not at > >>>>>> the lower bound of 1.2.0. > >>>>>> > >>>>> > >>>>> Here is a change that bumps the minimum required version of > >>>>> python-troveclient for mitaka to 2.1.0: > >>>>> > >>>>> https://review.openstack.org/#/c/290168/ > >>>>> > >>>> > >>>> Here is my preferred option now. > >>>> > >>>> 1. Revert the slave_of removal from python-troveclient and provide > >>>> it as a compat shim until liberty-eol: > >>>> > >>>> https://review.openstack.org/#/c/290048/ > >>>> > >>>> If slave_of is used, a deprecation warning is emitted (which we > >>>> should have been doing when this was deprecated in the first place). > >>>> It also never sends slave_of to the API, it's just a proxy to > >>>> replica_of. So this should work with both the liberty and mitaka > >>>> versions of the trove API. > >>>> > >>>> 2. Block python-troveclient 2.1.0 from stable/liberty g-r: > >>>> > >>>> https://review.openstack.org/#/c/290021/ > >>>> > >>>> That blocks the bad 2.1.0 version from being used in stable/liberty > >>>> which is needed for... > >>>> > >>>> 3. Using the g-r versions of python-troveclient when testing on > >>>> stable/liberty: > >>>> > >>>> https://review.openstack.org/#/c/290023/ > >>>> > >>>> If trove is following global-requirements, it should be using the > >>>> versions of python-troveclient from global-requirements rather than > >>>> installing it from the latest tarball on trunk. master and > >>>> stable/kilo are already doing this. > >>>> > >>> > >>> > >>> > >>> > >>> ____________________________________________________________________ > >>> __ ____ OpenStack Development Mailing List (not for usage questions) > >>> Unsubscribe: > >>> [email protected]?subject:unsubscribe > >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >>> > >> > >> -- > >> > >> Thanks, > >> > >> Matt Riedemann > >> > >> > >> _____________________________________________________________________ > >> _____ OpenStack Development Mailing List (not for usage questions) > >> Unsubscribe: > >> [email protected]?subject:unsubscribe > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ______________________________________________________________________ > > ____ OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > > [email protected]?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > > Thanks, > > Matt Riedemann > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
