On Fri, Apr 21, 2017 at 4:19 AM, MONTEIRO, FELIPE C <[email protected]> wrote: > Thanks Ghanshyam, > > That was really helpful. If Tempest is versioned via tags, then I agree that > Patrole should do the same thing. I didn't quite know what you meant by > "feature flag" but after finding [0] it became apparent to me: Patrole has > been doing this wherever Tempest does it. Granted, we probably need more > participation in the project to catch minute details like that, as we > currently have a large body of tests, but currently we have been using > feature flags where appropriate.
Very Nice. Feel free to ping us if you need any help on those things. > > Case 3 as you mentioned is something that we've already had to account for: > When Nova and then Keystone moved their policies into code, we had to add > logic to the framework to account for those changes. Case 3 in theory would > not be a problem, if there was a way to verify whether a policy action was > enforced, other than by checking logs. If there was an API in OpenStack that > could be called to retrieve oslo_policy enforcements per request id, then > Case 3 would just result in an error being thrown by Patrole for a given > test, which we would then notice and fix. Yea that's true and it was due to lack of policy documentation. Nova will (almost completed) have good documentation to solve that. John & OSIC folks started very nice documentation on policy which tells you what all policy being used by what all APIs with description [1]. And on same page Keystone also [2] This is how it looks like now [3]. This can help Patrole. ..1 https://blueprints.launchpad.net/nova/+spec/policy-docs ..2 https://blueprints.launchpad.net/keystone/+spec/policy-docs ..3 https://docs.openstack.org/developer/nova/sample_policy.html > > [0] https://review.openstack.org/#/c/418928/ > > --Felipe > > -----Original Message----- > From: Ghanshyam Mann [mailto:[email protected]] > Sent: Wednesday, April 19, 2017 12:05 AM > To: OpenStack Development Mailing List (not for usage questions) > <[email protected]> > Subject: Re: [openstack-dev] [qa] [patrole] Question regarding patrole > release management > > On Wed, Apr 19, 2017 at 2:29 AM, MONTEIRO, FELIPE C <[email protected]> wrote: >> Hi all, >> >> >> >> I have a question regarding patrole and release management. Many projects >> like heat or murano have a tempest plugin within their repos, so by >> extension their tempest plugins have releases, as the projects change over >> time. However, since patrole is just a tempest plugin, yet heavily reliant >> on tempest, how should patrole do release management? > > Hi Felipe, > > Release management depends on whether Patrole is planning the > branchless model (i think it is) like Tempest or branch model., If > branchless, then it does not fall under release management. It can > adopt release model what Tempest has [1]. > > Branchless model gives many benefit like avoid backward incompatible > changes, avoid maintaining multiple Patrole repo across releases etc. > >> Am I correct in >> thinking that it should, in the first place? With nova-network and other >> APIs slated for deprecation in Pike and beyond, Patrole will logically have >> to continuously be maintained to keep up, meaning that older tests, just >> like with Tempest, will have to be phased out. If Patrole, then, does not >> have releases, then older release-dependent tests and functionality will >> over time be lost. > > We have 3 cases here: > 1. Functionality is deprecated/removed in new release. > 2. Functionality newly added in new release. > 3. Policy enforcement change. > > For case 1, Tempest keep testing deprecated functionality till it is > marked deprecated across all supported stable branches. Once all > stable branch has that functionality as deprecated marked, then we > discuss of removing its testing from Tempest. > With API Microversion model that is little bit different where > functionality might be deprecated after specific version or it is > deprecated from base version itself. For example, Nova proxy APIs > deprecated after 2.36, Certificate APIs might be gone from base > version (which is under discussion). This is more case by case and > based on all stack holder point of view, we will decide their testing > should be removed or stay till when. > > For case 2, Tempest introduced testing of new functionality with > feature flag and those tests will be executed/skipped accordingly. > > Case 3 is something important to consider for Patrole. Usually policy > changes will be done with backward compatible way where changes does > not break upgrade. Any change in policy enforcement will be done at > least with supporting old and new rules [2] or with old rules > deprecation of period of 1 release cycle at least. And branchless > model can detect any accidental changes which going to break upgrade. > > IMO, branchless model is good value for Patrole in all 3 cases > consideration and feature flags to handle the new/old/policy-change > functionality. Similarly release model can be same as Tempest has. > > >> >> >> >> Thank You, >> >> Felipe Monteiro > > ..1 > https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.openstack.org_wiki_QA_releases&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=xgobDX0RcKxlQhqhOSmU6H11-k8u21HsinBnbYLB6mY&s=9eq8-o16PTMYfNYW0LjlVyRfN1W4wXWloI7jVij5W4g&e= > ..2 > https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_-23_c_391113_13&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=xgobDX0RcKxlQhqhOSmU6H11-k8u21HsinBnbYLB6mY&s=A5axQVIlZlVMGaW0vwQxpqwx5uMZvZ1ZNwWAXgrIoAc&e= > > -gmann > > >> >> >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: [email protected]?subject:unsubscribe >> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=xgobDX0RcKxlQhqhOSmU6H11-k8u21HsinBnbYLB6mY&s=DKOquEt2EsEYictxTecUKp5fwVaWZ3kNoFdm_tYFSsc&e= >> > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev&d=DwIGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=X4GwEru-SJ9DRnCxhze-aw&m=xgobDX0RcKxlQhqhOSmU6H11-k8u21HsinBnbYLB6mY&s=DKOquEt2EsEYictxTecUKp5fwVaWZ3kNoFdm_tYFSsc&e= > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
