Re: [Openstack-operators] [openstack-dev] [qa][tempest-plugins][release][tc][ptl]: Coordinated Release Model proposal for Tempest & Tempest Plugins
On Wed, 27 Jun 2018 10:19:17 +0900 Ghanshyam Mann wrote > ++ operator ML > > On Wed, 27 Jun 2018 10:17:33 +0900 Ghanshyam Mann > wrote > > > > > > > > On Tue, 26 Jun 2018 23:12:30 +0900 Doug Hellmann > wrote > > > Excerpts from Matthew Treinish's message of 2018-06-26 09:52:09 -0400: > > > > On Tue, Jun 26, 2018 at 08:53:21AM -0400, Doug Hellmann wrote: > > > > > Excerpts from Andrea Frittoli's message of 2018-06-26 13:35:11 > +0100: > > > > > > On Tue, 26 Jun 2018, 1:08 pm Thierry Carrez, > wrote: > > > > > > > > > > > > > Dmitry Tantsur wrote: > > > > > > > > [...] > > > > > > > > My suggestion: tempest has to be compatible with all > supported releases > > > > > > > > (of both services and plugins) OR be branched. > > > > > > > > [...] > > > > > > > I tend to agree with Dmitry... We have a model for things that > need > > > > > > > release alignment, and that's the cycle-bound series. The > reason tempest > > > > > > > is branchless was because there was no compatibility issue. If > the split > > > > > > > of tempest plugins introduces a potential incompatibility, > then I would > > > > > > > prefer aligning tempest to the existing model rather than > introduce a > > > > > > > parallel tempest-specific cycle just so that tempest can stay > > > > > > > release-independent... > > > > > > > > > > > > > > I seem to remember there were drawbacks in branching tempest, > though... > > > > > > > Can someone with functioning memory brain cells summarize them > again ? > > > > > > > > > > > > > > > > > > > > > > > > > Branchless Tempest enforces api stability across branches. > > > > > > > > > > I'm sorry, but I'm having a hard time taking this statement > seriously > > > > > when the current source of tension is that the Tempest API itself > > > > > is breaking for its plugins. > > > > > > > > > > Maybe rather than talking about how to release compatible things > > > > > together, we should go back and talk about why Tempest's API is > changing > > > > > in a way that can't be made backwards-compatible. Can you give > some more > > > > > detail about that? > > > > > > > > > > > > > Well it's not, if it did that would violate all the stability > guarantees > > > > provided by Tempest's library and plugin interface. I've not ever > heard of > > > > these kind of backwards incompatibilities in those interfaces and we > go to > > > > all effort to make sure we don't break them. Where did the idea that > > > > backwards incompatible changes where being introduced come from? > > > > > > In his original post, gmann said, "There might be some changes in > > > Tempest which might not work with older version of Tempest Plugins." > > > I was surprised to hear that, but I'm not sure how else to interpret > > > that statement. > > > > I did not mean to say that Tempest will introduce the changes in backward > incompatible way which can break plugins. That cannot happen as all plugins > and tempest are branchless and they are being tested with master Tempest so > if we change anything backward incompatible then it break the plugins gate. > Even we have to remove any deprecated interfaces from Tempest, we fix all > plugins first like - > https://review.openstack.org/#/q/topic:remove-support-of-cinder-v1-api+(status:open+OR+status:merged) > > > > > What I mean to say here is that adding new or removing deprecated > interface in Tempest might not work with all released version or unreleased > Plugins. That point is from point of view of using Tempest and Plugins in > production cloud testing not gate(where we keep the compatibility). > Production Cloud user use Tempest cycle based version. Pike based Cloud will > be tested by Tempest 17.0.0 not latest version (though latest version might > work). > > > > This thread is not just for gate testing point of view (which seems to be > always interpreted), this is more for user using Tempest and Plugins for > their cloud testing. I am looping operator mail list also which i forgot in > initial post. > > > > We do not have any tag/release from plugins to know what version of > plugin can work with what version of tempest. For Example If There is new > interface introduced by Tempest 19.0.0 and pluginX start using it. Now it > can create issues for pluginX in both release model 1. plugins with no > release (I will call this PluginNR), 2. plugins with independent release (I > will call it PluginIR). > > > > Users (Not Gate) will face below issues: > > - User cannot use PluginNR with Tempest <19.0.0 (where that new interface > was not present). And there is no PluginNR release/tag as this is unreleased > and not branched
Re: [Openstack-operators] [openstack-dev] [qa][tempest-plugins][release][tc][ptl]: Coordinated Release Model proposal for Tempest & Tempest Plugins
++ operator ML On Wed, 27 Jun 2018 10:17:33 +0900 Ghanshyam Mann wrote > > > > On Tue, 26 Jun 2018 23:12:30 +0900 Doug Hellmann > wrote > > Excerpts from Matthew Treinish's message of 2018-06-26 09:52:09 -0400: > > > On Tue, Jun 26, 2018 at 08:53:21AM -0400, Doug Hellmann wrote: > > > > Excerpts from Andrea Frittoli's message of 2018-06-26 13:35:11 +0100: > > > > > On Tue, 26 Jun 2018, 1:08 pm Thierry Carrez, > wrote: > > > > > > > > > > > Dmitry Tantsur wrote: > > > > > > > [...] > > > > > > > My suggestion: tempest has to be compatible with all supported > releases > > > > > > > (of both services and plugins) OR be branched. > > > > > > > [...] > > > > > > I tend to agree with Dmitry... We have a model for things that > need > > > > > > release alignment, and that's the cycle-bound series. The reason > tempest > > > > > > is branchless was because there was no compatibility issue. If > the split > > > > > > of tempest plugins introduces a potential incompatibility, then I > would > > > > > > prefer aligning tempest to the existing model rather than > introduce a > > > > > > parallel tempest-specific cycle just so that tempest can stay > > > > > > release-independent... > > > > > > > > > > > > I seem to remember there were drawbacks in branching tempest, > though... > > > > > > Can someone with functioning memory brain cells summarize them > again ? > > > > > > > > > > > > > > > > > > > > > Branchless Tempest enforces api stability across branches. > > > > > > > > I'm sorry, but I'm having a hard time taking this statement seriously > > > > when the current source of tension is that the Tempest API itself > > > > is breaking for its plugins. > > > > > > > > Maybe rather than talking about how to release compatible things > > > > together, we should go back and talk about why Tempest's API is > changing > > > > in a way that can't be made backwards-compatible. Can you give some > more > > > > detail about that? > > > > > > > > > > Well it's not, if it did that would violate all the stability > guarantees > > > provided by Tempest's library and plugin interface. I've not ever heard > of > > > these kind of backwards incompatibilities in those interfaces and we go > to > > > all effort to make sure we don't break them. Where did the idea that > > > backwards incompatible changes where being introduced come from? > > > > In his original post, gmann said, "There might be some changes in > > Tempest which might not work with older version of Tempest Plugins." > > I was surprised to hear that, but I'm not sure how else to interpret > > that statement. > > I did not mean to say that Tempest will introduce the changes in backward > incompatible way which can break plugins. That cannot happen as all plugins > and tempest are branchless and they are being tested with master Tempest so > if we change anything backward incompatible then it break the plugins gate. > Even we have to remove any deprecated interfaces from Tempest, we fix all > plugins first like - > https://review.openstack.org/#/q/topic:remove-support-of-cinder-v1-api+(status:open+OR+status:merged) > > > What I mean to say here is that adding new or removing deprecated interface > in Tempest might not work with all released version or unreleased Plugins. > That point is from point of view of using Tempest and Plugins in production > cloud testing not gate(where we keep the compatibility). Production Cloud > user use Tempest cycle based version. Pike based Cloud will be tested by > Tempest 17.0.0 not latest version (though latest version might work). > > This thread is not just for gate testing point of view (which seems to be > always interpreted), this is more for user using Tempest and Plugins for > their cloud testing. I am looping operator mail list also which i forgot in > initial post. > > We do not have any tag/release from plugins to know what version of plugin > can work with what version of tempest. For Example If There is new interface > introduced by Tempest 19.0.0 and pluginX start using it. Now it can create > issues for pluginX in both release model 1. plugins with no release (I will > call this PluginNR), 2. plugins with independent release (I will call it > PluginIR). > > Users (Not Gate) will face below issues: > - User cannot use PluginNR with Tempest <19.0.0 (where that new interface > was not present). And there is no PluginNR release/tag as this is unreleased > and not branched software. > - User cannot find a PluginIR particular tag/release which can work with > tempest <19.0.0 (where that new interface was not present). Only way for > user to make it work is to manually find out the PluginIR tag/commit before > PluginIR started consuming
