Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Brian Rosmaita]

Big snip ... the original message is here:
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113798.html

On 3/13/17 3:09 AM, Ghanshyam Mann wrote:
> On Sat, Mar 11, 2017 at 12:10 AM, Andrea Frittoli > wrote:
>> On Fri, Mar 10, 2017 at 2:49 PM Andrea Frittoli 
>> wrote:
>> Of course Glance v1 still has to run on the stable/newton gate jobs, until
>> Newton EOL (TBD), so tests will stay in Tempest for a cycle more at least.
>> I guess I shouldn't be sending emails on a Friday afternoon?
>>
> 
> ​humm, Till Mitaka right? Newton version of glance is with v1 API as
> deprecated. And Mitaka is v1 as supported APIs so we only need to care
> about Mitaka where v1 APIs were supported and keep tests till Mitaka EOL.

(Sorry for the lateness of my reply, I was under the weather earlier
this week.)

Glance v1 is a special case, because even though it's DEPRECATED, out of
courtesy to operators, we won't remove it until some specific features
(primarily, image import with copy-from) are available in v2.  We'd
originally thought that this feature was unnecessary, but heard
differently from operators once v1 went into deprecation.  Thus, through
the Ocata release, there are valid reasons for operators to deploy v1
even though it's deprecated.  We didn't un-deprecate v1 because we
wanted to send a clear message that v1 is definitely going away.

So, it's important that v1 continue to be tested in the gate.

cheers,
brian


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Ken'ichi Ohmichi]

2017-03-10 8:51 GMT-08:00 Sean McGinnis :
>> >
>> As far as I can tell:
>> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
>> deprecated in all supported releases.
>> - Glance v1 has been deprecated in Newton, so it's deprecated in all
>> supported releases
>> - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
>> Tempest until Mitaka EOL, which is in a month from now
>>
>> We should stop testing these three api versions in the common gate
>> including stable branches now (except for keystone v2 on stable/mitaka
>> which can run for one more month).
>>
>> Are cinder / glance / keystone willing to take over the API tests and run
>> them in their own gate until removal of the API version?
>>
>> Doug
>
> With Cinder's v1 API being deprecated for quite awhile now, I would
> actually prefer to just remove all tempest tests and drop the API
> completely. There was some concern about removal a few cycles back since
> there was concern (rightly so) that a lot of deployments and a lot of
> users were still using it.
>
> I think it has now been marked as deprecated long enough that if anyone
> is still using it, it's just out of obstinance. We've removed the v1
> api-ref documentation, and the default in the client has been v2 for
> awhile.
>
> Unless there's a strong objection, and a valid argument to support it,
> I really would just like to drop v1 from Cinder and not waste any more
> cycles on redoing tempest tests and reconfiguring jobs to support
> something we have stated for over two years that we were no longer going
> to support. Juno went EOL in December of 2015. I really hope it's safe
> now to remove.

OK, let's remove the Cinder v1 API tests from Tempest [1].

Thanks

---
[1]: https://review.openstack.org/#/c/446233/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Ghanshyam Mann]

On Sat, Mar 11, 2017 at 12:10 AM, Andrea Frittoli  wrote:

>
>
> On Fri, Mar 10, 2017 at 2:49 PM Andrea Frittoli 
> wrote:
>
>> On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann 
>> wrote:
>>
>> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
>> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad 
>> wrote:
>> >
>> > >
>> > >
>> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
>> > > wrote:
>> > >
>> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
>> > >> > Hi folks,
>> > >> >
>> > >> > I'm trying to figure out what's the best approach to fade out
>> testing of
>> > >> > deprecated API versions.
>> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
>> API
>> > >> v2
>> > >> > and Cinder API v1.
>> > >> >
>> > >> > According to the guidelines for the "follow-standard-deprecation"
>> tag
>> > >> [0],
>> > >> > when projects that have that tag deprecate a feature:
>> > >> >
>> > >> > "Code will be frozen and only receive minimal maintenance (just so
>> that
>> > >> it
>> > >> > continues to work as-is)."
>> > >> >
>> > >> > I interpret this so that projects should maintain some level of
>> testing
>> > >> of
>> > >> > the deprecated feature, including a deprecated API version.
>> > >> > The QA team does not see value in testing deprecated API versions
>> in the
>> > >> > common gate jobs, so my question is what do to with those tests.
>> > >> >
>> > >> > One option is to maintain them in Tempest until the API version is
>> > >> removed,
>> > >> > and run them in dedicated project jobs.
>> > >> > This means that tempest would have to run those jobs as well, so
>> three
>> > >> > extra jobs, until the API version is removed.
>> > >> >
>> > >> > The other option is to move those tests out of Tempest, into the
>> > >> projects.
>> > >> > This would imply back porting them to all relevant branches as
>> well,
>> > >> but it
>> > >> > would have the advantage of decoupling them from Tempest. It
>> should be
>> > >> no
>> > >> > concern from an API stability POV since the code for that API will
>> be
>> > >> > frozen.
>> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
>> as
>> > >> far
>> > >> > as I can tell - removed or deprecated from interoperability
>> guidelines,
>> > >> so
>> > >> > moving the tests out of Tempest would not be an issue in that
>> sense.
>> > >> >
>> > >> > The 2nd option involves a bit more initial overhead for the
>> removal of
>> > >> > tests, but I think it would works for the best on the long term.
>> > >> >
>> > >> > There is a 3rd option as well, which is to stop running integration
>> > >> testing
>> > >> > on deprecated API versions before they are actually removed, but I
>> feel
>> > >> > that would not meet the criteria defined by the
>> > >> follow-standard-deprecation
>> > >> > tag.
>> > >> >
>> > >> > Thoughts?
>> > >> >
>> > >> > andrea
>> > >>
>> > >> Are any of those tests used by the interoperability working group
>> > >> (formerly DefCore)?
>> > >>
>> > >>
>> > > That's a good question. I was very curious about this because last I
>> > > checked keystone had v2.0 calls required for defcore. Looks like that
>> might
>> > > not be the case anymore [0]? I started a similar thread to this after
>> the
>> > > PTG since that was something our group talked about extensively
>> during the
>> > > deprecation session [1].
>> > >
>> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
>> > 2017.01.json [0]​. But there are some compute tests which internally use
>> > glance v1 API call [2]. But on mentioned flagged action- Nova already
>> moved
>> > to v2 APIs and tempest part is pending which can be fixed to make call
>> on
>> > v2 APIs only (which can be part of this work and quick).
>> >
>> > ​From options about deprecated APIs testing, I am with options 2 which
>> > really take out the load of Tempest tests maintenance and gate.   ​
>> >
>> > ​But another question is about stable branch testing of those API, like
>> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
>> > ​As Tempest is responsible of testing all stable branch behavior too,
>> Should
>> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
>> > state in all stable branch) ?​
>>
>> Excellent point.
>>
>>
>> As far as I can tell:
>> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
>> deprecated in all supported releases.
>> - Glance v1 has been deprecated in Newton, so it's deprecated in all
>> supported releases
>>
>
> Of course Glance v1 still has to run on the stable/newton gate jobs, until
> Newton EOL (TBD), so tests will stay in Tempest for a cycle more at least.
> I guess I shouldn't be sending emails on a Friday afternoon?
>

​humm, Till Mitaka right? Newton version of glance is with v1 API as
deprecated. And Mitaka is 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[John Griffith]

On Fri, Mar 10, 2017 at 9:51 AM, Sean McGinnis 
wrote:

> > >
> > As far as I can tell:
> > - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> > deprecated in all supported releases.
> > - Glance v1 has been deprecated in Newton, so it's deprecated in all
> > supported releases
> > - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> > Tempest until Mitaka EOL, which is in a month from now
> >
> > We should stop testing these three api versions in the common gate
> > including stable branches now (except for keystone v2 on stable/mitaka
> > which can run for one more month).
> >
> > Are cinder / glance / keystone willing to take over the API tests and run
> > them in their own gate until removal of the API version?
> >
> > Doug
>
> With Cinder's v1 API being deprecated for quite awhile now, I would
> actually prefer to just remove all tempest tests and drop the API
> completely. There was some concern about removal a few cycles back since
> there was concern (rightly so) that a lot of deployments and a lot of
> users were still using it.
>

​+1​

>
> I think it has now been marked as deprecated long enough that if anyone
> is still using it, it's just out of obstinance. We've removed the v1
> api-ref documentation, and the default in the client has been v2 for
> awhile.
>
> Unless there's a strong objection, and a valid argument to support it,
> I really would just like to drop v1 from Cinder and not waste any more
> cycles on redoing tempest tests and reconfiguring jobs to support
> something we have stated for over two years that we were no longer going
> to support. Juno went EOL in December of 2015. I really hope it's safe
> now to remove.
>
> Sean
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Andrea Frittoli]

On Fri, Mar 10, 2017 at 4:55 PM Sean McGinnis  wrote:

> > >
> > As far as I can tell:
> > - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> > deprecated in all supported releases.
> > - Glance v1 has been deprecated in Newton, so it's deprecated in all
> > supported releases
> > - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> > Tempest until Mitaka EOL, which is in a month from now
> >
> > We should stop testing these three api versions in the common gate
> > including stable branches now (except for keystone v2 on stable/mitaka
> > which can run for one more month).
> >
> > Are cinder / glance / keystone willing to take over the API tests and run
> > them in their own gate until removal of the API version?
> >
> > Doug
>
> With Cinder's v1 API being deprecated for quite awhile now, I would
> actually prefer to just remove all tempest tests and drop the API
> completely. There was some concern about removal a few cycles back since
> there was concern (rightly so) that a lot of deployments and a lot of
> users were still using it.
>
> I think it has now been marked as deprecated long enough that if anyone
> is still using it, it's just out of obstinance. We've removed the v1
> api-ref documentation, and the default in the client has been v2 for
> awhile.
>
> Unless there's a strong objection, and a valid argument to support it,
> I really would just like to drop v1 from Cinder and not waste any more
> cycles on redoing tempest tests and reconfiguring jobs to support
> something we have stated for over two years that we were no longer going
> to support. Juno went EOL in December of 2015. I really hope it's safe
> now to remove.
>

sounds like a good plan to me.


>
> Sean
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Sean McGinnis]

> >
> As far as I can tell:
> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> deprecated in all supported releases.
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
> - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> Tempest until Mitaka EOL, which is in a month from now
> 
> We should stop testing these three api versions in the common gate
> including stable branches now (except for keystone v2 on stable/mitaka
> which can run for one more month).
> 
> Are cinder / glance / keystone willing to take over the API tests and run
> them in their own gate until removal of the API version?
> 
> Doug

With Cinder's v1 API being deprecated for quite awhile now, I would
actually prefer to just remove all tempest tests and drop the API
completely. There was some concern about removal a few cycles back since
there was concern (rightly so) that a lot of deployments and a lot of
users were still using it.

I think it has now been marked as deprecated long enough that if anyone
is still using it, it's just out of obstinance. We've removed the v1
api-ref documentation, and the default in the client has been v2 for
awhile.

Unless there's a strong objection, and a valid argument to support it,
I really would just like to drop v1 from Cinder and not waste any more
cycles on redoing tempest tests and reconfiguring jobs to support
something we have stated for over two years that we were no longer going
to support. Juno went EOL in December of 2015. I really hope it's safe
now to remove.

Sean

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Andrea Frittoli]

On Fri, Mar 10, 2017 at 1:59 AM Ghanshyam Mann 
wrote:

On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad  wrote:



On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann  wrote:

Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> Hi folks,
>
> I'm trying to figure out what's the best approach to fade out testing of
> deprecated API versions.
> We currently host in Tempest API tests for Glance API v1, Keystone API v2
> and Cinder API v1.
>
> According to the guidelines for the "follow-standard-deprecation" tag [0],
> when projects that have that tag deprecate a feature:
>
> "Code will be frozen and only receive minimal maintenance (just so that it
> continues to work as-is)."
>
> I interpret this so that projects should maintain some level of testing of
> the deprecated feature, including a deprecated API version.
> The QA team does not see value in testing deprecated API versions in the
> common gate jobs, so my question is what do to with those tests.
>
> One option is to maintain them in Tempest until the API version is
removed,
> and run them in dedicated project jobs.
> This means that tempest would have to run those jobs as well, so three
> extra jobs, until the API version is removed.
>
> The other option is to move those tests out of Tempest, into the projects.
> This would imply back porting them to all relevant branches as well, but
it
> would have the advantage of decoupling them from Tempest. It should be no
> concern from an API stability POV since the code for that API will be
> frozen.
> Tests for deprecated APIs in cinder, keystone and glance are all - as far
> as I can tell - removed or deprecated from interoperability guidelines, so
> moving the tests out of Tempest would not be an issue in that sense.
>
> The 2nd option involves a bit more initial overhead for the removal of
> tests, but I think it would works for the best on the long term.
>
> There is a 3rd option as well, which is to stop running integration
testing
> on deprecated API versions before they are actually removed, but I feel
> that would not meet the criteria defined by the
follow-standard-deprecation
> tag.
>
> Thoughts?
>
> andrea

Are any of those tests used by the interoperability working group
(formerly DefCore)?


That's a good question. I was very curious about this because last I
checked keystone had v2.0 calls required for defcore. Looks like that might
not be the case anymore [0]? I started a similar thread to this after the
PTG since that was something our group talked about extensively during the
deprecation session [1].

​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
2017.01.json [0]​. But there are some compute tests which internally use
glance v1 API call [2]. But on mentioned flagged action- Nova already moved
to v2 APIs and tempest part is pending which can be fixed to make call on
v2 APIs only (which can be part of this work and quick).


I just checked all non-glance tests that invoke glance v1 in the gate right
now.
None of them is in the 2017.01 guideline [0], and all of them will run with
glance v2 as long as v1 is not marked as enabled.

FlavorsV2NegativeTest:test_boot_with_low_ram
ServerActionsTestJSON:test_create_backup
TestMinimumBasicScenario:test_minimum_basic_scenario
TestVolumeBootPattern:test_create_ebs_image_and_check_boot
VolumesV2ActionsTest:test_volume_upload

[0]
https://refstack.openstack.org/api/v1/guidelines/2017.01/tests?target=platform=required=true=false



​From options about deprecated APIs testing, I am with options 2 which
really take out the load of Tempest tests maintenance and gate.   ​

​But another question is about stable branch testing of those API, like
glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
​As Tempest is responsible of testing all stable branch behavior too, Should
 we keep testing them till all Mitaka EOL (till APIs are in deprecated
state in all stable branch) ?​





[0] https://github.com/openstack/defcore/blob/master/2017.01.json
[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113166.html

​[2] https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294
 ​



Doug

>
> [0]
>
https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Jeremy Stanley]

On 2017-03-10 14:49:34 + (+), Andrea Frittoli wrote:
[...]
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
[...]

Newton released _after_ Mitaka, so it's still supported for a while
to come.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Andrea Frittoli]

On Fri, Mar 10, 2017 at 3:12 PM Lance Bragstad  wrote:

> On Fri, Mar 10, 2017 at 8:49 AM, Andrea Frittoli <
> andrea.fritt...@gmail.com> wrote:
>
>
>
> On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann 
> wrote:
>
> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad 
> wrote:
> >
> > >
> > >
> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
> > > wrote:
> > >
> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> > >> > Hi folks,
> > >> >
> > >> > I'm trying to figure out what's the best approach to fade out
> testing of
> > >> > deprecated API versions.
> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
> API
> > >> v2
> > >> > and Cinder API v1.
> > >> >
> > >> > According to the guidelines for the "follow-standard-deprecation"
> tag
> > >> [0],
> > >> > when projects that have that tag deprecate a feature:
> > >> >
> > >> > "Code will be frozen and only receive minimal maintenance (just so
> that
> > >> it
> > >> > continues to work as-is)."
> > >> >
> > >> > I interpret this so that projects should maintain some level of
> testing
> > >> of
> > >> > the deprecated feature, including a deprecated API version.
> > >> > The QA team does not see value in testing deprecated API versions
> in the
> > >> > common gate jobs, so my question is what do to with those tests.
> > >> >
> > >> > One option is to maintain them in Tempest until the API version is
> > >> removed,
> > >> > and run them in dedicated project jobs.
> > >> > This means that tempest would have to run those jobs as well, so
> three
> > >> > extra jobs, until the API version is removed.
> > >> >
> > >> > The other option is to move those tests out of Tempest, into the
> > >> projects.
> > >> > This would imply back porting them to all relevant branches as well,
> > >> but it
> > >> > would have the advantage of decoupling them from Tempest. It should
> be
> > >> no
> > >> > concern from an API stability POV since the code for that API will
> be
> > >> > frozen.
> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
> as
> > >> far
> > >> > as I can tell - removed or deprecated from interoperability
> guidelines,
> > >> so
> > >> > moving the tests out of Tempest would not be an issue in that sense.
> > >> >
> > >> > The 2nd option involves a bit more initial overhead for the removal
> of
> > >> > tests, but I think it would works for the best on the long term.
> > >> >
> > >> > There is a 3rd option as well, which is to stop running integration
> > >> testing
> > >> > on deprecated API versions before they are actually removed, but I
> feel
> > >> > that would not meet the criteria defined by the
> > >> follow-standard-deprecation
> > >> > tag.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > andrea
> > >>
> > >> Are any of those tests used by the interoperability working group
> > >> (formerly DefCore)?
> > >>
> > >>
> > > That's a good question. I was very curious about this because last I
> > > checked keystone had v2.0 calls required for defcore. Looks like that
> might
> > > not be the case anymore [0]? I started a similar thread to this after
> the
> > > PTG since that was something our group talked about extensively during
> the
> > > deprecation session [1].
> > >
> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
> > 2017.01.json [0]​. But there are some compute tests which internally use
> > glance v1 API call [2]. But on mentioned flagged action- Nova already
> moved
> > to v2 APIs and tempest part is pending which can be fixed to make call on
> > v2 APIs only (which can be part of this work and quick).
> >
> > ​From options about deprecated APIs testing, I am with options 2 which
> > really take out the load of Tempest tests maintenance and gate.   ​
> >
> > ​But another question is about stable branch testing of those API, like
> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
> > ​As Tempest is responsible of testing all stable branch behavior too,
> Should
> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
> > state in all stable branch) ?​
>
> Excellent point.
>
>
> As far as I can tell:
> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> deprecated in all supported releases.
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
> - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> Tempest until Mitaka EOL, which is in a month from now
>
> We should stop testing these three api versions in the common gate
> including stable branches now (except for keystone v2 on stable/mitaka
> which can run for one more month).
>
> Are cinder / glance / keystone willing to take over the API tests and run
> them in their own gate until removal of the API version?
>
>
> 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Lance Bragstad]

On Fri, Mar 10, 2017 at 8:49 AM, Andrea Frittoli 
wrote:

>
>
> On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann 
> wrote:
>
>> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
>> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad 
>> wrote:
>> >
>> > >
>> > >
>> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
>> > > wrote:
>> > >
>> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
>> > >> > Hi folks,
>> > >> >
>> > >> > I'm trying to figure out what's the best approach to fade out
>> testing of
>> > >> > deprecated API versions.
>> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
>> API
>> > >> v2
>> > >> > and Cinder API v1.
>> > >> >
>> > >> > According to the guidelines for the "follow-standard-deprecation"
>> tag
>> > >> [0],
>> > >> > when projects that have that tag deprecate a feature:
>> > >> >
>> > >> > "Code will be frozen and only receive minimal maintenance (just so
>> that
>> > >> it
>> > >> > continues to work as-is)."
>> > >> >
>> > >> > I interpret this so that projects should maintain some level of
>> testing
>> > >> of
>> > >> > the deprecated feature, including a deprecated API version.
>> > >> > The QA team does not see value in testing deprecated API versions
>> in the
>> > >> > common gate jobs, so my question is what do to with those tests.
>> > >> >
>> > >> > One option is to maintain them in Tempest until the API version is
>> > >> removed,
>> > >> > and run them in dedicated project jobs.
>> > >> > This means that tempest would have to run those jobs as well, so
>> three
>> > >> > extra jobs, until the API version is removed.
>> > >> >
>> > >> > The other option is to move those tests out of Tempest, into the
>> > >> projects.
>> > >> > This would imply back porting them to all relevant branches as
>> well,
>> > >> but it
>> > >> > would have the advantage of decoupling them from Tempest. It
>> should be
>> > >> no
>> > >> > concern from an API stability POV since the code for that API will
>> be
>> > >> > frozen.
>> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
>> as
>> > >> far
>> > >> > as I can tell - removed or deprecated from interoperability
>> guidelines,
>> > >> so
>> > >> > moving the tests out of Tempest would not be an issue in that
>> sense.
>> > >> >
>> > >> > The 2nd option involves a bit more initial overhead for the
>> removal of
>> > >> > tests, but I think it would works for the best on the long term.
>> > >> >
>> > >> > There is a 3rd option as well, which is to stop running integration
>> > >> testing
>> > >> > on deprecated API versions before they are actually removed, but I
>> feel
>> > >> > that would not meet the criteria defined by the
>> > >> follow-standard-deprecation
>> > >> > tag.
>> > >> >
>> > >> > Thoughts?
>> > >> >
>> > >> > andrea
>> > >>
>> > >> Are any of those tests used by the interoperability working group
>> > >> (formerly DefCore)?
>> > >>
>> > >>
>> > > That's a good question. I was very curious about this because last I
>> > > checked keystone had v2.0 calls required for defcore. Looks like that
>> might
>> > > not be the case anymore [0]? I started a similar thread to this after
>> the
>> > > PTG since that was something our group talked about extensively
>> during the
>> > > deprecation session [1].
>> > >
>> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
>> > 2017.01.json [0]​. But there are some compute tests which internally use
>> > glance v1 API call [2]. But on mentioned flagged action- Nova already
>> moved
>> > to v2 APIs and tempest part is pending which can be fixed to make call
>> on
>> > v2 APIs only (which can be part of this work and quick).
>> >
>> > ​From options about deprecated APIs testing, I am with options 2 which
>> > really take out the load of Tempest tests maintenance and gate.   ​
>> >
>> > ​But another question is about stable branch testing of those API, like
>> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
>> > ​As Tempest is responsible of testing all stable branch behavior too,
>> Should
>> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
>> > state in all stable branch) ?​
>>
>> Excellent point.
>>
>>
> As far as I can tell:
> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> deprecated in all supported releases.
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
> - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> Tempest until Mitaka EOL, which is in a month from now
>
> We should stop testing these three api versions in the common gate
> including stable branches now (except for keystone v2 on stable/mitaka
> which can run for one more month).
>
> Are cinder / glance / keystone willing to take over the API tests and run
> them in their own gate until 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Andrea Frittoli]

On Fri, Mar 10, 2017 at 2:49 PM Andrea Frittoli 
wrote:

> On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann 
> wrote:
>
> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad 
> wrote:
> >
> > >
> > >
> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
> > > wrote:
> > >
> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> > >> > Hi folks,
> > >> >
> > >> > I'm trying to figure out what's the best approach to fade out
> testing of
> > >> > deprecated API versions.
> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
> API
> > >> v2
> > >> > and Cinder API v1.
> > >> >
> > >> > According to the guidelines for the "follow-standard-deprecation"
> tag
> > >> [0],
> > >> > when projects that have that tag deprecate a feature:
> > >> >
> > >> > "Code will be frozen and only receive minimal maintenance (just so
> that
> > >> it
> > >> > continues to work as-is)."
> > >> >
> > >> > I interpret this so that projects should maintain some level of
> testing
> > >> of
> > >> > the deprecated feature, including a deprecated API version.
> > >> > The QA team does not see value in testing deprecated API versions
> in the
> > >> > common gate jobs, so my question is what do to with those tests.
> > >> >
> > >> > One option is to maintain them in Tempest until the API version is
> > >> removed,
> > >> > and run them in dedicated project jobs.
> > >> > This means that tempest would have to run those jobs as well, so
> three
> > >> > extra jobs, until the API version is removed.
> > >> >
> > >> > The other option is to move those tests out of Tempest, into the
> > >> projects.
> > >> > This would imply back porting them to all relevant branches as well,
> > >> but it
> > >> > would have the advantage of decoupling them from Tempest. It should
> be
> > >> no
> > >> > concern from an API stability POV since the code for that API will
> be
> > >> > frozen.
> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
> as
> > >> far
> > >> > as I can tell - removed or deprecated from interoperability
> guidelines,
> > >> so
> > >> > moving the tests out of Tempest would not be an issue in that sense.
> > >> >
> > >> > The 2nd option involves a bit more initial overhead for the removal
> of
> > >> > tests, but I think it would works for the best on the long term.
> > >> >
> > >> > There is a 3rd option as well, which is to stop running integration
> > >> testing
> > >> > on deprecated API versions before they are actually removed, but I
> feel
> > >> > that would not meet the criteria defined by the
> > >> follow-standard-deprecation
> > >> > tag.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > andrea
> > >>
> > >> Are any of those tests used by the interoperability working group
> > >> (formerly DefCore)?
> > >>
> > >>
> > > That's a good question. I was very curious about this because last I
> > > checked keystone had v2.0 calls required for defcore. Looks like that
> might
> > > not be the case anymore [0]? I started a similar thread to this after
> the
> > > PTG since that was something our group talked about extensively during
> the
> > > deprecation session [1].
> > >
> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
> > 2017.01.json [0]​. But there are some compute tests which internally use
> > glance v1 API call [2]. But on mentioned flagged action- Nova already
> moved
> > to v2 APIs and tempest part is pending which can be fixed to make call on
> > v2 APIs only (which can be part of this work and quick).
> >
> > ​From options about deprecated APIs testing, I am with options 2 which
> > really take out the load of Tempest tests maintenance and gate.   ​
> >
> > ​But another question is about stable branch testing of those API, like
> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
> > ​As Tempest is responsible of testing all stable branch behavior too,
> Should
> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
> > state in all stable branch) ?​
>
> Excellent point.
>
>
> As far as I can tell:
> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> deprecated in all supported releases.
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
>

Of course Glance v1 still has to run on the stable/newton gate jobs, until
Newton EOL (TBD), so tests will stay in Tempest for a cycle more at least.
I guess I shouldn't be sending emails on a Friday afternoon?

The question remains for Cinder and Keystone and in general - how we want
to deal with integration tests for deprecated APIs.

- Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> Tempest until Mitaka EOL, which is in a month from now
>
> We should stop testing these three api versions in the common gate
> 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Ihar Hrachyshka]

On Fri, Mar 10, 2017 at 6:49 AM, Andrea Frittoli
 wrote:
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases

Glance not maintaining Mitaka branch?

Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Andrea Frittoli]

Forgot the links:

Glance v1 deprecation:
https://docs.openstack.org/releasenotes/glance/newton.html#deprecation-notes
Keystone v2 deprecation:
https://docs.openstack.org/releasenotes/keystone/mitaka.html#deprecation-notes
Cinder v1 deprecation:
https://wiki.openstack.org/wiki/ReleaseNotes/Juno#OpenStack_Block_Storage_.28Cinder.29


On Fri, Mar 10, 2017 at 2:49 PM Andrea Frittoli 
wrote:

> On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann 
> wrote:
>
> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad 
> wrote:
> >
> > >
> > >
> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
> > > wrote:
> > >
> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> > >> > Hi folks,
> > >> >
> > >> > I'm trying to figure out what's the best approach to fade out
> testing of
> > >> > deprecated API versions.
> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
> API
> > >> v2
> > >> > and Cinder API v1.
> > >> >
> > >> > According to the guidelines for the "follow-standard-deprecation"
> tag
> > >> [0],
> > >> > when projects that have that tag deprecate a feature:
> > >> >
> > >> > "Code will be frozen and only receive minimal maintenance (just so
> that
> > >> it
> > >> > continues to work as-is)."
> > >> >
> > >> > I interpret this so that projects should maintain some level of
> testing
> > >> of
> > >> > the deprecated feature, including a deprecated API version.
> > >> > The QA team does not see value in testing deprecated API versions
> in the
> > >> > common gate jobs, so my question is what do to with those tests.
> > >> >
> > >> > One option is to maintain them in Tempest until the API version is
> > >> removed,
> > >> > and run them in dedicated project jobs.
> > >> > This means that tempest would have to run those jobs as well, so
> three
> > >> > extra jobs, until the API version is removed.
> > >> >
> > >> > The other option is to move those tests out of Tempest, into the
> > >> projects.
> > >> > This would imply back porting them to all relevant branches as well,
> > >> but it
> > >> > would have the advantage of decoupling them from Tempest. It should
> be
> > >> no
> > >> > concern from an API stability POV since the code for that API will
> be
> > >> > frozen.
> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
> as
> > >> far
> > >> > as I can tell - removed or deprecated from interoperability
> guidelines,
> > >> so
> > >> > moving the tests out of Tempest would not be an issue in that sense.
> > >> >
> > >> > The 2nd option involves a bit more initial overhead for the removal
> of
> > >> > tests, but I think it would works for the best on the long term.
> > >> >
> > >> > There is a 3rd option as well, which is to stop running integration
> > >> testing
> > >> > on deprecated API versions before they are actually removed, but I
> feel
> > >> > that would not meet the criteria defined by the
> > >> follow-standard-deprecation
> > >> > tag.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > andrea
> > >>
> > >> Are any of those tests used by the interoperability working group
> > >> (formerly DefCore)?
> > >>
> > >>
> > > That's a good question. I was very curious about this because last I
> > > checked keystone had v2.0 calls required for defcore. Looks like that
> might
> > > not be the case anymore [0]? I started a similar thread to this after
> the
> > > PTG since that was something our group talked about extensively during
> the
> > > deprecation session [1].
> > >
> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
> > 2017.01.json [0]​. But there are some compute tests which internally use
> > glance v1 API call [2]. But on mentioned flagged action- Nova already
> moved
> > to v2 APIs and tempest part is pending which can be fixed to make call on
> > v2 APIs only (which can be part of this work and quick).
> >
> > ​From options about deprecated APIs testing, I am with options 2 which
> > really take out the load of Tempest tests maintenance and gate.   ​
> >
> > ​But another question is about stable branch testing of those API, like
> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
> > ​As Tempest is responsible of testing all stable branch behavior too,
> Should
> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
> > state in all stable branch) ?​
>
> Excellent point.
>
>
> As far as I can tell:
> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> deprecated in all supported releases.
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
> - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> Tempest until Mitaka EOL, which is in a month from now
>
> We should stop testing these three api versions in the common gate
> 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Andrea Frittoli]

On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann  wrote:

> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad 
> wrote:
> >
> > >
> > >
> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
> > > wrote:
> > >
> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> > >> > Hi folks,
> > >> >
> > >> > I'm trying to figure out what's the best approach to fade out
> testing of
> > >> > deprecated API versions.
> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
> API
> > >> v2
> > >> > and Cinder API v1.
> > >> >
> > >> > According to the guidelines for the "follow-standard-deprecation"
> tag
> > >> [0],
> > >> > when projects that have that tag deprecate a feature:
> > >> >
> > >> > "Code will be frozen and only receive minimal maintenance (just so
> that
> > >> it
> > >> > continues to work as-is)."
> > >> >
> > >> > I interpret this so that projects should maintain some level of
> testing
> > >> of
> > >> > the deprecated feature, including a deprecated API version.
> > >> > The QA team does not see value in testing deprecated API versions
> in the
> > >> > common gate jobs, so my question is what do to with those tests.
> > >> >
> > >> > One option is to maintain them in Tempest until the API version is
> > >> removed,
> > >> > and run them in dedicated project jobs.
> > >> > This means that tempest would have to run those jobs as well, so
> three
> > >> > extra jobs, until the API version is removed.
> > >> >
> > >> > The other option is to move those tests out of Tempest, into the
> > >> projects.
> > >> > This would imply back porting them to all relevant branches as well,
> > >> but it
> > >> > would have the advantage of decoupling them from Tempest. It should
> be
> > >> no
> > >> > concern from an API stability POV since the code for that API will
> be
> > >> > frozen.
> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
> as
> > >> far
> > >> > as I can tell - removed or deprecated from interoperability
> guidelines,
> > >> so
> > >> > moving the tests out of Tempest would not be an issue in that sense.
> > >> >
> > >> > The 2nd option involves a bit more initial overhead for the removal
> of
> > >> > tests, but I think it would works for the best on the long term.
> > >> >
> > >> > There is a 3rd option as well, which is to stop running integration
> > >> testing
> > >> > on deprecated API versions before they are actually removed, but I
> feel
> > >> > that would not meet the criteria defined by the
> > >> follow-standard-deprecation
> > >> > tag.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > andrea
> > >>
> > >> Are any of those tests used by the interoperability working group
> > >> (formerly DefCore)?
> > >>
> > >>
> > > That's a good question. I was very curious about this because last I
> > > checked keystone had v2.0 calls required for defcore. Looks like that
> might
> > > not be the case anymore [0]? I started a similar thread to this after
> the
> > > PTG since that was something our group talked about extensively during
> the
> > > deprecation session [1].
> > >
> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
> > 2017.01.json [0]​. But there are some compute tests which internally use
> > glance v1 API call [2]. But on mentioned flagged action- Nova already
> moved
> > to v2 APIs and tempest part is pending which can be fixed to make call on
> > v2 APIs only (which can be part of this work and quick).
> >
> > ​From options about deprecated APIs testing, I am with options 2 which
> > really take out the load of Tempest tests maintenance and gate.   ​
> >
> > ​But another question is about stable branch testing of those API, like
> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
> > ​As Tempest is responsible of testing all stable branch behavior too,
> Should
> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
> > state in all stable branch) ?​
>
> Excellent point.
>
>
As far as I can tell:
- Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
deprecated in all supported releases.
- Glance v1 has been deprecated in Newton, so it's deprecated in all
supported releases
- Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
Tempest until Mitaka EOL, which is in a month from now

We should stop testing these three api versions in the common gate
including stable branches now (except for keystone v2 on stable/mitaka
which can run for one more month).

Are cinder / glance / keystone willing to take over the API tests and run
them in their own gate until removal of the API version?

Doug
>
> >
> > >
> > > [0] https://github.com/openstack/defcore/blob/master/2017.01.json
> > > [1] http://lists.openstack.org/pipermail/openstack-dev/
> > > 2017-March/113166.html
> > >
> > > ​[2]
> > 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Doug Hellmann]

Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
> On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad  wrote:
> 
> >
> >
> > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
> > wrote:
> >
> >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> >> > Hi folks,
> >> >
> >> > I'm trying to figure out what's the best approach to fade out testing of
> >> > deprecated API versions.
> >> > We currently host in Tempest API tests for Glance API v1, Keystone API
> >> v2
> >> > and Cinder API v1.
> >> >
> >> > According to the guidelines for the "follow-standard-deprecation" tag
> >> [0],
> >> > when projects that have that tag deprecate a feature:
> >> >
> >> > "Code will be frozen and only receive minimal maintenance (just so that
> >> it
> >> > continues to work as-is)."
> >> >
> >> > I interpret this so that projects should maintain some level of testing
> >> of
> >> > the deprecated feature, including a deprecated API version.
> >> > The QA team does not see value in testing deprecated API versions in the
> >> > common gate jobs, so my question is what do to with those tests.
> >> >
> >> > One option is to maintain them in Tempest until the API version is
> >> removed,
> >> > and run them in dedicated project jobs.
> >> > This means that tempest would have to run those jobs as well, so three
> >> > extra jobs, until the API version is removed.
> >> >
> >> > The other option is to move those tests out of Tempest, into the
> >> projects.
> >> > This would imply back porting them to all relevant branches as well,
> >> but it
> >> > would have the advantage of decoupling them from Tempest. It should be
> >> no
> >> > concern from an API stability POV since the code for that API will be
> >> > frozen.
> >> > Tests for deprecated APIs in cinder, keystone and glance are all - as
> >> far
> >> > as I can tell - removed or deprecated from interoperability guidelines,
> >> so
> >> > moving the tests out of Tempest would not be an issue in that sense.
> >> >
> >> > The 2nd option involves a bit more initial overhead for the removal of
> >> > tests, but I think it would works for the best on the long term.
> >> >
> >> > There is a 3rd option as well, which is to stop running integration
> >> testing
> >> > on deprecated API versions before they are actually removed, but I feel
> >> > that would not meet the criteria defined by the
> >> follow-standard-deprecation
> >> > tag.
> >> >
> >> > Thoughts?
> >> >
> >> > andrea
> >>
> >> Are any of those tests used by the interoperability working group
> >> (formerly DefCore)?
> >>
> >>
> > That's a good question. I was very curious about this because last I
> > checked keystone had v2.0 calls required for defcore. Looks like that might
> > not be the case anymore [0]? I started a similar thread to this after the
> > PTG since that was something our group talked about extensively during the
> > deprecation session [1].
> >
> > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
> 2017.01.json [0]​. But there are some compute tests which internally use
> glance v1 API call [2]. But on mentioned flagged action- Nova already moved
> to v2 APIs and tempest part is pending which can be fixed to make call on
> v2 APIs only (which can be part of this work and quick).
> 
> ​From options about deprecated APIs testing, I am with options 2 which
> really take out the load of Tempest tests maintenance and gate.   ​
> 
> ​But another question is about stable branch testing of those API, like
> glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
> ​As Tempest is responsible of testing all stable branch behavior too, Should
>  we keep testing them till all Mitaka EOL (till APIs are in deprecated
> state in all stable branch) ?​

Excellent point.

Doug

> 
> >
> > [0] https://github.com/openstack/defcore/blob/master/2017.01.json
> > [1] http://lists.openstack.org/pipermail/openstack-dev/
> > 2017-March/113166.html
> >
> > ​[2]
> https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294  ​
> 
> >
> >
> >> Doug
> >>
> >> >
> >> > [0]
> >> > https://governance.openstack.org/tc/reference/tags/assert_fo
> >> llows-standard-deprecation.html
> >>
> >> 
> >> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
> >> e
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >

__
OpenStack Development Mailing List (not for 

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Ghanshyam Mann]

On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad  wrote:

>
>
> On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann 
> wrote:
>
>> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
>> > Hi folks,
>> >
>> > I'm trying to figure out what's the best approach to fade out testing of
>> > deprecated API versions.
>> > We currently host in Tempest API tests for Glance API v1, Keystone API
>> v2
>> > and Cinder API v1.
>> >
>> > According to the guidelines for the "follow-standard-deprecation" tag
>> [0],
>> > when projects that have that tag deprecate a feature:
>> >
>> > "Code will be frozen and only receive minimal maintenance (just so that
>> it
>> > continues to work as-is)."
>> >
>> > I interpret this so that projects should maintain some level of testing
>> of
>> > the deprecated feature, including a deprecated API version.
>> > The QA team does not see value in testing deprecated API versions in the
>> > common gate jobs, so my question is what do to with those tests.
>> >
>> > One option is to maintain them in Tempest until the API version is
>> removed,
>> > and run them in dedicated project jobs.
>> > This means that tempest would have to run those jobs as well, so three
>> > extra jobs, until the API version is removed.
>> >
>> > The other option is to move those tests out of Tempest, into the
>> projects.
>> > This would imply back porting them to all relevant branches as well,
>> but it
>> > would have the advantage of decoupling them from Tempest. It should be
>> no
>> > concern from an API stability POV since the code for that API will be
>> > frozen.
>> > Tests for deprecated APIs in cinder, keystone and glance are all - as
>> far
>> > as I can tell - removed or deprecated from interoperability guidelines,
>> so
>> > moving the tests out of Tempest would not be an issue in that sense.
>> >
>> > The 2nd option involves a bit more initial overhead for the removal of
>> > tests, but I think it would works for the best on the long term.
>> >
>> > There is a 3rd option as well, which is to stop running integration
>> testing
>> > on deprecated API versions before they are actually removed, but I feel
>> > that would not meet the criteria defined by the
>> follow-standard-deprecation
>> > tag.
>> >
>> > Thoughts?
>> >
>> > andrea
>>
>> Are any of those tests used by the interoperability working group
>> (formerly DefCore)?
>>
>>
> That's a good question. I was very curious about this because last I
> checked keystone had v2.0 calls required for defcore. Looks like that might
> not be the case anymore [0]? I started a similar thread to this after the
> PTG since that was something our group talked about extensively during the
> deprecation session [1].
>
> ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
2017.01.json [0]​. But there are some compute tests which internally use
glance v1 API call [2]. But on mentioned flagged action- Nova already moved
to v2 APIs and tempest part is pending which can be fixed to make call on
v2 APIs only (which can be part of this work and quick).

​From options about deprecated APIs testing, I am with options 2 which
really take out the load of Tempest tests maintenance and gate.   ​

​But another question is about stable branch testing of those API, like
glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
​As Tempest is responsible of testing all stable branch behavior too, Should
 we keep testing them till all Mitaka EOL (till APIs are in deprecated
state in all stable branch) ?​




>
> [0] https://github.com/openstack/defcore/blob/master/2017.01.json
> [1] http://lists.openstack.org/pipermail/openstack-dev/
> 2017-March/113166.html
>
> ​[2]
https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294  ​

>
>
>> Doug
>>
>> >
>> > [0]
>> > https://governance.openstack.org/tc/reference/tags/assert_fo
>> llows-standard-deprecation.html
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Lance Bragstad]

On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann  wrote:

> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> > Hi folks,
> >
> > I'm trying to figure out what's the best approach to fade out testing of
> > deprecated API versions.
> > We currently host in Tempest API tests for Glance API v1, Keystone API v2
> > and Cinder API v1.
> >
> > According to the guidelines for the "follow-standard-deprecation" tag
> [0],
> > when projects that have that tag deprecate a feature:
> >
> > "Code will be frozen and only receive minimal maintenance (just so that
> it
> > continues to work as-is)."
> >
> > I interpret this so that projects should maintain some level of testing
> of
> > the deprecated feature, including a deprecated API version.
> > The QA team does not see value in testing deprecated API versions in the
> > common gate jobs, so my question is what do to with those tests.
> >
> > One option is to maintain them in Tempest until the API version is
> removed,
> > and run them in dedicated project jobs.
> > This means that tempest would have to run those jobs as well, so three
> > extra jobs, until the API version is removed.
> >
> > The other option is to move those tests out of Tempest, into the
> projects.
> > This would imply back porting them to all relevant branches as well, but
> it
> > would have the advantage of decoupling them from Tempest. It should be no
> > concern from an API stability POV since the code for that API will be
> > frozen.
> > Tests for deprecated APIs in cinder, keystone and glance are all - as far
> > as I can tell - removed or deprecated from interoperability guidelines,
> so
> > moving the tests out of Tempest would not be an issue in that sense.
> >
> > The 2nd option involves a bit more initial overhead for the removal of
> > tests, but I think it would works for the best on the long term.
> >
> > There is a 3rd option as well, which is to stop running integration
> testing
> > on deprecated API versions before they are actually removed, but I feel
> > that would not meet the criteria defined by the
> follow-standard-deprecation
> > tag.
> >
> > Thoughts?
> >
> > andrea
>
> Are any of those tests used by the interoperability working group
> (formerly DefCore)?
>
>
That's a good question. I was very curious about this because last I
checked keystone had v2.0 calls required for defcore. Looks like that might
not be the case anymore [0]? I started a similar thread to this after the
PTG since that was something our group talked about extensively during the
deprecation session [1].


[0] https://github.com/openstack/defcore/blob/master/2017.01.json
[1]
http://lists.openstack.org/pipermail/openstack-dev/2017-March/113166.html



> Doug
>
> >
> > [0]
> > https://governance.openstack.org/tc/reference/tags/assert_fo
> llows-standard-deprecation.html
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][qa][tc][glance][keystone][cinder] Testing of deprecated API versions

[Doug Hellmann]

Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +:
> Hi folks,
> 
> I'm trying to figure out what's the best approach to fade out testing of
> deprecated API versions.
> We currently host in Tempest API tests for Glance API v1, Keystone API v2
> and Cinder API v1.
> 
> According to the guidelines for the "follow-standard-deprecation" tag [0],
> when projects that have that tag deprecate a feature:
> 
> "Code will be frozen and only receive minimal maintenance (just so that it
> continues to work as-is)."
> 
> I interpret this so that projects should maintain some level of testing of
> the deprecated feature, including a deprecated API version.
> The QA team does not see value in testing deprecated API versions in the
> common gate jobs, so my question is what do to with those tests.
> 
> One option is to maintain them in Tempest until the API version is removed,
> and run them in dedicated project jobs.
> This means that tempest would have to run those jobs as well, so three
> extra jobs, until the API version is removed.
> 
> The other option is to move those tests out of Tempest, into the projects.
> This would imply back porting them to all relevant branches as well, but it
> would have the advantage of decoupling them from Tempest. It should be no
> concern from an API stability POV since the code for that API will be
> frozen.
> Tests for deprecated APIs in cinder, keystone and glance are all - as far
> as I can tell - removed or deprecated from interoperability guidelines, so
> moving the tests out of Tempest would not be an issue in that sense.
> 
> The 2nd option involves a bit more initial overhead for the removal of
> tests, but I think it would works for the best on the long term.
> 
> There is a 3rd option as well, which is to stop running integration testing
> on deprecated API versions before they are actually removed, but I feel
> that would not meet the criteria defined by the follow-standard-deprecation
> tag.
> 
> Thoughts?
> 
> andrea

Are any of those tests used by the interoperability working group
(formerly DefCore)?

Doug

> 
> [0]
> https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev