Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Ben Swartzlander]

On 09/02/2016 12:40 PM, Eric Harney wrote:

On 08/15/2016 04:48 AM, Thierry Carrez wrote:

Sean Dague wrote:

On 08/12/2016 01:10 PM, Walter A. Boring IV wrote:

I believe there is a compromise that we could implement in Cinder that
enables us to have a deprecation
of unsupported drivers that aren't meeting the Cinder driver
requirements and allow upgrades to work
without outright immediately removing a driver.

 1. Add a 'supported = True' attribute to every driver.
 2. When a driver no longer meets Cinder community requirements, put a
patch up against the driver
 3. When c-vol service starts, check the supported flag.  If the flag is
False, then log an exception, and disable the driver.
 4. Allow the admin to put an entry in cinder.conf for the driver in
question "enable_unsupported_driver = True".  This will allow the
c-vol service to start the driver and allow it to work.  Log a
warning on every driver call.
 5. This is a positive acknowledgement by the operator that they are
enabling a potentially broken driver. Use at your own risk.
 6. If the vendor doesn't get the CI working in the next release, then
remove the driver.
 7. If the vendor gets the CI working again, then set the supported flag
back to True and all is good.

This allows a deprecation period for a driver, and keeps operators who
upgrade their deployment from losing access to their volumes they have
on those back-ends.  It will give them time to contact the community
and/or do some research, and find out what happened to the driver.
This also potentially gives the operator time to find a new supported
backend and start migrating volumes.  I say potentially, because the
driver may be broken, or it may work enough to migrate volumes off of it
to a new backend.

Having unsupported drivers in tree is terrible for the Cinder community,
and in the long run terrible for operators.
Instantly removing drivers because CI is unstable is terrible for
operators in the short term, because as soon as they upgrade OpenStack,
they lose all access to managing their existing volumes.   Just because
we leave a driver in tree in this state, doesn't mean that the operator
will be able to migrate if the drive is broken, but they'll have a
chance depending on the state of the driver in question.  It could be
horribly broken, but the breakage might be something fixable by someone
that just knows Python.   If the driver is gone from tree entirely, then
that's a lot more to overcome.

I don't think there is a way to make everyone happy all the time, but I
think this buys operators a small window of opportunity to still manage
their existing volumes before the driver is removed.  It also still
allows the Cinder community to deal with unsupported drivers in a way
that will motivate vendors to keep their stuff working.


This seems very reasonable. It allows the cinder team to mark stuff
unsupported at any point that vendors do not meet their upstream
commitments, but still provides some path forward for operators that
didn't realize their chosen vendor abandoned them and the community
until after they are in the midst of upgrade. It's very important that
the cinder team is able to keep a very visible hammer for vendors not
living up to their commitments.

Keeping some visible data around drivers that are flapping (going
unsupported, showing up with CI to get back out of the state,
disappearing again) would be great as well, to further give operators
data on what vendors are working in good faith and which aren't.


I like this a lot, and it certainly would address the deprecation policy
part.

Sean: I was wondering if that would not still be considered breaking
upgrades, though... Since you end up upgrading and your c-vol would not
restart until you set enable_unsupported_driver = True ?



Kind of late jumping in here, but I'm curious what you guys think on
this last point.

I have a similar feeling, that this may still be breaking upgrades in an
undesirable way.

There are softer alternatives that could still communicate that a driver
is unsupported and not halt the upgrade path, such as printing warning
messages when the c-vol service starts up, etc.


It's important to remember that there are 2 goals behind the current policy:

1) Protect end users from buggy/unmaintained drivers.

2) Motivate vendors to actively maintain their drivers by associating a 
painful consequence with not maintaining them.


Most of this discussion seems to be about alternative ways to address 
the first goal. However the second goal is in direct conflict with many 
of the proposed solutions. The more kind and gentle the approach is to 
deprecating drivers, the less effective it will be as a motivator for 
vendors.


Personally I'm of 2 minds about this. I think it's useful for the 
community to have a stick to beat vendors with to keep drivers 
maintained, and I think backing off the current driver deletion policy 
would turn that stick into a nerf 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Eric Harney]

On 08/15/2016 04:48 AM, Thierry Carrez wrote:
> Sean Dague wrote:
>> On 08/12/2016 01:10 PM, Walter A. Boring IV wrote:
>>> I believe there is a compromise that we could implement in Cinder that
>>> enables us to have a deprecation
>>> of unsupported drivers that aren't meeting the Cinder driver
>>> requirements and allow upgrades to work
>>> without outright immediately removing a driver.
>>>
>>>  1. Add a 'supported = True' attribute to every driver.
>>>  2. When a driver no longer meets Cinder community requirements, put a
>>> patch up against the driver
>>>  3. When c-vol service starts, check the supported flag.  If the flag is
>>> False, then log an exception, and disable the driver.
>>>  4. Allow the admin to put an entry in cinder.conf for the driver in
>>> question "enable_unsupported_driver = True".  This will allow the
>>> c-vol service to start the driver and allow it to work.  Log a
>>> warning on every driver call.
>>>  5. This is a positive acknowledgement by the operator that they are
>>> enabling a potentially broken driver. Use at your own risk.
>>>  6. If the vendor doesn't get the CI working in the next release, then
>>> remove the driver. 
>>>  7. If the vendor gets the CI working again, then set the supported flag
>>> back to True and all is good. 
>>>
>>> This allows a deprecation period for a driver, and keeps operators who
>>> upgrade their deployment from losing access to their volumes they have
>>> on those back-ends.  It will give them time to contact the community
>>> and/or do some research, and find out what happened to the driver.  
>>> This also potentially gives the operator time to find a new supported
>>> backend and start migrating volumes.  I say potentially, because the
>>> driver may be broken, or it may work enough to migrate volumes off of it
>>> to a new backend.
>>>
>>> Having unsupported drivers in tree is terrible for the Cinder community,
>>> and in the long run terrible for operators.
>>> Instantly removing drivers because CI is unstable is terrible for
>>> operators in the short term, because as soon as they upgrade OpenStack,
>>> they lose all access to managing their existing volumes.   Just because
>>> we leave a driver in tree in this state, doesn't mean that the operator
>>> will be able to migrate if the drive is broken, but they'll have a
>>> chance depending on the state of the driver in question.  It could be
>>> horribly broken, but the breakage might be something fixable by someone
>>> that just knows Python.   If the driver is gone from tree entirely, then
>>> that's a lot more to overcome.
>>>
>>> I don't think there is a way to make everyone happy all the time, but I
>>> think this buys operators a small window of opportunity to still manage
>>> their existing volumes before the driver is removed.  It also still
>>> allows the Cinder community to deal with unsupported drivers in a way
>>> that will motivate vendors to keep their stuff working.
>>
>> This seems very reasonable. It allows the cinder team to mark stuff
>> unsupported at any point that vendors do not meet their upstream
>> commitments, but still provides some path forward for operators that
>> didn't realize their chosen vendor abandoned them and the community
>> until after they are in the midst of upgrade. It's very important that
>> the cinder team is able to keep a very visible hammer for vendors not
>> living up to their commitments.
>>
>> Keeping some visible data around drivers that are flapping (going
>> unsupported, showing up with CI to get back out of the state,
>> disappearing again) would be great as well, to further give operators
>> data on what vendors are working in good faith and which aren't.
> 
> I like this a lot, and it certainly would address the deprecation policy
> part.
> 
> Sean: I was wondering if that would not still be considered breaking
> upgrades, though... Since you end up upgrading and your c-vol would not
> restart until you set enable_unsupported_driver = True ?
> 

Kind of late jumping in here, but I'm curious what you guys think on
this last point.

I have a similar feeling, that this may still be breaking upgrades in an
undesirable way.

There are softer alternatives that could still communicate that a driver
is unsupported and not halt the upgrade path, such as printing warning
messages when the c-vol service starts up, etc.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Thierry Carrez]

Sean Dague wrote:
> On 08/12/2016 01:10 PM, Walter A. Boring IV wrote:
>> I believe there is a compromise that we could implement in Cinder that
>> enables us to have a deprecation
>> of unsupported drivers that aren't meeting the Cinder driver
>> requirements and allow upgrades to work
>> without outright immediately removing a driver.
>>
>>  1. Add a 'supported = True' attribute to every driver.
>>  2. When a driver no longer meets Cinder community requirements, put a
>> patch up against the driver
>>  3. When c-vol service starts, check the supported flag.  If the flag is
>> False, then log an exception, and disable the driver.
>>  4. Allow the admin to put an entry in cinder.conf for the driver in
>> question "enable_unsupported_driver = True".  This will allow the
>> c-vol service to start the driver and allow it to work.  Log a
>> warning on every driver call.
>>  5. This is a positive acknowledgement by the operator that they are
>> enabling a potentially broken driver. Use at your own risk.
>>  6. If the vendor doesn't get the CI working in the next release, then
>> remove the driver. 
>>  7. If the vendor gets the CI working again, then set the supported flag
>> back to True and all is good. 
>>
>> This allows a deprecation period for a driver, and keeps operators who
>> upgrade their deployment from losing access to their volumes they have
>> on those back-ends.  It will give them time to contact the community
>> and/or do some research, and find out what happened to the driver.  
>> This also potentially gives the operator time to find a new supported
>> backend and start migrating volumes.  I say potentially, because the
>> driver may be broken, or it may work enough to migrate volumes off of it
>> to a new backend.
>>
>> Having unsupported drivers in tree is terrible for the Cinder community,
>> and in the long run terrible for operators.
>> Instantly removing drivers because CI is unstable is terrible for
>> operators in the short term, because as soon as they upgrade OpenStack,
>> they lose all access to managing their existing volumes.   Just because
>> we leave a driver in tree in this state, doesn't mean that the operator
>> will be able to migrate if the drive is broken, but they'll have a
>> chance depending on the state of the driver in question.  It could be
>> horribly broken, but the breakage might be something fixable by someone
>> that just knows Python.   If the driver is gone from tree entirely, then
>> that's a lot more to overcome.
>>
>> I don't think there is a way to make everyone happy all the time, but I
>> think this buys operators a small window of opportunity to still manage
>> their existing volumes before the driver is removed.  It also still
>> allows the Cinder community to deal with unsupported drivers in a way
>> that will motivate vendors to keep their stuff working.
> 
> This seems very reasonable. It allows the cinder team to mark stuff
> unsupported at any point that vendors do not meet their upstream
> commitments, but still provides some path forward for operators that
> didn't realize their chosen vendor abandoned them and the community
> until after they are in the midst of upgrade. It's very important that
> the cinder team is able to keep a very visible hammer for vendors not
> living up to their commitments.
> 
> Keeping some visible data around drivers that are flapping (going
> unsupported, showing up with CI to get back out of the state,
> disappearing again) would be great as well, to further give operators
> data on what vendors are working in good faith and which aren't.

I like this a lot, and it certainly would address the deprecation policy
part.

Sean: I was wondering if that would not still be considered breaking
upgrades, though... Since you end up upgrading and your c-vol would not
restart until you set enable_unsupported_driver = True ?

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Jay Bryant]

I agree with Duncan here. Driver removal has been an important tool to keep
only maintained drivers in the tree and a way to get attention to ignored
drivers. I hate to see the tag removed but think we need to stay true to
the approach we have been taking.

Re-addressing the meaning of the tag with regards to drivers will be
necessary so that we can communicate that Cinder Core is compliant though
we can't guarantee driver compliance.

Jay

On Thu, Aug 11, 2016 at 9:48 AM Duncan Thomas 
wrote:

> Given the options, I'd agree with Sean and John that removing the tag is a
> far lesser evil than changing our policy.
>
> If we leave broken drivers in the tree, the end user (operator) is no
> better off - the thing they evaluated won't work - but it will be harder to
> tell why. The storage vendor won't suffer the pressure that comes from
> driver removal, so will have less incentive to fix their driver (there's
> enough examples of the threat of driver removal causing the immediate fix
> of things that have remained broken for months that we know, for certain
> that the policy works).
>
> I'd prefer to make the meaning of the tag sane WRT third party drivers,
> which I think would help other projects to be able to police their drivers
> and CI better too, without risking losing / not gaining the tag, which is
> likely to hurt a newer project far more than it will cinder.
>
>
>
> On 11 August 2016 at 17:29, John Griffith 
> wrote:
>
>>
>>
>> On Thu, Aug 11, 2016 at 7:14 AM, Erno Kuvaja  wrote:
>>
>>> On Thu, Aug 11, 2016 at 2:47 PM, Sean McGinnis 
>>> wrote:
>>> >> >>
>>> >> >> As follow up on the mailing list discussion [0], gerrit activity
>>> >> >> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
>>> >> >> discussion how Cinder follows, or rather does not follow, the
>>> standard
>>> >> >> deprecation policy [4] as the project has been tagged on the assert
>>> >> >> page [5].
>>> >> >>
>>> > 
>>> >> >>
>>> >> >> [0]
>>> http://lists.openstack.org/pipermail/openstack-dev/2016-August/100717.html
>>> >> >> [1] https://review.openstack.org/#/c/348032/
>>> >> >> [2] https://review.openstack.org/#/c/348042/
>>> >> >> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
>>> >> >> [4]
>>> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#requirements
>>> >> >> [5]
>>> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#application-to-current-projects
>>> >> >>
>>> >> >
>>> >> > Can you be more specific about what you mean? Are you saying that
>>> >> > the policy isn't being followed because the drivers were removed
>>> >> > without a deprecation period, or is there something else to it?
>>> >> >
>>> >> > Doug
>>> >> >
>>> >>
>>> >> Yes, that's how I see it. Cinder's own policy is that the drivers can
>>> >> be removed without any warning to the consumers while the standard
>>> >> deprecation policy defines quite strict lines about informing the
>>> >> consumer of the functionality deprecation before it gets removed.
>>> >>
>>> >> - Erno
>>> >
>>> > It is a good point. I think it highlights a common thread though with
>>> > the other discussion that, at least so far, third party drivers are
>>> > treated differently than the rest of the code.
>>> >
>>> > For any other functionality we certainly follow the deprecation policy.
>>> > Even in existing drivers we try to enforce that any driver renames,
>>> > config setting changes, and similar non-backwards compatible changes go
>>> > through the normal deprecation cycle before being removed.
>>> >
>>> > Ideally I would love it if we could comply with the deprecation policy
>>> > with regards to driver removal. But the reality is, if we don't see
>>> that
>>> > a driver is being supported and maintained by its vendor, then that
>>> > burden can't fall on the wider OpenStack and Cinder community that has
>>> > no way of validating against physical hardware.
>>> >
>>> > I think third party drivers need to be treated differently when it
>>> comes
>>> > to the deprecation policy. If that is not acceptable, then I suppose we
>>> > do need to remove that tag. Tag removal would be the lesser of the two
>>> > versus keeping around drivers that we know aren't really being
>>> > maintained.
>>> >
>>> > If it came to that, I would also consider creating a new cinder-drivers
>>> > project under the Cinder umbrella and move all of the drivers not
>>> tested
>>> > by Jenkins over to that. That wouldn't be a trivial undertaking, so I
>>> > would try to avoid that if possible. But it would at least allow us to
>>> > still get code reviews and all of the benefits of being in tree. Just
>>> > some thoughts.
>>> >
>>> > Sean
>>> >
>>>
>>> Sean,
>>>
>>> As said on my initial opening, I do understand and agree with the
>>> reasoning/treatment of the 3rd party drivers. My request for that tag
>>> 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Duncan Thomas]

There's so much misinformation in that email I barely know where to start.

There is nothing stopping out of tree drivers for cinder, and a few have
existed, though they don't seem to stick around. The driver is just a
python class referenced in the config file.

Turning a removed driver into an out of tree driver (or patching it back
into the tree) is trivial for anybody with basic python skills. They can
even just apply a reverse patch of the removal patch directly and cleanly
most of the time, since the drivers are clearly separated.

As has been said in the thread multiple times, by multiple people, the idea
of out of tree drivers has been discussed, passionately and at vast length,
with people on both sides of the debate. We've got storage vendors,
operators and distribution packagers at every single one of these
discussions, and have had each time it is discussed, which has been at
least the last three summits and the last three mid cycles.

It is getting tiring and distracting to keep rehashing that decision in
thread with nothing new being said, with somebody who neither has a driver
nor otherwise contributes to cinder. Please have the curtsey to follow some
of the provided historical references before repeatedly derailing the
thread by expounding the virtues of out of tree drivers. They have been
discussed, and soundly (though not unanimously, as Mike points out)
rejected. We have clearly decided there is a consensus that they aren't
what we want now. That is not what we're trying to discuss here.

To spell it pot one more time: we don't stop out of tree drivers. They
work, they're easy. We don't advertise them as supported because they're
not part of the review or testing process. We like in tree drivers. Vendors
like in tree drivers, and the advertised support we give them for doing so.
They will handle the Burton of keeping a third party CI going to maintain
that status, though a little begrudgingly - it has repeatedly and
continuously been necessary to have the option of the (apparently
substantial, given the effect it can have) threat of removal from tree in
order to persuade them to put enough resources into keeping their CI going.

On 13 Aug 2016 16:40, "Ihar Hrachyshka"  wrote:

> Clay Gerrard  wrote:
>
> The 
> use_untested_probably_broken_deprecated_manger_so_maybe_i_can_migrate_cross_fingers
>> option sounds good!  The experiment would be then if it's still enough of a
>> stick to keep 3rd party drivers pony'd up on their commitment to the Cinder
>> team to consistently ship quality releases?
>>
>>
> This commitment, is it the only model that you allow to extend Cinder for
> vendor technology? Because if not, then, in a way, you put vendors in an
> unfortunate situation where they are forced into a very specific model of
> commitment, that may be not in the best interest of that vendor. While
> there may be a value of keeping multiple drivers closer to the core code
> (code reusability, spotting common patterns, …), I feel that the benefit
> from such collaboration is worthwhile only when it's mutual, and not forced
> onto.
>
> I assume that if there would be alternatives available (including walking
> autonomously of Cinder release cycles and practices), then some of those
> vendors that you currently try hard to police into doing things the right
> and only way would actually choose that alternative path, that could be
> more in line with their production cycle. And maybe those vendors that
> break current centralized rules would voluntarily vote for leaving the tree
> to pursuit happiness as they see it, essentially freeing you from the need
> to police code that you cannot actually maintain.
>
> What about maybe the operator just not upgrading till post migration?
>> It's the migration that sucks right?  You either get to punt a release and
>> hope it gets "back in good faith" or do it now and that 3rd party driver
>> has lost your business/trust.
>>
>
> The deprecation tag indicates a good will of the community to do whatever
> it takes to fulfill the guarantee that a solution that worked in a previous
> cycle won’t be dropped with no prior notice (read: deprecation warnings in
> logs). Explicitly removing a driver just because you *think* it may no
> longer work is not in line with this thinking. Yes, there may be bugs in
> the code, but there is at least a path forward: for one, operators may try
> to fix bugs they hit in upgrade, or they can work with the vendor to fix
> the code and backport the needed fixes to stable branches. When you don’t
> have the code in tree at all, it’s impossible to backport, because stable
> branches don’t allow new features. And it’s not possible to play with
> (potentially broken) driver to understand which bugs block you from going
> forward.
>
> Ihar
>
> __
> OpenStack Development Mailing List (not for usage questions)
> 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Ihar Hrachyshka]

Clay Gerrard  wrote:

The  
use_untested_probably_broken_deprecated_manger_so_maybe_i_can_migrate_cross_fingers  
option sounds good!  The experiment would be then if it's still enough of  
a stick to keep 3rd party drivers pony'd up on their commitment to the  
Cinder team to consistently ship quality releases?




This commitment, is it the only model that you allow to extend Cinder for  
vendor technology? Because if not, then, in a way, you put vendors in an  
unfortunate situation where they are forced into a very specific model of  
commitment, that may be not in the best interest of that vendor. While  
there may be a value of keeping multiple drivers closer to the core code  
(code reusability, spotting common patterns, …), I feel that the benefit  
from such collaboration is worthwhile only when it's mutual, and not forced  
onto.


I assume that if there would be alternatives available (including walking  
autonomously of Cinder release cycles and practices), then some of those  
vendors that you currently try hard to police into doing things the right  
and only way would actually choose that alternative path, that could be  
more in line with their production cycle. And maybe those vendors that  
break current centralized rules would voluntarily vote for leaving the tree  
to pursuit happiness as they see it, essentially freeing you from the need  
to police code that you cannot actually maintain.


What about maybe the operator just not upgrading till post migration?   
It's the migration that sucks right?  You either get to punt a release  
and hope it gets "back in good faith" or do it now and that 3rd party  
driver has lost your business/trust.


The deprecation tag indicates a good will of the community to do whatever  
it takes to fulfill the guarantee that a solution that worked in a previous  
cycle won’t be dropped with no prior notice (read: deprecation warnings in  
logs). Explicitly removing a driver just because you *think* it may no  
longer work is not in line with this thinking. Yes, there may be bugs in  
the code, but there is at least a path forward: for one, operators may try  
to fix bugs they hit in upgrade, or they can work with the vendor to fix  
the code and backport the needed fixes to stable branches. When you don’t  
have the code in tree at all, it’s impossible to backport, because stable  
branches don’t allow new features. And it’s not possible to play with  
(potentially broken) driver to understand which bugs block you from going  
forward.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Doug Hellmann]

Excerpts from Sean Dague's message of 2016-08-12 14:43:57 -0400:
> On 08/12/2016 01:10 PM, Walter A. Boring IV wrote:
> > 
> >> I was leaning towards a separate repo until I started thinking about all
> >> the overhead and complications this would cause. It's another repo for
> >> cores to watch. It would cause everyone extra complication in setting up
> >> their CI, which is already one of the biggest roadblocks. It would make
> >> it a little harder to do things like https://review.openstack.org/297140
> >> and https://review.openstack.org/346470 to be able to generate this:
> >> http://docs.openstack.org/developer/cinder/drivers.html. Plus more infra
> >> setup, more moving parts to break, and just generally more
> >> complications.
> >>
> >> All things that can be solved for sure. I just question whether it would
> >> be worth having that overhead. Frankly, there are better things I'd like
> >> to spend my time on.
> >>
> >> I think at this point my first preference would actually be to define a
> >> new tag. This addresses both the driver removal issue as well as the
> >> backporting of driver bug fixes. I would like to see third party drivers
> >> recognized and treated as being different, because in reality they are
> >> very different than the rest of the code. Having something like
> >> follows_deprecation_but_has_third_party_drivers_that_dont would make a
> >> clear statement that their is a vendor component to this project that
> >> really has to be treated differently and has different concerns
> >> deployers need to be aware of.
> >>
> >> Barring that, I think my next choice would be to remove the tag. That
> >> would really be unfortunate as we do want to make it clear to users that
> >> Cinder will not arbitrarily break APIs or do anything between releases
> >> without warning when it comes to non-third party drivers. But if that is
> >> what we need to do to effectively communicate what to expect from
> >> Cinder, then I'm OK with that.
> >>
> >> My last choice (of the ones I'm favorable towards) would be marking a
> >> driver as untested/unstable/abandoned/etc rather than removing it. We
> >> could flag these a certain way and have then spam the logs like crazy
> >> after upgrade to make it very and painfully clear that they are not
> >> being maintained. But as Duncan pointed out, this doesn't have as much
> >> impact for getting vendor attention. It's amazing the level of executive
> >> involvement that can happen after a patch is put up for driver removal
> >> due to non-compliance.
> >>
> >> Sean
> >>
> >> __
> > I believe there is a compromise that we could implement in Cinder that
> > enables us to have a deprecation
> > of unsupported drivers that aren't meeting the Cinder driver
> > requirements and allow upgrades to work
> > without outright immediately removing a driver.
> > 
> >  1. Add a 'supported = True' attribute to every driver.
> >  2. When a driver no longer meets Cinder community requirements, put a
> > patch up against the driver
> >  3. When c-vol service starts, check the supported flag.  If the flag is
> > False, then log an exception, and disable the driver.
> >  4. Allow the admin to put an entry in cinder.conf for the driver in
> > question "enable_unsupported_driver = True".  This will allow the
> > c-vol service to start the driver and allow it to work.  Log a
> > warning on every driver call.
> >  5. This is a positive acknowledgement by the operator that they are
> > enabling a potentially broken driver. Use at your own risk.
> >  6. If the vendor doesn't get the CI working in the next release, then
> > remove the driver. 
> >  7. If the vendor gets the CI working again, then set the supported flag
> > back to True and all is good. 
> > 
> > 
> > This allows a deprecation period for a driver, and keeps operators who
> > upgrade their deployment from losing access to their volumes they have
> > on those back-ends.  It will give them time to contact the community
> > and/or do some research, and find out what happened to the driver.  
> > This also potentially gives the operator time to find a new supported
> > backend and start migrating volumes.  I say potentially, because the
> > driver may be broken, or it may work enough to migrate volumes off of it
> > to a new backend.
> > 
> > Having unsupported drivers in tree is terrible for the Cinder community,
> > and in the long run terrible for operators.
> > Instantly removing drivers because CI is unstable is terrible for
> > operators in the short term, because as soon as they upgrade OpenStack,
> > they lose all access to managing their existing volumes.   Just because
> > we leave a driver in tree in this state, doesn't mean that the operator
> > will be able to migrate if the drive is broken, but they'll have a
> > chance depending on the state of the driver in question.  It could be
> > horribly broken, 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Sean Dague]

On 08/12/2016 01:10 PM, Walter A. Boring IV wrote:
> 
>> I was leaning towards a separate repo until I started thinking about all
>> the overhead and complications this would cause. It's another repo for
>> cores to watch. It would cause everyone extra complication in setting up
>> their CI, which is already one of the biggest roadblocks. It would make
>> it a little harder to do things like https://review.openstack.org/297140
>> and https://review.openstack.org/346470 to be able to generate this:
>> http://docs.openstack.org/developer/cinder/drivers.html. Plus more infra
>> setup, more moving parts to break, and just generally more
>> complications.
>>
>> All things that can be solved for sure. I just question whether it would
>> be worth having that overhead. Frankly, there are better things I'd like
>> to spend my time on.
>>
>> I think at this point my first preference would actually be to define a
>> new tag. This addresses both the driver removal issue as well as the
>> backporting of driver bug fixes. I would like to see third party drivers
>> recognized and treated as being different, because in reality they are
>> very different than the rest of the code. Having something like
>> follows_deprecation_but_has_third_party_drivers_that_dont would make a
>> clear statement that their is a vendor component to this project that
>> really has to be treated differently and has different concerns
>> deployers need to be aware of.
>>
>> Barring that, I think my next choice would be to remove the tag. That
>> would really be unfortunate as we do want to make it clear to users that
>> Cinder will not arbitrarily break APIs or do anything between releases
>> without warning when it comes to non-third party drivers. But if that is
>> what we need to do to effectively communicate what to expect from
>> Cinder, then I'm OK with that.
>>
>> My last choice (of the ones I'm favorable towards) would be marking a
>> driver as untested/unstable/abandoned/etc rather than removing it. We
>> could flag these a certain way and have then spam the logs like crazy
>> after upgrade to make it very and painfully clear that they are not
>> being maintained. But as Duncan pointed out, this doesn't have as much
>> impact for getting vendor attention. It's amazing the level of executive
>> involvement that can happen after a patch is put up for driver removal
>> due to non-compliance.
>>
>> Sean
>>
>> __
> I believe there is a compromise that we could implement in Cinder that
> enables us to have a deprecation
> of unsupported drivers that aren't meeting the Cinder driver
> requirements and allow upgrades to work
> without outright immediately removing a driver.
> 
>  1. Add a 'supported = True' attribute to every driver.
>  2. When a driver no longer meets Cinder community requirements, put a
> patch up against the driver
>  3. When c-vol service starts, check the supported flag.  If the flag is
> False, then log an exception, and disable the driver.
>  4. Allow the admin to put an entry in cinder.conf for the driver in
> question "enable_unsupported_driver = True".  This will allow the
> c-vol service to start the driver and allow it to work.  Log a
> warning on every driver call.
>  5. This is a positive acknowledgement by the operator that they are
> enabling a potentially broken driver. Use at your own risk.
>  6. If the vendor doesn't get the CI working in the next release, then
> remove the driver. 
>  7. If the vendor gets the CI working again, then set the supported flag
> back to True and all is good. 
> 
> 
> This allows a deprecation period for a driver, and keeps operators who
> upgrade their deployment from losing access to their volumes they have
> on those back-ends.  It will give them time to contact the community
> and/or do some research, and find out what happened to the driver.  
> This also potentially gives the operator time to find a new supported
> backend and start migrating volumes.  I say potentially, because the
> driver may be broken, or it may work enough to migrate volumes off of it
> to a new backend.
> 
> Having unsupported drivers in tree is terrible for the Cinder community,
> and in the long run terrible for operators.
> Instantly removing drivers because CI is unstable is terrible for
> operators in the short term, because as soon as they upgrade OpenStack,
> they lose all access to managing their existing volumes.   Just because
> we leave a driver in tree in this state, doesn't mean that the operator
> will be able to migrate if the drive is broken, but they'll have a
> chance depending on the state of the driver in question.  It could be
> horribly broken, but the breakage might be something fixable by someone
> that just knows Python.   If the driver is gone from tree entirely, then
> that's a lot more to overcome.
> 
> I don't think there is a way to make everyone happy all the time, but 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[John Griffith]

On Fri, Aug 12, 2016 at 12:10 PM, Walter A. Boring IV  wrote:

>
> I was leaning towards a separate repo until I started thinking about all
> the overhead and complications this would cause. It's another repo for
> cores to watch. It would cause everyone extra complication in setting up
> their CI, which is already one of the biggest roadblocks. It would make
> it a little harder to do things like https://review.openstack.org/297140
> and https://review.openstack.org/346470 to be able to generate 
> this:http://docs.openstack.org/developer/cinder/drivers.html. Plus more infra
> setup, more moving parts to break, and just generally more
> complications.
>
> All things that can be solved for sure. I just question whether it would
> be worth having that overhead. Frankly, there are better things I'd like
> to spend my time on.
>
> I think at this point my first preference would actually be to define a
> new tag. This addresses both the driver removal issue as well as the
> backporting of driver bug fixes. I would like to see third party drivers
> recognized and treated as being different, because in reality they are
> very different than the rest of the code. Having something like
> follows_deprecation_but_has_third_party_drivers_that_dont would make a
> clear statement that their is a vendor component to this project that
> really has to be treated differently and has different concerns
> deployers need to be aware of.
>
> Barring that, I think my next choice would be to remove the tag. That
> would really be unfortunate as we do want to make it clear to users that
> Cinder will not arbitrarily break APIs or do anything between releases
> without warning when it comes to non-third party drivers. But if that is
> what we need to do to effectively communicate what to expect from
> Cinder, then I'm OK with that.
>
> My last choice (of the ones I'm favorable towards) would be marking a
> driver as untested/unstable/abandoned/etc rather than removing it. We
> could flag these a certain way and have then spam the logs like crazy
> after upgrade to make it very and painfully clear that they are not
> being maintained. But as Duncan pointed out, this doesn't have as much
> impact for getting vendor attention. It's amazing the level of executive
> involvement that can happen after a patch is put up for driver removal
> due to non-compliance.
>
> Sean
>
> __
>
> I believe there is a compromise that we could implement in Cinder that
> enables us to have a deprecation
> of unsupported drivers that aren't meeting the Cinder driver requirements
> and allow upgrades to work
> without outright immediately removing a driver.
>
>
>1. Add a 'supported = True' attribute to every driver.
>2. When a driver no longer meets Cinder community requirements, put a
>patch up against the driver
>3. When c-vol service starts, check the supported flag.  If the flag
>is False, then log an exception, and disable the driver.
>4. Allow the admin to put an entry in cinder.conf for the driver in
>question "enable_unsupported_driver = True".  This will allow the c-vol
>service to start the driver and allow it to work.  Log a warning on every
>driver call.
>5. This is a positive acknowledgement by the operator that they are
>enabling a potentially broken driver. Use at your own risk.
>6. If the vendor doesn't get the CI working in the next release, then
>remove the driver.
>7. If the vendor gets the CI working again, then set the supported
>flag back to True and all is good.
>
>
> This allows a deprecation period for a driver, and keeps operators who
> upgrade their deployment from losing access to their volumes they have on
> those back-ends.  It will give them time to contact the community and/or do
> some research, and find out what happened to the driver.   This also
> potentially gives the operator time to find a new supported backend and
> start migrating volumes.  I say potentially, because the driver may be
> broken, or it may work enough to migrate volumes off of it to a new backend.
>
> Having unsupported drivers in tree is terrible for the Cinder community,
> and in the long run terrible for operators.
> Instantly removing drivers because CI is unstable is terrible for
> operators in the short term, because as soon as they upgrade OpenStack,
> they lose all access to managing their existing volumes.   Just because we
> leave a driver in tree in this state, doesn't mean that the operator will
> be able to migrate if the drive is broken, but they'll have a chance
> depending on the state of the driver in question.  It could be horribly
> broken, but the breakage might be something fixable by someone that just
> knows Python.   If the driver is gone from tree entirely, then that's a lot
> more to overcome.
>
> I don't think there is a way to make everyone happy all the time, but I
> 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Walter A. Boring IV]

I was leaning towards a separate repo until I started thinking about all
the overhead and complications this would cause. It's another repo for
cores to watch. It would cause everyone extra complication in setting up
their CI, which is already one of the biggest roadblocks. It would make
it a little harder to do things like https://review.openstack.org/297140
and https://review.openstack.org/346470 to be able to generate this:
http://docs.openstack.org/developer/cinder/drivers.html. Plus more infra
setup, more moving parts to break, and just generally more
complications.

All things that can be solved for sure. I just question whether it would
be worth having that overhead. Frankly, there are better things I'd like
to spend my time on.

I think at this point my first preference would actually be to define a
new tag. This addresses both the driver removal issue as well as the
backporting of driver bug fixes. I would like to see third party drivers
recognized and treated as being different, because in reality they are
very different than the rest of the code. Having something like
follows_deprecation_but_has_third_party_drivers_that_dont would make a
clear statement that their is a vendor component to this project that
really has to be treated differently and has different concerns
deployers need to be aware of.

Barring that, I think my next choice would be to remove the tag. That
would really be unfortunate as we do want to make it clear to users that
Cinder will not arbitrarily break APIs or do anything between releases
without warning when it comes to non-third party drivers. But if that is
what we need to do to effectively communicate what to expect from
Cinder, then I'm OK with that.

My last choice (of the ones I'm favorable towards) would be marking a
driver as untested/unstable/abandoned/etc rather than removing it. We
could flag these a certain way and have then spam the logs like crazy
after upgrade to make it very and painfully clear that they are not
being maintained. But as Duncan pointed out, this doesn't have as much
impact for getting vendor attention. It's amazing the level of executive
involvement that can happen after a patch is put up for driver removal
due to non-compliance.

Sean

__
I believe there is a compromise that we could implement in Cinder that 
enables us to have a deprecation
of unsupported drivers that aren't meeting the Cinder driver 
requirements and allow upgrades to work

without outright immediately removing a driver.

1. Add a 'supported = True' attribute to every driver.
2. When a driver no longer meets Cinder community requirements, put a
   patch up against the driver
3. When c-vol service starts, check the supported flag.  If the flag is
   False, then log an exception, and disable the driver.
4. Allow the admin to put an entry in cinder.conf for the driver in
   question "enable_unsupported_driver = True".  This will allow the
   c-vol service to start the driver and allow it to work.  Log a
   warning on every driver call.
5. This is a positive acknowledgement by the operator that they are
   enabling a potentially broken driver. Use at your own risk.
6. If the vendor doesn't get the CI working in the next release, then
   remove the driver.
7. If the vendor gets the CI working again, then set the supported flag
   back to True and all is good.


This allows a deprecation period for a driver, and keeps operators who 
upgrade their deployment from losing access to their volumes they have 
on those back-ends.  It will give them time to contact the community 
and/or do some research, and find out what happened to the driver.   
This also potentially gives the operator time to find a new supported 
backend and start migrating volumes.  I say potentially, because the 
driver may be broken, or it may work enough to migrate volumes off of it 
to a new backend.


Having unsupported drivers in tree is terrible for the Cinder community, 
and in the long run terrible for operators.
Instantly removing drivers because CI is unstable is terrible for 
operators in the short term, because as soon as they upgrade OpenStack, 
they lose all access to managing their existing volumes.   Just because 
we leave a driver in tree in this state, doesn't mean that the operator 
will be able to migrate if the drive is broken, but they'll have a 
chance depending on the state of the driver in question.  It could be 
horribly broken, but the breakage might be something fixable by someone 
that just knows Python.   If the driver is gone from tree entirely, then 
that's a lot more to overcome.


I don't think there is a way to make everyone happy all the time, but I 
think this buys operators a small window of opportunity to still manage 
their existing volumes before the driver is removed. It also still 
allows the Cinder community to deal with unsupported drivers in a way 
that will motivate vendors to keep their 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Clay Gerrard]

On Thu, Aug 11, 2016 at 7:14 AM, Erno Kuvaja  wrote:

>
> Lets say I was ops evaluating different options as storage vendor for
> my cloud and I get told that "Here is the list of supported drivers
> for different OpenStack Cinder back ends delivered by Cinder team", I
> start looking what the support level of those drivers are and see that
> Cinder follows standard deprecation which is fairly user/ops friendly
> with decent warning etc. I'm happy with that, not knowing OpenStack I
> would not even look if different subcomponents of Cinder happens to
> follow different policy. Now I buy storage vendor X HW and at Oct I
> realize that the vendor's driver is not shipped, nor any remains of it
> is visible anymore, I'd be reasonably pissed off. If I knew that the
> risk is there I would select my HW based on the negotiations that my
> HW is contractually tied to maintain that driver and it's CI, and that
> would be fine as well or if not possible I'd select some other
> solution I could get reasonably guarantee that it will be
> supported/valid at it's expected life time. As said I don't think
> there is anything wrong with the 3rd party driver policy, but
> maintaining that and the tag about standard-deprecation project wide
> is sending wrong message to those who do not know better to safeguard
> their rear ends.
>

Can we clarify if anyone is aware of this *actually* happening?Because
this description of events sounds *terrible*?  If we have a case-in-point I
think it'd be down right negligent to not give the situation a proper RCA,
but I'd be *real* curious to hear the previous "4 whys" that lead to
"ultimately; the problems was the tags..."

I'm much more inclined to think that we should trust the Cinder team to do
what they think is best based on their experience.  If their experience is
that it's better for their operators that they *not* ship "deprecated (but
probably broken)" drivers - GOOD FOR THEM!  I think it'd be great if the
"standard deprecation policy" can be informed and updated based on the
experience of a successful project like Cinder - if not, yeah I really hope
they continue to do the *right* thing over the *standard* thing.

OTOH, if what they think is right is causing *real* problems, let's surface
those - if they got to this policy based on experience, new information
will spur new ideas.  But that's different than some pontification based on
hypotheticals.  Speaking of which, why is this even coming up in the
*development* ML?

-Clay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Anita Kuno]

On 16-08-12 09:21 AM, Thierry Carrez wrote:

Sean Dague wrote:

I 100% understand the cinder policy of kicking drivers out without CI.
And I think there is a lot of value in ensuring what's in tree is tested.

However, from a user perspective basically it means that if you deploy
Newton cinder and build a storage infrastructure around anything other
than ceph, lvm, or NFS, you have a very real chance of never being able
to upgrade to Ocata, because your driver was fully deleted, unless you
are willing to completely change up your storage architecture during the
upgrade.

That is the kind of reality that should be front and center to the
users. Because it's not just a drop of standard deprecation, it's also a
removal of 'supports upgrade', as Netwon cinder config won't work with
Ocata.

Could there be more of an off ramp / on ramp here to the drivers? If a
driver CI fails to meet the reporting window mark it deprecated for the
next delete window. If a driver is in a deprecated state they need some
long window of continuous reporting to get out of that state (like 120
days or something). Bring in all new drivers in a
deprecated/experimental/untested state, which they only get to shrug off
after the onramp window?

It's definitely important that the project has the ability to clean out
the cruft, but it would be nice to not be overly brutal to our operators
at the same time.

And if not, I think that tags (or lack there of) aren't fully
communicating the situation here. Cinder docs should basically say "only
use ceph / lvm / nfs, as those are the only drivers that we can
guarantee will be in the next release".

+1

Both of the options (keeping cruft in tree vs. having no assurance at
all that your choice of driver will be around in 6 months) are horrible
from an operators standpoint. But I think that's a false dichotomy and
we need a more subtle solution: communicate about sane drivers where we
trust the ability of core team or the vendor to still provide a workable
solution in the next release (following standard deprecation policy)
while still being able to remove cruft if a driver goes stale /
untested. That means defining multiple tiers of trust, and having each
driver build that trust over time.


I think building trust over time is the crucial point here. I think we 
as a community have learned that in certain areas, there is a vast 
amount of clean up to be done by allotting trust prior to it being earned.


Giving folks an opportunity to earn trust, actual trust, not just gaming 
a process, enables everyone to be able to work together optimally. We 
have learned some folks are unwilling to do the work to earn it.


But some folks are worthy of trust and have demonstrated it. Thanks to 
those who have wanted that for themselves.


Thank you,
Anita.



In that other thread I proposed two tiers (in openstack/cinder following
deprecation and stable policies and in a separate Cinder repository if
you don't trust it to follow the policies) since the Cinder team sees
value in keeping them cinder-core-reviewed and in a limited number of
repositories.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[John Griffith]

On Fri, Aug 12, 2016 at 7:37 AM, Sean McGinnis 
wrote:

> On Fri, Aug 12, 2016 at 03:40:47PM +0300, Duncan Thomas wrote:
> > On 12 Aug 2016 15:28, "Thierry Carrez"  wrote:
> > >
> > > Duncan Thomas wrote:
> >
> > > I agree that leaving broken drivers in tree is not significantly better
> > > from an operational perspective. But I think the best operational
> > > experience would be to have an idea of how much risk you expose
> yourself
> > > when you pick a driver, and have a number of them that are actually
> > > /covered/ by the standard deprecation policy.
> > >
> > > So ideally there would be a number of in-tree drivers (on which the
> > > Cinder team would apply the standard deprecation policy), and a
> separate
> > > repository for 3rd-party drivers that can be removed at any time (and
> > > which would /not/ have the follows-standard-deprecation-policy tag).
> >
> > So we'd certainly have to move out all of the backends requiring
> > proprietary hardware, since we couldn't commit to keeping them working if
> > their vendors turn of their CI. That leaves ceph, lvm, NFS, drdb, and
> > sheepdog, I think. There is not enough broad knowledge in the core team
> > currently to support sheepdog or drdb without 'vendor' help. That would
> > leave us with three drivers in the tree, and not actually provide much
> > useful risk information to deployers at all.
> >
> > > I understand that this kind of reorganization is a bit painful for
> > > little (developer-side) gain, but I think it would provide the most
> > > useful information to our users and therefore the best operational
> > > experience...
> >
> > In theory this might be true, but see above - in practice it doesn't work
> > that way.
>
> I was leaning towards a separate repo until I started thinking about all
> the overhead and complications this would cause. It's another repo for
> cores to watch. It would cause everyone extra complication in setting up
> their CI, which is already one of the biggest roadblocks. It would make
> it a little harder to do things like https://review.openstack.org/297140
> and https://review.openstack.org/346470 to be able to generate this:
> http://docs.openstack.org/developer/cinder/drivers.html. Plus more infra
> setup, more moving parts to break, and just generally more
> complications.
>
> All things that can be solved for sure. I just question whether it would
> be worth having that overhead. Frankly, there are better things I'd like
> to spend my time on.
>
> I think at this point my first preference would actually be to define a
> new tag. This addresses both the driver removal issue as well as the
> backporting of driver bug fixes. I would like to see third party drivers
> recognized and treated as being different, because in reality they are
> very different than the rest of the code. Having something like
> follows_deprecation_but_has_third_party_drivers_that_dont would make a
> clear statement that their is a vendor component to this project that
> really has to be treated differently and has different concerns
> deployers need to be aware of.
>
> Barring that, I think my next choice would be to remove the tag. That
> would really be unfortunate as we do want to make it clear to users that
> Cinder will not arbitrarily break APIs or do anything between releases
> without warning when it comes to non-third party drivers. But if that is
> what we need to do to effectively communicate what to expect from
> Cinder, then I'm OK with that.
>
> My last choice (of the ones I'm favorable towards) would be marking a
> driver as untested/unstable/abandoned/etc rather than removing it. We
> could flag these a certain way and have then spam the logs like crazy
> after upgrade to make it very and painfully clear that they are not
> being maintained. But as Duncan pointed out, this doesn't have as much
> impact for getting vendor attention. It's amazing the level of executive
> involvement that can happen after a patch is put up for driver removal
> due to non-compliance.
>
> Sean
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
Yeah, I think something like a "passes-upstream-integration" tag per driver
would be a better option.  Whether that's collected via automation looking
at the gerrit info from 3'rd party CI or we bring back the old manual Cert
scripts (or some form of them) is another conversation worth having next
time we're all together.​  Now to try and agree on the criteria might be a
bit of work.

By going with a tag we don't remove anything but we also don't pretend that
we know it works or anything either.
​
 The statement suggesting that if it's not in the infra gate then it must
be considered as maybe not there in the future 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Sean McGinnis]

On Fri, Aug 12, 2016 at 03:40:47PM +0300, Duncan Thomas wrote:
> On 12 Aug 2016 15:28, "Thierry Carrez"  wrote:
> >
> > Duncan Thomas wrote:
> 
> > I agree that leaving broken drivers in tree is not significantly better
> > from an operational perspective. But I think the best operational
> > experience would be to have an idea of how much risk you expose yourself
> > when you pick a driver, and have a number of them that are actually
> > /covered/ by the standard deprecation policy.
> >
> > So ideally there would be a number of in-tree drivers (on which the
> > Cinder team would apply the standard deprecation policy), and a separate
> > repository for 3rd-party drivers that can be removed at any time (and
> > which would /not/ have the follows-standard-deprecation-policy tag).
> 
> So we'd certainly have to move out all of the backends requiring
> proprietary hardware, since we couldn't commit to keeping them working if
> their vendors turn of their CI. That leaves ceph, lvm, NFS, drdb, and
> sheepdog, I think. There is not enough broad knowledge in the core team
> currently to support sheepdog or drdb without 'vendor' help. That would
> leave us with three drivers in the tree, and not actually provide much
> useful risk information to deployers at all.
> 
> > I understand that this kind of reorganization is a bit painful for
> > little (developer-side) gain, but I think it would provide the most
> > useful information to our users and therefore the best operational
> > experience...
> 
> In theory this might be true, but see above - in practice it doesn't work
> that way.

I was leaning towards a separate repo until I started thinking about all
the overhead and complications this would cause. It's another repo for
cores to watch. It would cause everyone extra complication in setting up
their CI, which is already one of the biggest roadblocks. It would make
it a little harder to do things like https://review.openstack.org/297140
and https://review.openstack.org/346470 to be able to generate this:
http://docs.openstack.org/developer/cinder/drivers.html. Plus more infra
setup, more moving parts to break, and just generally more
complications.

All things that can be solved for sure. I just question whether it would
be worth having that overhead. Frankly, there are better things I'd like
to spend my time on.

I think at this point my first preference would actually be to define a
new tag. This addresses both the driver removal issue as well as the
backporting of driver bug fixes. I would like to see third party drivers
recognized and treated as being different, because in reality they are
very different than the rest of the code. Having something like
follows_deprecation_but_has_third_party_drivers_that_dont would make a
clear statement that their is a vendor component to this project that
really has to be treated differently and has different concerns
deployers need to be aware of.

Barring that, I think my next choice would be to remove the tag. That
would really be unfortunate as we do want to make it clear to users that
Cinder will not arbitrarily break APIs or do anything between releases
without warning when it comes to non-third party drivers. But if that is
what we need to do to effectively communicate what to expect from
Cinder, then I'm OK with that.

My last choice (of the ones I'm favorable towards) would be marking a
driver as untested/unstable/abandoned/etc rather than removing it. We
could flag these a certain way and have then spam the logs like crazy
after upgrade to make it very and painfully clear that they are not
being maintained. But as Duncan pointed out, this doesn't have as much
impact for getting vendor attention. It's amazing the level of executive
involvement that can happen after a patch is put up for driver removal
due to non-compliance.

Sean

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Thierry Carrez]

Sean Dague wrote:
> I 100% understand the cinder policy of kicking drivers out without CI.
> And I think there is a lot of value in ensuring what's in tree is tested.
> 
> However, from a user perspective basically it means that if you deploy
> Newton cinder and build a storage infrastructure around anything other
> than ceph, lvm, or NFS, you have a very real chance of never being able
> to upgrade to Ocata, because your driver was fully deleted, unless you
> are willing to completely change up your storage architecture during the
> upgrade.
> 
> That is the kind of reality that should be front and center to the
> users. Because it's not just a drop of standard deprecation, it's also a
> removal of 'supports upgrade', as Netwon cinder config won't work with
> Ocata.
> 
> Could there be more of an off ramp / on ramp here to the drivers? If a
> driver CI fails to meet the reporting window mark it deprecated for the
> next delete window. If a driver is in a deprecated state they need some
> long window of continuous reporting to get out of that state (like 120
> days or something). Bring in all new drivers in a
> deprecated/experimental/untested state, which they only get to shrug off
> after the onramp window?
> 
> It's definitely important that the project has the ability to clean out
> the cruft, but it would be nice to not be overly brutal to our operators
> at the same time.
> 
> And if not, I think that tags (or lack there of) aren't fully
> communicating the situation here. Cinder docs should basically say "only
> use ceph / lvm / nfs, as those are the only drivers that we can
> guarantee will be in the next release".

+1

Both of the options (keeping cruft in tree vs. having no assurance at
all that your choice of driver will be around in 6 months) are horrible
from an operators standpoint. But I think that's a false dichotomy and
we need a more subtle solution: communicate about sane drivers where we
trust the ability of core team or the vendor to still provide a workable
solution in the next release (following standard deprecation policy)
while still being able to remove cruft if a driver goes stale /
untested. That means defining multiple tiers of trust, and having each
driver build that trust over time.

In that other thread I proposed two tiers (in openstack/cinder following
deprecation and stable policies and in a separate Cinder repository if
you don't trust it to follow the policies) since the Cinder team sees
value in keeping them cinder-core-reviewed and in a limited number of
repositories.

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Duncan Thomas]

Strictly speaking, we only guarantee lvm... If any other driver starts
failing CI and nobody steps up to fix it then it wool be removed. I listed
ceph and NFS because I think there's enough knowledge and interest in the
core team to keep them working without needing any particular company to
help out.

We could make the windows larger as you suggest, but experience has shown
that this just causes vendors to make CI less of a priority, and
realistically we are already struggling to get meaningful results from CI.

If we remove a driver, it is highly likely that a forward port from the
previous release is trivial. Anybody building a deployment without some
sort of contracted driver commitment from their storage vendor is probably
doing themselves and the community a disservice though.

120 days of broken CI probably means we're shipping broken code, and I'm
not sure tags help most deployers - we have a lot of them, and the fine
details of their meaning is not obvious. There's only one tag appropriate
in prestige for a driver that has no passing CI - BROKEN. Shipping broken
code does not help anybody who's trying to rely on it, even during upgrade.
We might as well be honest and force them to do the forward port. If we
leave the broken driver in and they upgrade and everything breaks, it just
makes cinder look broken, without putting the blame squarely where it
belong - with the storage vendor who hadn't kept up the support for their
product. Giving a fake façade of 'support' just allows vendors to sell more
unsupported stuff, it doesn't help users, OpenStack developers or operators.

We could split and the drivers out into a new tree and give it different
tags, but it would slow down development, and frankly we've enough problems
on that front already. As far as I can tell, we (the cinder team) are
better off shrugging about the tags and carrying on as we are.

On 12 Aug 2016 15:54, "Sean Dague"  wrote:

> On 08/12/2016 08:40 AM, Duncan Thomas wrote:
> > On 12 Aug 2016 15:28, "Thierry Carrez"  > > wrote:
> >>
> >> Duncan Thomas wrote:
> >
> >> I agree that leaving broken drivers in tree is not significantly better
> >> from an operational perspective. But I think the best operational
> >> experience would be to have an idea of how much risk you expose yourself
> >> when you pick a driver, and have a number of them that are actually
> >> /covered/ by the standard deprecation policy.
> >>
> >> So ideally there would be a number of in-tree drivers (on which the
> >> Cinder team would apply the standard deprecation policy), and a separate
> >> repository for 3rd-party drivers that can be removed at any time (and
> >> which would /not/ have the follows-standard-deprecation-policy tag).
> >
> > So we'd certainly have to move out all of the backends requiring
> > proprietary hardware, since we couldn't commit to keeping them working
> > if their vendors turn of their CI. That leaves ceph, lvm, NFS, drdb, and
> > sheepdog, I think. There is not enough broad knowledge in the core team
> > currently to support sheepdog or drdb without 'vendor' help. That would
> > leave us with three drivers in the tree, and not actually provide much
> > useful risk information to deployers at all.
>
> I 100% understand the cinder policy of kicking drivers out without CI.
> And I think there is a lot of value in ensuring what's in tree is tested.
>
> However, from a user perspective basically it means that if you deploy
> Newton cinder and build a storage infrastructure around anything other
> than ceph, lvm, or NFS, you have a very real chance of never being able
> to upgrade to Ocata, because your driver was fully deleted, unless you
> are willing to completely change up your storage architecture during the
> upgrade.
>
> That is the kind of reality that should be front and center to the
> users. Because it's not just a drop of standard deprecation, it's also a
> removal of 'supports upgrade', as Netwon cinder config won't work with
> Ocata.
>
> Could there be more of an off ramp / on ramp here to the drivers? If a
> driver CI fails to meet the reporting window mark it deprecated for the
> next delete window. If a driver is in a deprecated state they need some
> long window of continuous reporting to get out of that state (like 120
> days or something). Bring in all new drivers in a
> deprecated/experimental/untested state, which they only get to shrug off
> after the onramp window?
>
> It's definitely important that the project has the ability to clean out
> the cruft, but it would be nice to not be overly brutal to our operators
> at the same time.
>
> And if not, I think that tags (or lack there of) aren't fully
> communicating the situation here. Cinder docs should basically say "only
> use ceph / lvm / nfs, as those are the only drivers that we can
> guarantee will be in the next release".
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Sean Dague]

On 08/12/2016 08:40 AM, Duncan Thomas wrote:
> On 12 Aug 2016 15:28, "Thierry Carrez"  > wrote:
>>
>> Duncan Thomas wrote:
> 
>> I agree that leaving broken drivers in tree is not significantly better
>> from an operational perspective. But I think the best operational
>> experience would be to have an idea of how much risk you expose yourself
>> when you pick a driver, and have a number of them that are actually
>> /covered/ by the standard deprecation policy.
>>
>> So ideally there would be a number of in-tree drivers (on which the
>> Cinder team would apply the standard deprecation policy), and a separate
>> repository for 3rd-party drivers that can be removed at any time (and
>> which would /not/ have the follows-standard-deprecation-policy tag).
> 
> So we'd certainly have to move out all of the backends requiring
> proprietary hardware, since we couldn't commit to keeping them working
> if their vendors turn of their CI. That leaves ceph, lvm, NFS, drdb, and
> sheepdog, I think. There is not enough broad knowledge in the core team
> currently to support sheepdog or drdb without 'vendor' help. That would
> leave us with three drivers in the tree, and not actually provide much
> useful risk information to deployers at all.

I 100% understand the cinder policy of kicking drivers out without CI.
And I think there is a lot of value in ensuring what's in tree is tested.

However, from a user perspective basically it means that if you deploy
Newton cinder and build a storage infrastructure around anything other
than ceph, lvm, or NFS, you have a very real chance of never being able
to upgrade to Ocata, because your driver was fully deleted, unless you
are willing to completely change up your storage architecture during the
upgrade.

That is the kind of reality that should be front and center to the
users. Because it's not just a drop of standard deprecation, it's also a
removal of 'supports upgrade', as Netwon cinder config won't work with
Ocata.

Could there be more of an off ramp / on ramp here to the drivers? If a
driver CI fails to meet the reporting window mark it deprecated for the
next delete window. If a driver is in a deprecated state they need some
long window of continuous reporting to get out of that state (like 120
days or something). Bring in all new drivers in a
deprecated/experimental/untested state, which they only get to shrug off
after the onramp window?

It's definitely important that the project has the ability to clean out
the cruft, but it would be nice to not be overly brutal to our operators
at the same time.

And if not, I think that tags (or lack there of) aren't fully
communicating the situation here. Cinder docs should basically say "only
use ceph / lvm / nfs, as those are the only drivers that we can
guarantee will be in the next release".

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Duncan Thomas]

On 12 Aug 2016 15:28, "Thierry Carrez"  wrote:
>
> Duncan Thomas wrote:

> I agree that leaving broken drivers in tree is not significantly better
> from an operational perspective. But I think the best operational
> experience would be to have an idea of how much risk you expose yourself
> when you pick a driver, and have a number of them that are actually
> /covered/ by the standard deprecation policy.
>
> So ideally there would be a number of in-tree drivers (on which the
> Cinder team would apply the standard deprecation policy), and a separate
> repository for 3rd-party drivers that can be removed at any time (and
> which would /not/ have the follows-standard-deprecation-policy tag).

So we'd certainly have to move out all of the backends requiring
proprietary hardware, since we couldn't commit to keeping them working if
their vendors turn of their CI. That leaves ceph, lvm, NFS, drdb, and
sheepdog, I think. There is not enough broad knowledge in the core team
currently to support sheepdog or drdb without 'vendor' help. That would
leave us with three drivers in the tree, and not actually provide much
useful risk information to deployers at all.

> I understand that this kind of reorganization is a bit painful for
> little (developer-side) gain, but I think it would provide the most
> useful information to our users and therefore the best operational
> experience...

In theory this might be true, but see above - in practice it doesn't work
that way.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Thierry Carrez]

Duncan Thomas wrote:
> Given the options, I'd agree with Sean and John that removing the tag is
> a far lesser evil than changing our policy.

Agree that the tag should be removed. The Cinder core certainly follows
standard deprecation policy, but the Cinder drivers are not.

> If we leave broken drivers in the tree, the end user (operator) is no
> better off - the thing they evaluated won't work - but it will be harder
> to tell why. The storage vendor won't suffer the pressure that comes
> from driver removal, so will have less incentive to fix their driver
> (there's enough examples of the threat of driver removal causing the
> immediate fix of things that have remained broken for months that we
> know, for certain that the policy works).

I agree that leaving broken drivers in tree is not significantly better
from an operational perspective. But I think the best operational
experience would be to have an idea of how much risk you expose yourself
when you pick a driver, and have a number of them that are actually
/covered/ by the standard deprecation policy.

So ideally there would be a number of in-tree drivers (on which the
Cinder team would apply the standard deprecation policy), and a separate
repository for 3rd-party drivers that can be removed at any time (and
which would /not/ have the follows-standard-deprecation-policy tag).

I understand that this kind of reorganization is a bit painful for
little (developer-side) gain, but I think it would provide the most
useful information to our users and therefore the best operational
experience...

-- 
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Duncan Thomas]

Given the options, I'd agree with Sean and John that removing the tag is a
far lesser evil than changing our policy.

If we leave broken drivers in the tree, the end user (operator) is no
better off - the thing they evaluated won't work - but it will be harder to
tell why. The storage vendor won't suffer the pressure that comes from
driver removal, so will have less incentive to fix their driver (there's
enough examples of the threat of driver removal causing the immediate fix
of things that have remained broken for months that we know, for certain
that the policy works).

I'd prefer to make the meaning of the tag sane WRT third party drivers,
which I think would help other projects to be able to police their drivers
and CI better too, without risking losing / not gaining the tag, which is
likely to hurt a newer project far more than it will cinder.



On 11 August 2016 at 17:29, John Griffith  wrote:

>
>
> On Thu, Aug 11, 2016 at 7:14 AM, Erno Kuvaja  wrote:
>
>> On Thu, Aug 11, 2016 at 2:47 PM, Sean McGinnis 
>> wrote:
>> >> >>
>> >> >> As follow up on the mailing list discussion [0], gerrit activity
>> >> >> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
>> >> >> discussion how Cinder follows, or rather does not follow, the
>> standard
>> >> >> deprecation policy [4] as the project has been tagged on the assert
>> >> >> page [5].
>> >> >>
>> > 
>> >> >>
>> >> >> [0] http://lists.openstack.org/pipermail/openstack-dev/2016-Augu
>> st/100717.html
>> >> >> [1] https://review.openstack.org/#/c/348032/
>> >> >> [2] https://review.openstack.org/#/c/348042/
>> >> >> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
>> >> >> [4] https://governance.openstack.org/reference/tags/assert_follo
>> ws-standard-deprecation.html#requirements
>> >> >> [5] https://governance.openstack.org/reference/tags/assert_follo
>> ws-standard-deprecation.html#application-to-current-projects
>> >> >>
>> >> >
>> >> > Can you be more specific about what you mean? Are you saying that
>> >> > the policy isn't being followed because the drivers were removed
>> >> > without a deprecation period, or is there something else to it?
>> >> >
>> >> > Doug
>> >> >
>> >>
>> >> Yes, that's how I see it. Cinder's own policy is that the drivers can
>> >> be removed without any warning to the consumers while the standard
>> >> deprecation policy defines quite strict lines about informing the
>> >> consumer of the functionality deprecation before it gets removed.
>> >>
>> >> - Erno
>> >
>> > It is a good point. I think it highlights a common thread though with
>> > the other discussion that, at least so far, third party drivers are
>> > treated differently than the rest of the code.
>> >
>> > For any other functionality we certainly follow the deprecation policy.
>> > Even in existing drivers we try to enforce that any driver renames,
>> > config setting changes, and similar non-backwards compatible changes go
>> > through the normal deprecation cycle before being removed.
>> >
>> > Ideally I would love it if we could comply with the deprecation policy
>> > with regards to driver removal. But the reality is, if we don't see that
>> > a driver is being supported and maintained by its vendor, then that
>> > burden can't fall on the wider OpenStack and Cinder community that has
>> > no way of validating against physical hardware.
>> >
>> > I think third party drivers need to be treated differently when it comes
>> > to the deprecation policy. If that is not acceptable, then I suppose we
>> > do need to remove that tag. Tag removal would be the lesser of the two
>> > versus keeping around drivers that we know aren't really being
>> > maintained.
>> >
>> > If it came to that, I would also consider creating a new cinder-drivers
>> > project under the Cinder umbrella and move all of the drivers not tested
>> > by Jenkins over to that. That wouldn't be a trivial undertaking, so I
>> > would try to avoid that if possible. But it would at least allow us to
>> > still get code reviews and all of the benefits of being in tree. Just
>> > some thoughts.
>> >
>> > Sean
>> >
>>
>> Sean,
>>
>> As said on my initial opening, I do understand and agree with the
>> reasoning/treatment of the 3rd party drivers. My request for that tag
>> removal is out of the remains of my ops hat.
>>
>> Lets say I was ops evaluating different options as storage vendor for
>> my cloud and I get told that "Here is the list of supported drivers
>> for different OpenStack Cinder back ends delivered by Cinder team", I
>> start looking what the support level of those drivers are and see that
>> Cinder follows standard deprecation which is fairly user/ops friendly
>> with decent warning etc. I'm happy with that, not knowing OpenStack I
>> would not even look if different subcomponents of Cinder happens to
>> follow different policy. Now I buy storage vendor X HW and at Oct I
>> realize that the 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Hayes, Graham]

On 11/08/2016 15:35, John Griffith wrote:
>
>
> On Thu, Aug 11, 2016 at 7:14 AM, Erno Kuvaja  > wrote:
>
> On Thu, Aug 11, 2016 at 2:47 PM, Sean McGinnis
> > wrote:
> >> >>
> >> >> As follow up on the mailing list discussion [0], gerrit activity
> >> >> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
> >> >> discussion how Cinder follows, or rather does not follow, the
> standard
> >> >> deprecation policy [4] as the project has been tagged on the
> assert
> >> >> page [5].
> >> >>
> > 
> >> >>
> >> >> [0]
> http://lists.openstack.org/pipermail/openstack-dev/2016-August/100717.html
> 
> 
> >> >> [1] https://review.openstack.org/#/c/348032/
> 
> >> >> [2] https://review.openstack.org/#/c/348042/
> 
> >> >> [3]
> https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
> 
> >> >> [4]
> 
> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#requirements
> 
> 
> >> >> [5]
> 
> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#application-to-current-projects
> 
> 
> >> >>
> >> >
> >> > Can you be more specific about what you mean? Are you saying that
> >> > the policy isn't being followed because the drivers were removed
> >> > without a deprecation period, or is there something else to it?
> >> >
> >> > Doug
> >> >
> >>
> >> Yes, that's how I see it. Cinder's own policy is that the drivers can
> >> be removed without any warning to the consumers while the standard
> >> deprecation policy defines quite strict lines about informing the
> >> consumer of the functionality deprecation before it gets removed.
> >>
> >> - Erno
> >
> > It is a good point. I think it highlights a common thread though with
> > the other discussion that, at least so far, third party drivers are
> > treated differently than the rest of the code.
> >
> > For any other functionality we certainly follow the deprecation
> policy.
> > Even in existing drivers we try to enforce that any driver renames,
> > config setting changes, and similar non-backwards compatible
> changes go
> > through the normal deprecation cycle before being removed.
> >
> > Ideally I would love it if we could comply with the deprecation policy
> > with regards to driver removal. But the reality is, if we don't
> see that
> > a driver is being supported and maintained by its vendor, then that
> > burden can't fall on the wider OpenStack and Cinder community that has
> > no way of validating against physical hardware.
> >
> > I think third party drivers need to be treated differently when it
> comes
> > to the deprecation policy. If that is not acceptable, then I
> suppose we
> > do need to remove that tag. Tag removal would be the lesser of the two
> > versus keeping around drivers that we know aren't really being
> > maintained.
> >
> > If it came to that, I would also consider creating a new
> cinder-drivers
> > project under the Cinder umbrella and move all of the drivers not
> tested
> > by Jenkins over to that. That wouldn't be a trivial undertaking, so I
> > would try to avoid that if possible. But it would at least allow us to
> > still get code reviews and all of the benefits of being in tree. Just
> > some thoughts.
> >
> > Sean
> >
>
> Sean,
>
> As said on my initial opening, I do understand and agree with the
> reasoning/treatment of the 3rd party drivers. My request for that tag
> removal is out of the remains of my ops hat.
>
> Lets say I was ops evaluating different options as storage vendor for
> my cloud and I get told that "Here is the list of supported drivers
> for different OpenStack Cinder back ends delivered by Cinder team", I
> start looking what the support level of those drivers are and see that
> Cinder follows standard deprecation which is fairly user/ops friendly
> with decent warning etc. I'm happy with that, not knowing OpenStack I
> would not even look if different subcomponents of Cinder happens to
> follow different policy. Now I buy storage vendor X HW and at Oct I
> realize that the vendor's driver is not shipped, nor any 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[John Griffith]

On Thu, Aug 11, 2016 at 7:14 AM, Erno Kuvaja  wrote:

> On Thu, Aug 11, 2016 at 2:47 PM, Sean McGinnis 
> wrote:
> >> >>
> >> >> As follow up on the mailing list discussion [0], gerrit activity
> >> >> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
> >> >> discussion how Cinder follows, or rather does not follow, the
> standard
> >> >> deprecation policy [4] as the project has been tagged on the assert
> >> >> page [5].
> >> >>
> > 
> >> >>
> >> >> [0] http://lists.openstack.org/pipermail/openstack-dev/2016-
> August/100717.html
> >> >> [1] https://review.openstack.org/#/c/348032/
> >> >> [2] https://review.openstack.org/#/c/348042/
> >> >> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
> >> >> [4] https://governance.openstack.org/reference/tags/assert_
> follows-standard-deprecation.html#requirements
> >> >> [5] https://governance.openstack.org/reference/tags/assert_
> follows-standard-deprecation.html#application-to-current-projects
> >> >>
> >> >
> >> > Can you be more specific about what you mean? Are you saying that
> >> > the policy isn't being followed because the drivers were removed
> >> > without a deprecation period, or is there something else to it?
> >> >
> >> > Doug
> >> >
> >>
> >> Yes, that's how I see it. Cinder's own policy is that the drivers can
> >> be removed without any warning to the consumers while the standard
> >> deprecation policy defines quite strict lines about informing the
> >> consumer of the functionality deprecation before it gets removed.
> >>
> >> - Erno
> >
> > It is a good point. I think it highlights a common thread though with
> > the other discussion that, at least so far, third party drivers are
> > treated differently than the rest of the code.
> >
> > For any other functionality we certainly follow the deprecation policy.
> > Even in existing drivers we try to enforce that any driver renames,
> > config setting changes, and similar non-backwards compatible changes go
> > through the normal deprecation cycle before being removed.
> >
> > Ideally I would love it if we could comply with the deprecation policy
> > with regards to driver removal. But the reality is, if we don't see that
> > a driver is being supported and maintained by its vendor, then that
> > burden can't fall on the wider OpenStack and Cinder community that has
> > no way of validating against physical hardware.
> >
> > I think third party drivers need to be treated differently when it comes
> > to the deprecation policy. If that is not acceptable, then I suppose we
> > do need to remove that tag. Tag removal would be the lesser of the two
> > versus keeping around drivers that we know aren't really being
> > maintained.
> >
> > If it came to that, I would also consider creating a new cinder-drivers
> > project under the Cinder umbrella and move all of the drivers not tested
> > by Jenkins over to that. That wouldn't be a trivial undertaking, so I
> > would try to avoid that if possible. But it would at least allow us to
> > still get code reviews and all of the benefits of being in tree. Just
> > some thoughts.
> >
> > Sean
> >
>
> Sean,
>
> As said on my initial opening, I do understand and agree with the
> reasoning/treatment of the 3rd party drivers. My request for that tag
> removal is out of the remains of my ops hat.
>
> Lets say I was ops evaluating different options as storage vendor for
> my cloud and I get told that "Here is the list of supported drivers
> for different OpenStack Cinder back ends delivered by Cinder team", I
> start looking what the support level of those drivers are and see that
> Cinder follows standard deprecation which is fairly user/ops friendly
> with decent warning etc. I'm happy with that, not knowing OpenStack I
> would not even look if different subcomponents of Cinder happens to
> follow different policy. Now I buy storage vendor X HW and at Oct I
> realize that the vendor's driver is not shipped, nor any remains of it
> is visible anymore, I'd be reasonably pissed off. If I knew that the
> risk is there I would select my HW based on the negotiations that my
> HW is contractually tied to maintain that driver and it's CI, and that
> would be fine as well or if not possible I'd select some other
> solution I could get reasonably guarantee that it will be
> supported/valid at it's expected life time. As said I don't think
> there is anything wrong with the 3rd party driver policy, but
> maintaining that and the tag about standard-deprecation project wide
> is sending wrong message to those who do not know better to safeguard
> their rear ends.
>
> The other option would be to leave the drivers in tree, tag them with
> deprecation message, something like "This driver has not been tested
> by vendor CI since 15.3.2016 and cannot be guaranteed working. Unless
> testing will be resumed the driver will be removed on Unicorn
> release". Which would give as clear indication that the driver 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Erno Kuvaja]

On Thu, Aug 11, 2016 at 2:47 PM, Sean McGinnis  wrote:
>> >>
>> >> As follow up on the mailing list discussion [0], gerrit activity
>> >> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
>> >> discussion how Cinder follows, or rather does not follow, the standard
>> >> deprecation policy [4] as the project has been tagged on the assert
>> >> page [5].
>> >>
> 
>> >>
>> >> [0] 
>> >> http://lists.openstack.org/pipermail/openstack-dev/2016-August/100717.html
>> >> [1] https://review.openstack.org/#/c/348032/
>> >> [2] https://review.openstack.org/#/c/348042/
>> >> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
>> >> [4] 
>> >> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#requirements
>> >> [5] 
>> >> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#application-to-current-projects
>> >>
>> >
>> > Can you be more specific about what you mean? Are you saying that
>> > the policy isn't being followed because the drivers were removed
>> > without a deprecation period, or is there something else to it?
>> >
>> > Doug
>> >
>>
>> Yes, that's how I see it. Cinder's own policy is that the drivers can
>> be removed without any warning to the consumers while the standard
>> deprecation policy defines quite strict lines about informing the
>> consumer of the functionality deprecation before it gets removed.
>>
>> - Erno
>
> It is a good point. I think it highlights a common thread though with
> the other discussion that, at least so far, third party drivers are
> treated differently than the rest of the code.
>
> For any other functionality we certainly follow the deprecation policy.
> Even in existing drivers we try to enforce that any driver renames,
> config setting changes, and similar non-backwards compatible changes go
> through the normal deprecation cycle before being removed.
>
> Ideally I would love it if we could comply with the deprecation policy
> with regards to driver removal. But the reality is, if we don't see that
> a driver is being supported and maintained by its vendor, then that
> burden can't fall on the wider OpenStack and Cinder community that has
> no way of validating against physical hardware.
>
> I think third party drivers need to be treated differently when it comes
> to the deprecation policy. If that is not acceptable, then I suppose we
> do need to remove that tag. Tag removal would be the lesser of the two
> versus keeping around drivers that we know aren't really being
> maintained.
>
> If it came to that, I would also consider creating a new cinder-drivers
> project under the Cinder umbrella and move all of the drivers not tested
> by Jenkins over to that. That wouldn't be a trivial undertaking, so I
> would try to avoid that if possible. But it would at least allow us to
> still get code reviews and all of the benefits of being in tree. Just
> some thoughts.
>
> Sean
>

Sean,

As said on my initial opening, I do understand and agree with the
reasoning/treatment of the 3rd party drivers. My request for that tag
removal is out of the remains of my ops hat.

Lets say I was ops evaluating different options as storage vendor for
my cloud and I get told that "Here is the list of supported drivers
for different OpenStack Cinder back ends delivered by Cinder team", I
start looking what the support level of those drivers are and see that
Cinder follows standard deprecation which is fairly user/ops friendly
with decent warning etc. I'm happy with that, not knowing OpenStack I
would not even look if different subcomponents of Cinder happens to
follow different policy. Now I buy storage vendor X HW and at Oct I
realize that the vendor's driver is not shipped, nor any remains of it
is visible anymore, I'd be reasonably pissed off. If I knew that the
risk is there I would select my HW based on the negotiations that my
HW is contractually tied to maintain that driver and it's CI, and that
would be fine as well or if not possible I'd select some other
solution I could get reasonably guarantee that it will be
supported/valid at it's expected life time. As said I don't think
there is anything wrong with the 3rd party driver policy, but
maintaining that and the tag about standard-deprecation project wide
is sending wrong message to those who do not know better to safeguard
their rear ends.

The other option would be to leave the drivers in tree, tag them with
deprecation message, something like "This driver has not been tested
by vendor CI since 15.3.2016 and cannot be guaranteed working. Unless
testing will be resumed the driver will be removed on Unicorn
release". Which would give as clear indication that the driver seems
abandoned, but still provide the consumer easier way to test in their
staging if the driver is something they still dare to use in
production or not.

IMHO this is purely to set the expectations right for our consumers so
that they know what 

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Sean McGinnis]

> >>
> >> As follow up on the mailing list discussion [0], gerrit activity
> >> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
> >> discussion how Cinder follows, or rather does not follow, the standard
> >> deprecation policy [4] as the project has been tagged on the assert
> >> page [5].
> >>

> >>
> >> [0] 
> >> http://lists.openstack.org/pipermail/openstack-dev/2016-August/100717.html
> >> [1] https://review.openstack.org/#/c/348032/
> >> [2] https://review.openstack.org/#/c/348042/
> >> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
> >> [4] 
> >> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#requirements
> >> [5] 
> >> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#application-to-current-projects
> >>
> >
> > Can you be more specific about what you mean? Are you saying that
> > the policy isn't being followed because the drivers were removed
> > without a deprecation period, or is there something else to it?
> >
> > Doug
> >
> 
> Yes, that's how I see it. Cinder's own policy is that the drivers can
> be removed without any warning to the consumers while the standard
> deprecation policy defines quite strict lines about informing the
> consumer of the functionality deprecation before it gets removed.
> 
> - Erno

It is a good point. I think it highlights a common thread though with
the other discussion that, at least so far, third party drivers are
treated differently than the rest of the code.

For any other functionality we certainly follow the deprecation policy.
Even in existing drivers we try to enforce that any driver renames,
config setting changes, and similar non-backwards compatible changes go
through the normal deprecation cycle before being removed.

Ideally I would love it if we could comply with the deprecation policy
with regards to driver removal. But the reality is, if we don't see that
a driver is being supported and maintained by its vendor, then that
burden can't fall on the wider OpenStack and Cinder community that has
no way of validating against physical hardware.

I think third party drivers need to be treated differently when it comes
to the deprecation policy. If that is not acceptable, then I suppose we
do need to remove that tag. Tag removal would be the lesser of the two
versus keeping around drivers that we know aren't really being
maintained.

If it came to that, I would also consider creating a new cinder-drivers
project under the Cinder umbrella and move all of the drivers not tested
by Jenkins over to that. That wouldn't be a trivial undertaking, so I
would try to avoid that if possible. But it would at least allow us to
still get code reviews and all of the benefits of being in tree. Just
some thoughts.

Sean

> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Erno Kuvaja]

On Thu, Aug 11, 2016 at 1:32 PM, Doug Hellmann  wrote:
> Excerpts from Erno Kuvaja's message of 2016-08-11 12:26:59 +0100:
>> Hi all,
>>
>> As follow up on the mailing list discussion [0], gerrit activity
>> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
>> discussion how Cinder follows, or rather does not follow, the standard
>> deprecation policy [4] as the project has been tagged on the assert
>> page [5].
>>
>> I'm not here to argue about the justification or rightfulness of the
>> Cinder policy regarding the drivers with 3rd party CI. (quite frankly
>> I think the Cinder policy makes lots of sense and thus propose the
>> removal of the tag rather than change of that policy) Just stating the
>> obvious that the Cinder policy does not comply with the
>> follows-standard-deprecation and thus the tag should be removed from
>> Cinder.
>>
>> Best,
>> Erno (jokke) Kuvaja
>>
>> [0] 
>> http://lists.openstack.org/pipermail/openstack-dev/2016-August/100717.html
>> [1] https://review.openstack.org/#/c/348032/
>> [2] https://review.openstack.org/#/c/348042/
>> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
>> [4] 
>> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#requirements
>> [5] 
>> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#application-to-current-projects
>>
>
> Can you be more specific about what you mean? Are you saying that
> the policy isn't being followed because the drivers were removed
> without a deprecation period, or is there something else to it?
>
> Doug
>

Yes, that's how I see it. Cinder's own policy is that the drivers can
be removed without any warning to the consumers while the standard
deprecation policy defines quite strict lines about informing the
consumer of the functionality deprecation before it gets removed.

- Erno

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][cinder] tag:follows-standard-deprecation should be removed

[Doug Hellmann]

Excerpts from Erno Kuvaja's message of 2016-08-11 12:26:59 +0100:
> Hi all,
> 
> As follow up on the mailing list discussion [0], gerrit activity
> [1][2] and cinder 3rd party CI policy [3] I'd like to initiate
> discussion how Cinder follows, or rather does not follow, the standard
> deprecation policy [4] as the project has been tagged on the assert
> page [5].
> 
> I'm not here to argue about the justification or rightfulness of the
> Cinder policy regarding the drivers with 3rd party CI. (quite frankly
> I think the Cinder policy makes lots of sense and thus propose the
> removal of the tag rather than change of that policy) Just stating the
> obvious that the Cinder policy does not comply with the
> follows-standard-deprecation and thus the tag should be removed from
> Cinder.
> 
> Best,
> Erno (jokke) Kuvaja
> 
> [0] http://lists.openstack.org/pipermail/openstack-dev/2016-August/100717.html
> [1] https://review.openstack.org/#/c/348032/
> [2] https://review.openstack.org/#/c/348042/
> [3] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers
> [4] 
> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#requirements
> [5] 
> https://governance.openstack.org/reference/tags/assert_follows-standard-deprecation.html#application-to-current-projects
> 

Can you be more specific about what you mean? Are you saying that
the policy isn't being followed because the drivers were removed
without a deprecation period, or is there something else to it?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev