Re: [openstack-dev] backwards compatibility followup

[Brian Demers]

On Tue, Apr 26, 2016 at 10:55 PM, Robert Collins 
wrote:

> On 26 April 2016 at 18:41, Ian Cordasco  wrote:
> >
> >> On Apr 26, 2016, at 6:32 PM, Perry, Sean  wrote:
> >>
> >> What if we update the docs and tell people to put any services on a
> shared system into independent virtualenvs?
> >>
> >> If a box only runs neutron or whatever all is well. The issue happens
> when you mix and match services on a single host. Which is exactly what
> virtualenv was intended to fix.
> >
> > Except that a fair portion of respondents to the OpenStack User Survey
> talk about using packages provided by the distribution of Linux they happen
> to be on. Those cannot be installed into virtual environments.
>
> Right. The specific situation is 'same python environment' - so distro
> packages all land in the same python environment. You can make
> packages of virtualenvs and other such things - and folk wanting to be
> able to do same-node-no-container-distro-package in-place upgrade
> solutions would be looking at such things if we decide we don't want
> to support/enable this.
>
> Some distros were in the room, and e.g. I believe it was Dan that said
> RH don't support - in their packaging - mixed versions: upgrade nova,
> and neutron would be upgraded too in the given example. So its
> possible that while some folk aspire to it, many other folk are either
> accepting downtime, or running their control plane on isolated
> operating system instances (whether physical, VM or containers is
> irrelevant).
>
> At the party tonight I ran into Brian Demers and a colleague whose
> name I have forgotten :(

Sam Betts

> - but they have a backwards compat use case
> we hadn't touched on in the session: running their Neutron plugin
> against both stable and master of Neutron.
>
> This is a classic colocation situation. There are a few possible scenarios:
>  - run stable branches of the plugin and backport everything
> - tricky if new oslo features are needed, but the existing pattern.
> - also tricky for users, since there would be lots of releases of
> both stable and master
>  - run master against stable neutron using stable neutron oslo
>  - means they can't use any new oslo features, and they would
> depend on oslo not breaking any API's they use in master - so depends
> on oslo API compat
>  - or they have to provide backports within their tree and monkey
> patch them into place, of new/changed things from oslo
>  - run master against stable neutron using master oslo
> - means neutron would have to work with master oslo - so the same
> compat story, just from the other side
>

Thanks for following up Rob.

Related to this, (and not to steer this off on a tangent) is the frequency
of major version changes in libraries, more specifically API breaking
changes.

The items mentioned in this thread would be difficult at best unless we
have stable [backwards compatible] APIs.

-Brian
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] backwards compatibility followup

[Robert Collins]

On 26 April 2016 at 18:41, Ian Cordasco  wrote:
>
>> On Apr 26, 2016, at 6:32 PM, Perry, Sean  wrote:
>>
>> What if we update the docs and tell people to put any services on a shared 
>> system into independent virtualenvs?
>>
>> If a box only runs neutron or whatever all is well. The issue happens when 
>> you mix and match services on a single host. Which is exactly what 
>> virtualenv was intended to fix.
>
> Except that a fair portion of respondents to the OpenStack User Survey talk 
> about using packages provided by the distribution of Linux they happen to be 
> on. Those cannot be installed into virtual environments.

Right. The specific situation is 'same python environment' - so distro
packages all land in the same python environment. You can make
packages of virtualenvs and other such things - and folk wanting to be
able to do same-node-no-container-distro-package in-place upgrade
solutions would be looking at such things if we decide we don't want
to support/enable this.

Some distros were in the room, and e.g. I believe it was Dan that said
RH don't support - in their packaging - mixed versions: upgrade nova,
and neutron would be upgraded too in the given example. So its
possible that while some folk aspire to it, many other folk are either
accepting downtime, or running their control plane on isolated
operating system instances (whether physical, VM or containers is
irrelevant).

At the party tonight I ran into Brian Demers and a colleague whose
name I have forgotten :( - but they have a backwards compat use case
we hadn't touched on in the session: running their Neutron plugin
against both stable and master of Neutron.

This is a classic colocation situation. There are a few possible scenarios:
 - run stable branches of the plugin and backport everything
- tricky if new oslo features are needed, but the existing pattern.
- also tricky for users, since there would be lots of releases of
both stable and master
 - run master against stable neutron using stable neutron oslo
 - means they can't use any new oslo features, and they would
depend on oslo not breaking any API's they use in master - so depends
on oslo API compat
 - or they have to provide backports within their tree and monkey
patch them into place, of new/changed things from oslo
 - run master against stable neutron using master oslo
- means neutron would have to work with master oslo - so the same
compat story, just from the other side


(Clearly there are also Neutron API driver things to discuss, but this
is in the context of the libraries thread :)).

I wonder how many other folk are trying to attempt such a thing -
anyone care to raise their metaphorical hand?

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] backwards compatibility followup

[Ian Cordasco]

> On Apr 26, 2016, at 6:32 PM, Perry, Sean  wrote:
> 
> What if we update the docs and tell people to put any services on a shared 
> system into independent virtualenvs?
> 
> If a box only runs neutron or whatever all is well. The issue happens when 
> you mix and match services on a single host. Which is exactly what virtualenv 
> was intended to fix.

Except that a fair portion of respondents to the OpenStack User Survey talk 
about using packages provided by the distribution of Linux they happen to be 
on. Those cannot be installed into virtual environments.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] backwards compatibility followup

[Perry, Sean]

From: Robert Collins [robe...@robertcollins.net]
Sent: Tuesday, April 26, 2016 4:07 PM
To: OpenStack Development Mailing List
Subject: [openstack-dev] backwards compatibility followup

Here is the choice I think we're still struggling with:
 - should we support in-place upgrades? If we do, we need at least 1,
possibly more, versions of compatibility such that e.g. mitaka Nova
can run with newton olso+clientlibs
 - or should we explicitly state that we don't support in-place
upgrades - that deployment methods must be architected to avoid ever
encountering the situation where a client or one of N services is
going to be upgraded on a single python environment: all clients and
services must be upgraded together, or none.


What if we update the docs and tell people to put any services on a shared 
system into independent virtualenvs?

If a box only runs neutron or whatever all is well. The issue happens when you 
mix and match services on a single host. Which is exactly what virtualenv was 
intended to fix.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] backwards compatibility followup

[Robert Collins]

Thanks everyone for coming to the backwards compat for clients and
libraries session.

Here are the things I think we agreed on:
 - clients need to talk to all versions of openstack clouds
 - oslo libraries already do need to do backwards compat - josh,
myself, dims, doug are all striving for that in reviews
 - some fraction of our deploys - somewhere between 1% and 50% - are
trying to do in-place upgrades where e.g. nova is upgraded and then
later neutron, which runs into neutron having to work with the new
libraries the nova upgrade dragged in.

Here is the choice I think we're still struggling with:
 - should we support in-place upgrades? If we do, we need at least 1,
possibly more, versions of compatibility such that e.g. mitaka Nova
can run with newton olso+clientlibs
 - or should we explicitly state that we don't support in-place
upgrades - that deployment methods must be architected to avoid ever
encountering the situation where a client or one of N services is
going to be upgraded on a single python environment: all clients and
services must be upgraded together, or none.

If we decide to support in-place upgrades, we can figure out how to
test that effectively; its a linear growth with the number of stable
releases we choose to support.

If we decide not to support them, we have no further requirement to
have any cross-over compat between OpenStack releases - while we will
still have to be backwards compatible on individual changes (so that
we can get them rolled out and used), we can do our garbage collection
much more rapidly - even same-cycle.


https://etherpad.openstack.org/p/newton-backwards-compat-libs

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev