Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Russell Bryant]

On 01/14/2016 08:04 PM, Jeremy Stanley wrote:
> On 2016-01-14 22:14:09 + (+), Sean M. Collins wrote:
> [...]
>> The problem we have is - most operators are using Ubuntu according to
>> the user survey. Most likely they are using LTS releases. We already get
>> flack for our pace of releases and our release lifecycle duration, so
>> if we were to move off LTS for our gate, we would be pushing operators
>> to move to more frequent upgrades for their base operating system. Maybe
>> that's a discussion that needs to be had, but it will be contentious.
> 
> As a point of reference, the OpenStack Infrastructure team only uses
> LTS distro releases to run production systems. We've also got a
> modest sized OpenStack deployment on its way to production, again on
> an LTS distro release. I agree that releasing server software which
> is only well tested on "desktop" pace distro releases would be a
> serious misstep for the project.
> 

but are most people using that LTS release with cloud-archive enabled?

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Jeremy Stanley]

On 2016-01-15 09:51:57 -0500 (-0500), Russell Bryant wrote:
> but are most people using that LTS release with cloud-archive
> enabled?

I can't answer whether most Ubuntu users are adding it, though the
(several) times we've tried to enable UCA in our test platforms in
the past we've reported crippling bugs in newer versions of system
dependencies which have caused us to scurry to roll that back. While
it may be time to try our luck again and hope we fare better, we're
also only a few months from the next Ubuntu LTS (and as also
mentioned CentOS 7 is available for use right now).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Jakub Libosvar]

On 01/14/2016 12:38 AM, Matthew Treinish wrote:
> On Wed, Jan 13, 2016 at 10:47:21PM +, Sean M. Collins wrote:
>> On Wed, Jan 13, 2016 at 03:57:37PM CST, Mooney, Sean K wrote:
>>> One of the ideas that I have been thinking about over the last month or two 
>>> is do we
>>> Want to create a dedicated library file in devstack to support compilation 
>>> and installation
>>> Of ovs. 
>>
>> So, my suggestion is as follows: create a new devstack plugin that is
>> *specifically* geared towards just compiling OVS and installing it to
>> get the bits that you need.
> 
> +1, you do not want a monolithic plugin that does everything. Creating a
> separate small plugin to install ovs from source and the other pieces 
> necessary
> for doing that is the best path for enabling this.
If I understand correctly, this requires to either create a new git
repository that just provides a function to compile ovs or add it to
some already existing project like ovs.

Given that we will likely not even use it with devstack but only in
gate_hook.sh for functional/ job, I don't see the point in creating new
repository for one bash function or any "small plugin".

I know it was me who originally put the code in the Neutron devstack
plugin. So seems like the thing we need to solve is "Where will it
live". How about creating a separate script in tools?

> 
>> I'm just concerned about the feature creep
>> that is happening in the Neutron DevStack plugin ( which I didn't like in
>> the first place ) where now every little thing is getting proposed
>> against it.
>>
>> I'd prefer to see small, very specific DevStack plugins that have narrow
>> focus, and jobs that need them for specific things adding them to their
>> local.conf settings explicitly via enable_repo lines.
> 
> This is the intended way to use plugins. Dumping everything but the kitchen
> sink into a single plugin is just going to tightly couple too much and lead
> to an undebugable tangled ball of yarn. The idea with plugins is to have
> smaller plugins that can be used in conjunction to configure and enable
> various additional pieces.
> 
>>
>> The concern I have with compiling bleeding edge OVS and then running our
>> Neutron jobs is that yes, we get new features, but yes we also get
>> the newest bugs and the configuration matrix for Neutron now gets a new
>> dimension of 'packaged OVS versus git commit SHA'
>>
> 
> I don't think you ever want to have a gating job with OVS from source.
> There is way too much potential instability when building something like this
> from source to rely on it. An experimental job would probably be as far as I
> would go with something like this. Going any further than that is just asking
> for trouble.
Experimental job sounds like the best candidate but it also brings
disadvantages like having the new code prone to regressions, as tests
for new features will need to be skipped in normal gate jobs.

In functional tests we don't use any complex scenarios on OVS. It mostly
tests interface drivers with simple steps like creating bridge and
plugging interface to it. I believe this is pretty basic thing that
works even in unstable development branch of OVS.

Thanks for ideas.
Kuba

> 
> -Matt Treinish
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Russell Bryant]

On 01/13/2016 11:51 PM, Tony Breeds wrote:
> The challenge for you guys is the kernel side of things but if I
> understood correctly you can get the kenrel module from the ovs
> source tree and just compile it against the stock ubuntu kernel
> (assuming the kernel devel headers are available) is that right?

It's kernel and userspace.  There's multiple current development
efforts that involve changes to OpenStack, OVS userspace, and the
appropriate datapath (OVS kernel module or DPDK).

The consensus I'm picking up roughly is that for those working on the
features, testing with source builds seems to be working fine.  It's
just not something anyone wants to gate the main Neutron repo with.
That seems quite reasonable.  If the features aren't in proper
releases yet, I don't see gating as that important anyway.

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Ihar Hrachyshka]

Sean M. Collins  wrote:


On Wed, Jan 13, 2016 at 03:57:37PM CST, Mooney, Sean K wrote:
One of the ideas that I have been thinking about over the last month or  
two is do we
Want to create a dedicated library file in devstack to support  
compilation and installation

Of ovs.


So, my suggestion is as follows: create a new devstack plugin that is
*specifically* geared towards just compiling OVS and installing it to
get the bits that you need. I'm just concerned about the feature creep
that is happening in the Neutron DevStack plugin ( which I didn't like in
the first place ) where now every little thing is getting proposed
against it.


Currently, devstack plugin has code for:
* qos
* sr-iov
* l2 agent extensions
* flavors

I think most of those could indeed live in a separate plugin (except  
probably l2 agent extensions that seems like a common feature for different  
agent types).


I wonder whether we can extend devstack plugin interface to support  
completely separate *per-feature* plugins, but in the *same* repo. In that  
case, we would have best of both worlds: code separation, the need for  
explicit enable_plugin call to enable a specific feature; and at the same  
time, no tiny git repos to maintain 10s of lines of bash code in each.




I'd prefer to see small, very specific DevStack plugins that have narrow
focus, and jobs that need them for specific things adding them to their
local.conf settings explicitly via enable_repo lines.

The concern I have with compiling bleeding edge OVS and then running our
Neutron jobs is that yes, we get new features, but yes we also get
the newest bugs and the configuration matrix for Neutron now gets a new
dimension of 'packaged OVS versus git commit SHA'

--
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Sean M. Collins]

On Thu, Jan 14, 2016 at 10:00:03AM CST, Ihar Hrachyshka wrote:
> Currently, devstack plugin has code for:
> * qos
> * sr-iov
> * l2 agent extensions
> * flavors

Right - and from the start on the review that introduced the Neutron devstack
plugin in the neutron tree, as well on IRC I advocated that the QoS work
should have been a small, narrowly defined DevStack plugin. Instead, we
now find ourselves where the kitchen sink has been thrown into this
thing.

Can we please take this all out of Neutron and put all these different pieces
into their own separate DevStack plugins?

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Jeremy Stanley]

On 2016-01-14 22:14:09 + (+), Sean M. Collins wrote:
[...]
> The problem we have is - most operators are using Ubuntu according to
> the user survey. Most likely they are using LTS releases. We already get
> flack for our pace of releases and our release lifecycle duration, so
> if we were to move off LTS for our gate, we would be pushing operators
> to move to more frequent upgrades for their base operating system. Maybe
> that's a discussion that needs to be had, but it will be contentious.

As a point of reference, the OpenStack Infrastructure team only uses
LTS distro releases to run production systems. We've also got a
modest sized OpenStack deployment on its way to production, again on
an LTS distro release. I agree that releasing server software which
is only well tested on "desktop" pace distro releases would be a
serious misstep for the project.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Russell Bryant]

On 01/14/2016 03:43 PM, Assaf Muller wrote:
> On Thu, Jan 14, 2016 at 9:28 AM, Russell Bryant  wrote:
>> On 01/13/2016 11:51 PM, Tony Breeds wrote:
>>> The challenge for you guys is the kernel side of things but if I
>>> understood correctly you can get the kenrel module from the ovs
>>> source tree and just compile it against the stock ubuntu kernel
>>> (assuming the kernel devel headers are available) is that right?
>>
>> It's kernel and userspace.  There's multiple current development
>> efforts that involve changes to OpenStack, OVS userspace, and the
>> appropriate datapath (OVS kernel module or DPDK).
>>
>> The consensus I'm picking up roughly is that for those working on the
>> features, testing with source builds seems to be working fine.  It's
>> just not something anyone wants to gate the main Neutron repo with.
>> That seems quite reasonable.  If the features aren't in proper
>> releases yet, I don't see gating as that important anyway.
> 
> I want to have voting tests for new features. For the past year the
> OVS agent ARP responder feature has been without proper coverage, and
> now it's the upcoming OVS firewall driver. I think that as long as we
> compile from a specific OVS patch (And not a moving target), I don't
> see much of a difference between gating on OVS 2.0 and gating on, for
> example, the current tip of the OVS 2.5 branch (But continuing to
> gate on that patch, so when the OVS 2.5 branch gets backports we
> won't gate on those, and we'll be able to move to a new tip in our
> own pace). As long as we pick a patch to compile against and run the
> functional tests a few times and verify that it works, I think it's
> reasonable. We've been gating against OVS 2.0 for the past few years,
> that to me seems unreasonable. We're gating against an OVS version
> nobody is using in production anymore.

I would agree that still using OVS 2.0 doesn't make any sense.
-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Clark Boylan]

On Thu, Jan 14, 2016, at 01:07 PM, Russell Bryant wrote:
> On 01/14/2016 03:43 PM, Assaf Muller wrote:
> > On Thu, Jan 14, 2016 at 9:28 AM, Russell Bryant  wrote:
> >> On 01/13/2016 11:51 PM, Tony Breeds wrote:
> >>> The challenge for you guys is the kernel side of things but if I
> >>> understood correctly you can get the kenrel module from the ovs
> >>> source tree and just compile it against the stock ubuntu kernel
> >>> (assuming the kernel devel headers are available) is that right?
> >>
> >> It's kernel and userspace.  There's multiple current development
> >> efforts that involve changes to OpenStack, OVS userspace, and the
> >> appropriate datapath (OVS kernel module or DPDK).
> >>
> >> The consensus I'm picking up roughly is that for those working on the
> >> features, testing with source builds seems to be working fine.  It's
> >> just not something anyone wants to gate the main Neutron repo with.
> >> That seems quite reasonable.  If the features aren't in proper
> >> releases yet, I don't see gating as that important anyway.
> > 
> > I want to have voting tests for new features. For the past year the
> > OVS agent ARP responder feature has been without proper coverage, and
> > now it's the upcoming OVS firewall driver. I think that as long as we
> > compile from a specific OVS patch (And not a moving target), I don't
> > see much of a difference between gating on OVS 2.0 and gating on, for
> > example, the current tip of the OVS 2.5 branch (But continuing to
> > gate on that patch, so when the OVS 2.5 branch gets backports we
> > won't gate on those, and we'll be able to move to a new tip in our
> > own pace). As long as we pick a patch to compile against and run the
> > functional tests a few times and verify that it works, I think it's
> > reasonable. We've been gating against OVS 2.0 for the past few years,
> > that to me seems unreasonable. We're gating against an OVS version
> > nobody is using in production anymore.
> 
> I would agree that still using OVS 2.0 doesn't make any sense.
>
Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
is what is shipped with the distros that we test on. Are we saying that
no one uses the distro provided OVS packages to run Neutron? If not what
are they using?

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Sean M. Collins]

On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
> Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
> is what is shipped with the distros that we test on. Are we saying that
> no one uses the distro provided OVS packages to run Neutron? If not what
> are they using?

Right - this was my impression as well.

I know that at least in Ubuntu, they have repos like Cloud-Archive[1]
that are targeted towards OpenStack releases and more recent packages of
libraries. Perhaps we need to investigate if our Ubuntu images at the
gate should be using this repo or something like this to pick up more
recent packaged versions of components like Open vSwitch?


[1]: https://launchpad.net/cloud-archive

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Ben Pfaff]

On Thu, Jan 14, 2016 at 09:30:03PM +, Sean M. Collins wrote:
> On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
> > Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
> > is what is shipped with the distros that we test on. Are we saying that
> > no one uses the distro provided OVS packages to run Neutron? If not what
> > are they using?
> 
> Right - this was my impression as well.

Even Debian stable (jessie) has OVS 2.3, what distro has 2.0?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Assaf Muller]

On Thu, Jan 14, 2016 at 9:28 AM, Russell Bryant  wrote:
> On 01/13/2016 11:51 PM, Tony Breeds wrote:
>> The challenge for you guys is the kernel side of things but if I
>> understood correctly you can get the kenrel module from the ovs
>> source tree and just compile it against the stock ubuntu kernel
>> (assuming the kernel devel headers are available) is that right?
>
> It's kernel and userspace.  There's multiple current development
> efforts that involve changes to OpenStack, OVS userspace, and the
> appropriate datapath (OVS kernel module or DPDK).
>
> The consensus I'm picking up roughly is that for those working on the
> features, testing with source builds seems to be working fine.  It's
> just not something anyone wants to gate the main Neutron repo with.
> That seems quite reasonable.  If the features aren't in proper
> releases yet, I don't see gating as that important anyway.

I want to have voting tests for new features. For the past year the OVS agent
ARP responder feature has been without proper coverage, and now it's the
upcoming OVS firewall driver. I think that as long as we compile from a specific
OVS patch (And not a moving target), I don't see much of a difference between
gating on OVS 2.0 and gating on, for example, the current tip of the
OVS 2.5 branch
(But continuing to gate on that patch, so when the OVS 2.5 branch gets backports
we won't gate on those, and we'll be able to move to a new tip in our own pace).
As long as we pick a patch to compile against and run the functional
tests a few times
and verify that it works, I think it's reasonable. We've been gating
against OVS 2.0
for the past few years, that to me seems unreasonable. We're gating
against an OVS
version nobody is using in production anymore.

>
> --
> Russell Bryant
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Brian Haley]

On 1/14/16 9:28 AM, Russell Bryant wrote:


The consensus I'm picking up roughly is that for those working on the
features, testing with source builds seems to be working fine.  It's
just not something anyone wants to gate the main Neutron repo with.
That seems quite reasonable.  If the features aren't in proper
releases yet, I don't see gating as that important anyway.


Yes, I don't see it as being used for gating (yet), just for easily 
selecting a version of OVS I want to use for testing, like something 
that supports IPv6 tunnels, whether it's pre-built or built by the script.


-Brian

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Russell Bryant]

On 01/14/2016 04:32 PM, Ben Pfaff wrote:
> On Thu, Jan 14, 2016 at 09:30:03PM +, Sean M. Collins wrote:
>> On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
>>> Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
>>> is what is shipped with the distros that we test on. Are we saying that
>>> no one uses the distro provided OVS packages to run Neutron? If not what
>>> are they using?
>>
>> Right - this was my impression as well.
> 
> Even Debian stable (jessie) has OVS 2.3, what distro has 2.0?

Ubuntu 14.04 (latest LTS)

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Sean M. Collins]

On Thu, Jan 14, 2016 at 03:32:21PM CST, Ben Pfaff wrote:
> On Thu, Jan 14, 2016 at 09:30:03PM +, Sean M. Collins wrote:
> > On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
> > > Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
> > > is what is shipped with the distros that we test on. Are we saying that
> > > no one uses the distro provided OVS packages to run Neutron? If not what
> > > are they using?
> > 
> > Right - this was my impression as well.
> 
> Even Debian stable (jessie) has OVS 2.3, what distro has 2.0?

The "devstack-trusty" image the gate uses.

http://logs.openstack.org/42/201942/26/check/gate-tempest-dsvm-neutron-full/e4da38e/logs/devstacklog.txt.gz#_2016-01-14_16_02_55_354


-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Clark Boylan]

On Thu, Jan 14, 2016, at 01:32 PM, Ben Pfaff wrote:
> On Thu, Jan 14, 2016 at 09:30:03PM +, Sean M. Collins wrote:
> > On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
> > > Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
> > > is what is shipped with the distros that we test on. Are we saying that
> > > no one uses the distro provided OVS packages to run Neutron? If not what
> > > are they using?
> > 
> > Right - this was my impression as well.
> 
> Even Debian stable (jessie) has OVS 2.3, what distro has 2.0?
> 
For the three distro releases we currently test on
Ubuntu Trusty: 2.0.2
CentOS7: 2.4.0
Fedora 23 (not actually used just yet, but is being worked on): 2.4.0

If we enable cloud archive it looks like we would have access to 2.4.0
on Trusty. Historically we haven't used the cloud archive because it was
broken. At least libvirt and mongodb did not function well enough to
gate against. That may no longer be the case and is something we could
consider adding to jobs, but we would have to be careful to avoid the
same problems we have had in the past.

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Ben Pfaff]

On Thu, Jan 14, 2016 at 01:45:39PM -0800, Clark Boylan wrote:
> On Thu, Jan 14, 2016, at 01:32 PM, Ben Pfaff wrote:
> > On Thu, Jan 14, 2016 at 09:30:03PM +, Sean M. Collins wrote:
> > > On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
> > > > Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
> > > > is what is shipped with the distros that we test on. Are we saying that
> > > > no one uses the distro provided OVS packages to run Neutron? If not what
> > > > are they using?
> > > 
> > > Right - this was my impression as well.
> > 
> > Even Debian stable (jessie) has OVS 2.3, what distro has 2.0?
> > 
> For the three distro releases we currently test on
> Ubuntu Trusty: 2.0.2
> CentOS7: 2.4.0
> Fedora 23 (not actually used just yet, but is being worked on): 2.4.0

OK, I misunderstood the problem.  It's not "distros have old OVS" but
"we're using an old distro with old OVS".  Fair enough.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Sean M. Collins]

On Thu, Jan 14, 2016 at 03:45:39PM CST, Clark Boylan wrote:
> On Thu, Jan 14, 2016, at 01:32 PM, Ben Pfaff wrote:
> > On Thu, Jan 14, 2016 at 09:30:03PM +, Sean M. Collins wrote:
> > > On Thu, Jan 14, 2016 at 03:13:16PM CST, Clark Boylan wrote:
> > > > Forgive my ignorance, but if we are gating on OVS 2.0 that is because it
> > > > is what is shipped with the distros that we test on. Are we saying that
> > > > no one uses the distro provided OVS packages to run Neutron? If not what
> > > > are they using?
> > > 
> > > Right - this was my impression as well.
> > 
> > Even Debian stable (jessie) has OVS 2.3, what distro has 2.0?
> > 
> For the three distro releases we currently test on
> Ubuntu Trusty: 2.0.2
> CentOS7: 2.4.0
> Fedora 23 (not actually used just yet, but is being worked on): 2.4.0

I have DevStack running on Ubuntu Wily (15.10) which has OVS 2.4.0 - it at least
runs a stack.sh to completion with no issues. Haven't had a chance to do
more comprehensive validation. 

In a couple months when 16.04 comes out we'll have an LTS release that
will have a more recent OVS version packaged up.

The problem we have is - most operators are using Ubuntu according to
the user survey. Most likely they are using LTS releases. We already get
flack for our pace of releases and our release lifecycle duration, so
if we were to move off LTS for our gate, we would be pushing operators
to move to more frequent upgrades for their base operating system. Maybe
that's a discussion that needs to be had, but it will be contentious.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Russell Bryant]

On 01/13/2016 03:59 PM, Assaf Muller wrote:
> On Wed, Jan 13, 2016 at 4:50 AM, Jakub Libosvar  wrote:
>> Hi all,
>>
>> recently I was working on firewall driver [1] that requires latest
>> features in OVS, specifically conntrack support. In order to get the
>> driver tested, we need to have the latest OVS kernel modules on machines
>> running tests but AFAIK there is no stable "2.5 like" release of OVS yet.
>>
>> Facing the same problem OVN did in the past, I decided to try to steal
>> their solution and apply it in our Neutron repo [2]. Sean and Matthew
>> rightfully expressed worries about this approach on review of [2].
>>
>> So I'd like to bring this to a broader audience and ask for help or
>> suggestion on how to test such Neutron features that require some newer
>> versions.
>>
>> The first idea was to have an option to compile trunk ovs and use it in
>> particular jobs like functional one. That would bring faster development
>> of features going along with ovs features.
> 
> I think we should use a newer OVS version that for the functional and
> fullstack jobs at the very least. This will enable us to test the OVS
> firewall driver you're working on, as well as the OVS ARP responder
> (Which was merged a long time ago, but lacks proper testing because
> it needs OVS 2.1+ and we gate using OVS 2.0).
> 
> So, that's the problem. How we solve it is another manner - I was 
> under the impression that compiling it is our only option. Right now
> OVN are compiling master, I think we should avoid that and compile a
> specific git hash instead so the gate won't break every time OVS
> breaks. Then, when OVS branch out 2.5, we can 'pin' on that.

FWIW, a 2.5 branch already exists, but hasn't been released yet.  It
should only be receiving bug fixes at this point.

https://github.com/openvswitch/ovs/tree/branch-2.5

We compile from master since OVN is under such active development in
parallel, so that's really what we want.  It also doesn't break anyone
but networking-ovn if there's an ovs issue.  I probably wouldn't do that
for a more general voting neutron job.

> At that point we could perhaps switch to a packaged OVS off some
> sort of experimental repo.

Note that just using OVS 2.5 isn't enough.  You must also have kernel
support for ovs+conntrack integration.  That is in Linux 4.3+ or the
openvswitch kernel module included in the ovs tree.

networking-ovn's tempest job uses the openvswitch kernel module from ovs
master, as well.  That seems to be working on the devstack jenkins nodes
just fine.

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Assaf Muller]

On Wed, Jan 13, 2016 at 4:50 AM, Jakub Libosvar  wrote:
> Hi all,
>
> recently I was working on firewall driver [1] that requires latest
> features in OVS, specifically conntrack support. In order to get the
> driver tested, we need to have the latest OVS kernel modules on machines
> running tests but AFAIK there is no stable "2.5 like" release of OVS yet.
>
> Facing the same problem OVN did in the past, I decided to try to steal
> their solution and apply it in our Neutron repo [2]. Sean and Matthew
> rightfully expressed worries about this approach on review of [2].
>
> So I'd like to bring this to a broader audience and ask for help or
> suggestion on how to test such Neutron features that require some newer
> versions.
>
> The first idea was to have an option to compile trunk ovs and use it in
> particular jobs like functional one. That would bring faster development
> of features going along with ovs features.

I think we should use a newer OVS version that for the functional and fullstack
jobs at the very least. This will enable us to test the OVS firewall
driver you're
working on, as well as the OVS ARP responder (Which was merged a long time
ago, but lacks proper testing because it needs OVS 2.1+ and we gate
using OVS 2.0).

So, that's the problem. How we solve it is another manner - I was
under the impression
that compiling it is our only option. Right now OVN are compiling
master, I think we
should avoid that and compile a specific git hash instead so the gate
won't break every
time OVS breaks. Then, when OVS branch out 2.5, we can 'pin' on that.
At that point
we could perhaps switch to a packaged OVS off some sort of experimental repo.

>
> Please share your ideas :)
>
> Thanks!
> Kuba
>
> [1] https://review.openstack.org/#/c/249337
> [2] https://review.openstack.org/#/c/266423
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Sean M. Collins]

On Wed, Jan 13, 2016 at 03:57:37PM CST, Mooney, Sean K wrote:
> One of the ideas that I have been thinking about over the last month or two 
> is do we
> Want to create a dedicated library file in devstack to support compilation 
> and installation
> Of ovs. 

So, my suggestion is as follows: create a new devstack plugin that is
*specifically* geared towards just compiling OVS and installing it to
get the bits that you need. I'm just concerned about the feature creep
that is happening in the Neutron DevStack plugin ( which I didn't like in
the first place ) where now every little thing is getting proposed
against it.

I'd prefer to see small, very specific DevStack plugins that have narrow
focus, and jobs that need them for specific things adding them to their
local.conf settings explicitly via enable_repo lines.

The concern I have with compiling bleeding edge OVS and then running our
Neutron jobs is that yes, we get new features, but yes we also get
the newest bugs and the configuration matrix for Neutron now gets a new
dimension of 'packaged OVS versus git commit SHA'

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Sean M. Collins]

On Wed, Jan 13, 2016 at 04:47:19PM CST, Sean M. Collins wrote:
> I'd prefer to see small, very specific DevStack plugins that have narrow
> focus, and jobs that need them for specific things adding them to their
> local.conf settings explicitly via enable_repo lines.

Sorry, enable_plugin lines


-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Matthew Treinish]

On Wed, Jan 13, 2016 at 10:47:21PM +, Sean M. Collins wrote:
> On Wed, Jan 13, 2016 at 03:57:37PM CST, Mooney, Sean K wrote:
> > One of the ideas that I have been thinking about over the last month or two 
> > is do we
> > Want to create a dedicated library file in devstack to support compilation 
> > and installation
> > Of ovs. 
> 
> So, my suggestion is as follows: create a new devstack plugin that is
> *specifically* geared towards just compiling OVS and installing it to
> get the bits that you need.

+1, you do not want a monolithic plugin that does everything. Creating a
separate small plugin to install ovs from source and the other pieces necessary
for doing that is the best path for enabling this.

> I'm just concerned about the feature creep
> that is happening in the Neutron DevStack plugin ( which I didn't like in
> the first place ) where now every little thing is getting proposed
> against it.
> 
> I'd prefer to see small, very specific DevStack plugins that have narrow
> focus, and jobs that need them for specific things adding them to their
> local.conf settings explicitly via enable_repo lines.

This is the intended way to use plugins. Dumping everything but the kitchen
sink into a single plugin is just going to tightly couple too much and lead
to an undebugable tangled ball of yarn. The idea with plugins is to have
smaller plugins that can be used in conjunction to configure and enable
various additional pieces.

> 
> The concern I have with compiling bleeding edge OVS and then running our
> Neutron jobs is that yes, we get new features, but yes we also get
> the newest bugs and the configuration matrix for Neutron now gets a new
> dimension of 'packaged OVS versus git commit SHA'
> 

I don't think you ever want to have a gating job with OVS from source.
There is way too much potential instability when building something like this
from source to rely on it. An experimental job would probably be as far as I
would go with something like this. Going any further than that is just asking
for trouble.

-Matt Treinish



signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Testing Neutron with latest OVS

[Tony Breeds]

On Wed, Jan 13, 2016 at 10:47:21PM +, Sean M. Collins wrote:
> On Wed, Jan 13, 2016 at 03:57:37PM CST, Mooney, Sean K wrote:
> > One of the ideas that I have been thinking about over the last month or two 
> > is do we
> > Want to create a dedicated library file in devstack to support compilation 
> > and installation
> > Of ovs. 
> 
> So, my suggestion is as follows: create a new devstack plugin that is
> *specifically* geared towards just compiling OVS and installing it to
> get the bits that you need. I'm just concerned about the feature creep
> that is happening in the Neutron DevStack plugin ( which I didn't like in
> the first place ) where now every little thing is getting proposed
> against it.
> 
> I'd prefer to see small, very specific DevStack plugins that have narrow
> focus, and jobs that need them for specific things adding them to their
> local.conf settings explicitly via enable_repo lines.
> 
> The concern I have with compiling bleeding edge OVS and then running our
> Neutron jobs is that yes, we get new features, but yes we also get
> the newest bugs and the configuration matrix for Neutron now gets a new
> dimension of 'packaged OVS versus git commit SHA'

For nova we're working on a devstack plugin that will add a repo config to the
host and then upgrade a set of packages from that repo.  The idea being to
decouple the compile step from the job.

basically:
echo deb http://host/path main > /etc/apt/sources.list.d/bleeding_edge.list
apt-get udpate
apt-get install $package_list

but you know with error handling etc.  I'm sure it can be abstracted to handle
multiple repos and package lists if you're interested?

The devstack plugin could be taught to installed from a per-compiled tgz rather
than a deb/rpm if you wanted but really packages aren't that hard.

The challenge for you guys is the kernel side of things but if I understood
correctly you can get the kenrel module from the ovs source tree and just
compile it against the stock ubuntu kernel (assuming the kernel devel headers
are available) is that right?

Yours Tony.


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev