Re: [openstack-dev] [Neutron] Neutron team social event in Barcelona

[Paul Michali]

+1 Thanks for setting this up!

On Sat, Oct 15, 2016, 9:53 AM Alonso Hernandez, Rodolfo <
rodolfo.alonso.hernan...@intel.com> wrote:

> +1
>
>
>
> *From:* Doug Wiegley [mailto:doug...@parksidesoftware.com]
> *Sent:* Saturday, October 15, 2016 1:57 AM
> *To:* OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>
> *Subject:* Re: [openstack-dev] [Neutron] Neutron team social event in
> Barcelona
>
>
>
> +1
>
> Doug
>
>
> On Oct 14, 2016, at 6:24 PM, Kevin Benton  wrote:
>
> +1
>
>
>
> On Oct 14, 2016 1:33 PM, "Miguel Lavalle"  wrote:
>
> Dear Neutrinos,
>
> I am organizing a social event for the team on Thursday 27th at 19:30.
> After doing some Google research, I am proposing Raco de la Vila, which is
> located in Poblenou: http://www.racodelavila.com/en/index.htm. The menu
> is here: http://www.racodelavila.com/en/carta-racodelavila.htm
>
> It is easy to get there by subway from the Summit venue:
> https://goo.gl/maps/HjaTEcBbDUR2. I made a reservation for 25 people
> under 'Neutron' or "Miguel Lavalle". Please confirm your attendance so we
> can get a final count.
>
> Here's some reviews:
> https://www.tripadvisor.com/Restaurant_Review-g187497-d1682057-Reviews-Raco_De_La_Vila-Barcelona_Catalonia.html
>
>
> Cheers
>
>
>
> Miguel
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Adding ihrachys to theneutron-drivers team

[Paul Michali]

Congratulations Ihar! Great addition to the Drivers team.

On Mon, Sep 19, 2016 at 4:52 AM Vikram Choudhary <
vikram.choudh...@huawei.com> wrote:

> Congrats Ihar!
>
>
>
> *From:* reedip banerjee [mailto:reedi...@gmail.com]
> *Sent:* 19 September 2016 14:08
>
>
> *To:* OpenStack Development Mailing List (not for usage questions)
>
> *Subject:* Re: [openstack-dev] [Neutron] Adding ihrachys to
> theneutron-drivers team
>
>
>
> Congratulations Ihar :)
>
>
>
> On Mon, Sep 19, 2016 at 2:00 PM, Martin Hickey 
> wrote:
>
> I totally agree. Congrats Ihar.
>
> Regards,
> Martin
> -
> Martin Hickey | Software Development | Open Technologies | Tel: + 353 21
> 7306040
>
> [image: Inactive hide details for Assaf Muller ---17/09/2016
> 23:12:03---Well deserved is an understatement! Ihar is consistently 
> integr]Assaf
> Muller ---17/09/2016 23:12:03---Well deserved is an understatement! Ihar is
> consistently integral to the Neutron project. Ihar has w
>
> From: Assaf Muller 
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: 17/09/2016 23:12
> Subject: Re: [openstack-dev] [Neutron] Adding ihrachys to the
> neutron-drivers team
> --
>
>
>
>
> Well deserved is an understatement! Ihar is consistently integral to
> the Neutron project. Ihar has wide knowledge not only of Neutron but
> of OpenStack workings and is a key factor to Neutron playing nice with
> other projects. Ihar has shown consistent good judgement of priorities
> and will in no doubt strengthen the drivers team.
>
> On Sat, Sep 17, 2016 at 1:19 PM, Dariusz Śmigiel
>  wrote:
> > Congrats Ihar. You deserve this!
> >
> > 2016-09-17 18:40 GMT+02:00 Armando M. :
> >> Hi folks,
> >>
> >> I would like to propose Ihar to become a member of the Neutron drivers
> team
> >> [1].
> >>
> >> Ihar wide knowledge of the Neutron codebase, and his longstanding
> duties as
> >> stable core, downstream package whisperer, release and oslo liaison (I
> am
> >> sure I am forgetting some other capacity he is in) is going to put him
> at
> >> great comfort in the newly appointed role, and help him grow and become
> wise
> >> even further.
> >>
> >> Even though we have not been meeting regularly lately we will resume our
> >> Thursday meetings soon [2], and having Ihar onboard by then will be
> highly
> >> beneficial.
> >>
> >> Please, join me in welcome Ihar to the team.
> >>
> >> Cheers,
> >> Armando
> >>
> >> [1]
> >>
> http://docs.openstack.org/developer/neutron/policies/neutron-teams.html#drivers-team
> >> [2] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers
> >>
> >>
> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> >
> >
> > --
> > Darek "dasm" Śmigiel
> >
> > 
> > Q: Why is this email five sentences or less?
> > A: http://five.sentenc.es
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
>
> --
>
> Thanks and Regards,
> Reedip Banerjee
>
> IRC: reedip
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Is this a bug in metadata proxy...

[Paul Michali]

Yeah, the setting for user was neutron. The setting for group was different
than the process that started up the proxy (which had user neutron and
group neutron). The review 161494, was it applied to Liberty?

I think in our case, DHCP agent only had dhcp ini file and not metadata ini
file, so it didn't have the user/group setting.

I was wondering if the user/group should be (only) set in a common config,
like neutron.conf, if it should be duplicated in dhcp and metadata config
files, or if the metadata ini should be added to the list of ini files,
when starting up the DHCP agent.

With the wrong config, I hit the access denied issue and had no info
indicating that is what has happened. Was wondering if there was any
protection against that misconfiguration case, or way to get an indication
of it.

P.S. Sorry, I didn't see your reply till now...

PCM


On Wed, Aug 31, 2016 at 10:36 AM ZZelle <zze...@gmail.com> wrote:

> Hi,
>
> Are you sure metadata_proxy_user==neutron?
>
> neutron-metadata-proxy must be able to connect to the metadata-agent
> socket and watchs its log files and neutron user should be able to do both
> with usual file permissions.
>
> Otherwise the metadata proxy is generally no more able to:
> - watch log[1] so you should set metadata_proxy_watch_log=False
> - connect to the metadata-agent because of socket permissions, so you
> should set metadata_proxy_socket_mode option[2] in order to let the
> metadata agent set the correct perms on metadata socket.
>
> If you provide metadata_proxy_user/group in l3/dhcp-agent and
> metadata-agent config then neutron should be able to deduce both
> metadata_proxy_watch_log and metadata_proxy_socket_mode values.
>
>
>
> [1] https://review.openstack.org/#/c/161494/
> [2] https://review.openstack.org/#/c/165115/
>
> Cédric/ZZelle
>
> On Wed, Aug 31, 2016 at 2:16 PM, Paul Michali <p...@michali.net> wrote:
>
>> Hi,
>>
>> I had seen something and was not sure if this was a subtle bug or not.
>>
>> I have a Liberty based openstack setup. The account that is setting up
>> processes was user=neutron, group=neutron, however the metadata_agent.ini
>> config file was set up for a different group. So there was a
>> metadata_proxy_user=neutron, and metadata_proxy_group=foo config setting.
>>
>> This ini file was used by the metadata agent process, but it was not
>> included in the DHCP agent process (not sure if I should have included the
>> metadata_agent.ini in the startup of DHCP or should have added these two
>> metadata proxy settings to neutron.conf, so that they were available to
>> DHCP).
>>
>> In any case, here is what I saw happen...
>>
>> I created a subnet (not using a router in this setup). It looks like DHCP
>> starts up the metadata agent proxy daemon) and the DHCP configuration is
>> used, which does NOT include the metadata_proxy_user/group, so the current
>> user's uid and gid are used (neutron/neutron) for the
>> metadata_proxy_user/group settings.
>>
>> The proxy calls drop_privileges(), which because the group is different,
>> the log file can no longer be accessed by the daemon. An OSError occurs
>> with permission denied on the log file for this process, and the process
>> exits without any indications.
>>
>> When I then try to use metadata services it fails (obviously). Looking,
>> we see that the metadata service is running (but the proxy is not, and I
>> don't see a way for an end user to check that - is there a way?).
>>
>> Looking in the proxy log, the initial startup messages are seen, showing
>> all the configuration settings, and then there is nothing more. No
>> indication that it is lowering privileges to run under some other
>> user/group, that there was a fatal error, or that it is working and ready
>> to process requests. Nothing more appears in the log, as it was working and
>> there were no metadata proxy requests occurring.
>>
>> I was only able to figure it out, by first checking to see if the proxy
>> was running, and then manually trying to start the proxy, using the command
>> line in the log, under a debugger, to find out that there was a permission
>> denied error.
>>
>> So, it is likely a misconfiguration error on the user's part, but it was
>> really hard to figure that out.
>>
>> Should/could we somehow indicate if there is an error lowering privs?
>>
>> Is there a (user) way to tell if proxy is running?
>>
>> Is there some documentation indicating that the proxy user/group settings
>> need to be available for both the metadata agent and for other agents that
>> may spawn the proxy (DHCP, L3)?
>>
>

[openstack-dev] [neutron] Is this a bug in metadata proxy...

[Paul Michali]

Hi,

I had seen something and was not sure if this was a subtle bug or not.

I have a Liberty based openstack setup. The account that is setting up
processes was user=neutron, group=neutron, however the metadata_agent.ini
config file was set up for a different group. So there was a
metadata_proxy_user=neutron, and metadata_proxy_group=foo config setting.

This ini file was used by the metadata agent process, but it was not
included in the DHCP agent process (not sure if I should have included the
metadata_agent.ini in the startup of DHCP or should have added these two
metadata proxy settings to neutron.conf, so that they were available to
DHCP).

In any case, here is what I saw happen...

I created a subnet (not using a router in this setup). It looks like DHCP
starts up the metadata agent proxy daemon) and the DHCP configuration is
used, which does NOT include the metadata_proxy_user/group, so the current
user's uid and gid are used (neutron/neutron) for the
metadata_proxy_user/group settings.

The proxy calls drop_privileges(), which because the group is different,
the log file can no longer be accessed by the daemon. An OSError occurs
with permission denied on the log file for this process, and the process
exits without any indications.

When I then try to use metadata services it fails (obviously). Looking, we
see that the metadata service is running (but the proxy is not, and I don't
see a way for an end user to check that - is there a way?).

Looking in the proxy log, the initial startup messages are seen, showing
all the configuration settings, and then there is nothing more. No
indication that it is lowering privileges to run under some other
user/group, that there was a fatal error, or that it is working and ready
to process requests. Nothing more appears in the log, as it was working and
there were no metadata proxy requests occurring.

I was only able to figure it out, by first checking to see if the proxy was
running, and then manually trying to start the proxy, using the command
line in the log, under a debugger, to find out that there was a permission
denied error.

So, it is likely a misconfiguration error on the user's part, but it was
really hard to figure that out.

Should/could we somehow indicate if there is an error lowering privs?

Is there a (user) way to tell if proxy is running?

Is there some documentation indicating that the proxy user/group settings
need to be available for both the metadata agent and for other agents that
may spawn the proxy (DHCP, L3)?

Regards,

PCM
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

I did have a question about the current implementation as described by
292499, 324379, and 292500.

Looking at the code, when a NUMAPagesTopology object is create, a new
parameter is passed for the "reserved" pages. This reservation comes from a
dictionary, which is populated at LIbvirtDriver init time, via grabbing the
multi-string configuration settings from nova.conf. Because the object's
API is changed, a version change is required.

Is it possible to, instead of adding a new argument to reduce the "total"
argument (Ian Wells suggested this to me on a patch I had), by the number
of reserved pages from the config file? This would prevent the need to
alter the object's API.  So, instead of:

mempages = [
objects.NUMAPagesTopology(
size_kb=pages.size,
total=pages.total,
used=0,
reserved=_get_reserved_memory_for_cell(
self, cell.id, pages.size))
for pages in cell.mempages]


Do something like this...

 mempages = [

objects.NUMAPagesTopology( size_kb=pages.size, used=0, total=pages.total -
_get_reserved_memory_for_cell( self, cell.id, pages.size)) for pages in
cell.mempages]
If we do this, would it avoid issues with back porting the change?

Thanks!

PCM


On Wed, Jun 15, 2016 at 5:52 PM Matt Riedemann <mrie...@linux.vnet.ibm.com>
wrote:

> On 6/15/2016 3:10 PM, Paul Michali wrote:
> > Is the plan to back port that change to Mitaka?
> >
> > Thanks,
> >
> > PCM
> >
> >
> > On Wed, Jun 15, 2016 at 1:31 PM Matt Riedemann
> > <mrie...@linux.vnet.ibm.com <mailto:mrie...@linux.vnet.ibm.com>> wrote:
> >
> > On 6/14/2016 3:09 PM, Jay Pipes wrote:
> > >
> > > Yes. Code merged recently from Sahid does this:
> > >
> > > https://review.openstack.org/#/c/277422/
> > >
> > > Best,
> > > -jay
> > >
> >
> > That was actually reverted out of mitaka:
> >
> > https://review.openstack.org/#/c/292290/
> >
> > The feature change that got into newton was this:
> >
> > https://review.openstack.org/#/c/292499/
> >
> > Which was busted, and required:
> >
> > https://review.openstack.org/#/c/324379/
> >
> > Well, required as long as you want your compute service to start. :)
> >
> > And no, we aren't backporting these, especially to liberty which is
> > security / critical fix mode only now.
> >
> > --
> >
> > Thanks,
> >
> > Matt Riedemann
> >
> >
> >
>  __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > <
> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> No, it's really a feature.
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Is the plan to back port that change to Mitaka?

Thanks,

PCM


On Wed, Jun 15, 2016 at 1:31 PM Matt Riedemann 
wrote:

> On 6/14/2016 3:09 PM, Jay Pipes wrote:
> >
> > Yes. Code merged recently from Sahid does this:
> >
> > https://review.openstack.org/#/c/277422/
> >
> > Best,
> > -jay
> >
>
> That was actually reverted out of mitaka:
>
> https://review.openstack.org/#/c/292290/
>
> The feature change that got into newton was this:
>
> https://review.openstack.org/#/c/292499/
>
> Which was busted, and required:
>
> https://review.openstack.org/#/c/324379/
>
> Well, required as long as you want your compute service to start. :)
>
> And no, we aren't backporting these, especially to liberty which is
> security / critical fix mode only now.
>
> --
>
> Thanks,
>
> Matt Riedemann
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Yeah, was thinking more of technically vs policy. Wondering if there are
other dependencies or if I could patch this into a Liberty code base.


On Wed, Jun 15, 2016 at 12:38 PM Jay Pipes <jaypi...@gmail.com> wrote:

> On 06/15/2016 03:58 AM, Paul Michali wrote:
> > Awesome Jay!
> >
> > Do you think this is something that can be backporting to Liberty w/o
> > other dependencies? We're running Liberty on our system right now.
>
> Doubtful, Paul :( The policy for upstream is not to backport feature
> patches. This would be something you would need to do yourself -- i.e.
> keep patch technical debt for whichever distribution of OpenStack you
> are using (or sources if you're going that route).
>
> Best,
> -jay
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Awesome Jay!

Do you think this is something that can be backporting to Liberty w/o other
dependencies? We're running Liberty on our system right now.


On Tue, Jun 14, 2016 at 4:10 PM Jay Pipes <jaypi...@gmail.com> wrote:

> On 06/14/2016 12:30 PM, Paul Michali wrote:
> > Well, looks like we figured out what is going on - maybe folks have some
> > ideas on how we could handle this issue.
> >
> > What I see is that for each VM create (small flavor), 1024 huge pages
> > are used and NUMA node 0 used. It appears that, when there is no longer
> > enough huge pages on that NUMA node, Nova with then schedule to the
> > other NUMA node and use those huge pages.
> >
> > In our case, we happen to have a special container running on the
> > compute nodes, that uses 512 huge pages. As a result, when there are 768
> > huge pages left, Nova thinks there are 1280 pages left and thinks one
> > more VM can be create. It tries, but the create fails.
> >
> > Some questions...
> >
> > 1) Is there some way to "reserve" huge pages in Nova?
>
> Yes. Code merged recently from Sahid does this:
>
> https://review.openstack.org/#/c/277422/
>
> Best,
> -jay
>
> > 2) If the create fails, should Nova try the other NUMA node (or is this
> > because it doesn't know why it failed)?
> > 3) Any ideas on how we can deal with this - without changing Nova?
> >
> > Thanks!
> >
> > PCM
> >
> >
> >
> > On Tue, Jun 14, 2016 at 1:09 PM Paul Michali <p...@michali.net
> > <mailto:p...@michali.net>> wrote:
> >
> > Great info Chris and thanks for confirming the assignment of blocks
> > of pages to a numa node.
> >
> > I'm still struggling with why each VM is being assigned to NUMA node
> > 0. Any ideas on where I should look to see why Nova is not using
> > NUMA id 1?
> >
> > Thanks!
> >
> >
> > PCM
> >
> >
> > On Tue, Jun 14, 2016 at 10:29 AM Chris Friesen
> > <chris.frie...@windriver.com <mailto:chris.frie...@windriver.com>>
> > wrote:
> >
> > On 06/13/2016 02:17 PM, Paul Michali wrote:
> >  > Hmm... I tried Friday and again today, and I'm not seeing the
> > VMs being evenly
> >  > created on the NUMA nodes. Every Cirros VM is created on
> > nodeid 0.
> >  >
> >  > I have the m1/small flavor (@GB) selected and am using
> > hw:numa_nodes=1 and
> >  > hw:mem_page_size=2048 flavor-key settings. Each VM is
> > consuming 1024 huge pages
> >  > (of size 2MB), but is on nodeid 0 always. Also, it seems that
> > when I reach 1/2
> >  > of the total number of huge pages used, libvirt gives an
> > error saying there is
> >  > not enough memory to create the VM. Is it expected that the
> > huge pages are
> >  > "allocated" to each NUMA node?
> >
> > Yes, any given memory page exists on one NUMA node, and a
> > single-NUMA-node VM
> > will be constrained to a single host NUMA node and will use
> > memory from that
> > host NUMA node.
> >
> > You can see and/or adjust how many hugepages are available on
> > each NUMA node via
> > /sys/devices/system/node/nodeX/hugepages/hugepages-2048kB/*
> > where X is the host
> > NUMA node number.
> >
> > Chris
> >
> >
> >
>  __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > <
> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
> >
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Well, looks like we figured out what is going on - maybe folks have some
ideas on how we could handle this issue.

What I see is that for each VM create (small flavor), 1024 huge pages are
used and NUMA node 0 used. It appears that, when there is no longer enough
huge pages on that NUMA node, Nova with then schedule to the other NUMA
node and use those huge pages.

In our case, we happen to have a special container running on the compute
nodes, that uses 512 huge pages. As a result, when there are 768 huge pages
left, Nova thinks there are 1280 pages left and thinks one more VM can be
create. It tries, but the create fails.

Some questions...

1) Is there some way to "reserve" huge pages in Nova?
2) If the create fails, should Nova try the other NUMA node (or is this
because it doesn't know why it failed)?
3) Any ideas on how we can deal with this - without changing Nova?

Thanks!

PCM



On Tue, Jun 14, 2016 at 1:09 PM Paul Michali <p...@michali.net> wrote:

> Great info Chris and thanks for confirming the assignment of blocks of
> pages to a numa node.
>
> I'm still struggling with why each VM is being assigned to NUMA node 0.
> Any ideas on where I should look to see why Nova is not using NUMA id 1?
>
> Thanks!
>
>
> PCM
>
>
> On Tue, Jun 14, 2016 at 10:29 AM Chris Friesen <
> chris.frie...@windriver.com> wrote:
>
>> On 06/13/2016 02:17 PM, Paul Michali wrote:
>> > Hmm... I tried Friday and again today, and I'm not seeing the VMs being
>> evenly
>> > created on the NUMA nodes. Every Cirros VM is created on nodeid 0.
>> >
>> > I have the m1/small flavor (@GB) selected and am using hw:numa_nodes=1
>> and
>> > hw:mem_page_size=2048 flavor-key settings. Each VM is consuming 1024
>> huge pages
>> > (of size 2MB), but is on nodeid 0 always. Also, it seems that when I
>> reach 1/2
>> > of the total number of huge pages used, libvirt gives an error saying
>> there is
>> > not enough memory to create the VM. Is it expected that the huge pages
>> are
>> > "allocated" to each NUMA node?
>>
>> Yes, any given memory page exists on one NUMA node, and a
>> single-NUMA-node VM
>> will be constrained to a single host NUMA node and will use memory from
>> that
>> host NUMA node.
>>
>> You can see and/or adjust how many hugepages are available on each NUMA
>> node via
>> /sys/devices/system/node/nodeX/hugepages/hugepages-2048kB/* where X is
>> the host
>> NUMA node number.
>>
>> Chris
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [VPNaaS] Support for Stronger hashes and combined mode ciphers

[Paul Michali]

I think Kyle polled operators and a few mentioned using VPNaaS for
site-to-site IPSec - do a search in this ML for VPNaaS. AFAIK, no one so
far is stepping up to work on VPNaaS.

Regards,

PCM


On Tue, Jun 14, 2016 at 1:40 PM Mark Fenwick <mark.fenw...@oracle.com>
wrote:

> Hi Paul,
>
> On 06/14/16 10:27, Paul Michali wrote:
> > Certainly the ciphers and hashes could be enhanced for VPNaaS. This would
> > require converting the user selections into options for the underlying
> > device driver, modifying the neutron client (OSC) to allow entry of the
> new
> > selections, updating unit tests, and likely adding some validators to
> > reject these options on drivers that may not support them (e.g. if
> OpenSwan
> > doesn't support an option, you'll want to reject it).
> >
>
> I made some changes and got this working quiet quickly, would need some
> polish.
>
> > There is not an active VPNaaS team any more, so, if this is something
> that
> > you'd like to see, you'll need to provide some sweat equity to make it
> > happen. There are still some people that can core review changes, but
> don't
> > expect much community support for VPNaaS at this time. In fact, I think
> the
> > plan is to archive/mothball/whatever VPNaaS in a few months (it's on
> double
> > secret probation :)), if there is no-one actively supporting it (I'll
> leave
> > to the PTL to define what "support" means - not sure what the
> > qualifications will be to maintain this project).
>
> So I'm curious, does anybody actually use VPNaaS for anything ?
>
> Thanks
>
> Mark
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [VPNaaS] Support for Stronger hashes and combined mode ciphers

[Paul Michali]

Certainly the ciphers and hashes could be enhanced for VPNaaS. This would
require converting the user selections into options for the underlying
device driver, modifying the neutron client (OSC) to allow entry of the new
selections, updating unit tests, and likely adding some validators to
reject these options on drivers that may not support them (e.g. if OpenSwan
doesn't support an option, you'll want to reject it).

There is not an active VPNaaS team any more, so, if this is something that
you'd like to see, you'll need to provide some sweat equity to make it
happen. There are still some people that can core review changes, but don't
expect much community support for VPNaaS at this time. In fact, I think the
plan is to archive/mothball/whatever VPNaaS in a few months (it's on double
secret probation :)), if there is no-one actively supporting it (I'll leave
to the PTL to define what "support" means - not sure what the
qualifications will be to maintain this project).

Regards,

PCM


On Wed, Jun 8, 2016 at 5:19 PM Mark Fenwick  wrote:

> Hi,
>
> I was wondering if there are any plans to extend support for IPsec and
> IKE algorithms. Looks like only AES-CBC mode and SHA1 are supported.
>
> It would be nice to see:
>
> SHA256, SHA384, SHA512
>
> As well as the combined mode ciphers:
>
> AES-CCM and AES-GCM
>
> StrongSWAN already supports all of these ciphers and hashes.
>
> Thanks
>
> Mark
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Great info Chris and thanks for confirming the assignment of blocks of
pages to a numa node.

I'm still struggling with why each VM is being assigned to NUMA node 0. Any
ideas on where I should look to see why Nova is not using NUMA id 1?

Thanks!


PCM


On Tue, Jun 14, 2016 at 10:29 AM Chris Friesen <chris.frie...@windriver.com>
wrote:

> On 06/13/2016 02:17 PM, Paul Michali wrote:
> > Hmm... I tried Friday and again today, and I'm not seeing the VMs being
> evenly
> > created on the NUMA nodes. Every Cirros VM is created on nodeid 0.
> >
> > I have the m1/small flavor (@GB) selected and am using hw:numa_nodes=1
> and
> > hw:mem_page_size=2048 flavor-key settings. Each VM is consuming 1024
> huge pages
> > (of size 2MB), but is on nodeid 0 always. Also, it seems that when I
> reach 1/2
> > of the total number of huge pages used, libvirt gives an error saying
> there is
> > not enough memory to create the VM. Is it expected that the huge pages
> are
> > "allocated" to each NUMA node?
>
> Yes, any given memory page exists on one NUMA node, and a single-NUMA-node
> VM
> will be constrained to a single host NUMA node and will use memory from
> that
> host NUMA node.
>
> You can see and/or adjust how many hugepages are available on each NUMA
> node via
> /sys/devices/system/node/nodeX/hugepages/hugepages-2048kB/* where X is the
> host
> NUMA node number.
>
> Chris
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Hmm... I tried Friday and again today, and I'm not seeing the VMs being
evenly created on the NUMA nodes. Every Cirros VM is created on nodeid 0.

I have the m1/small flavor (@GB) selected and am using hw:numa_nodes=1 and
hw:mem_page_size=2048 flavor-key settings. Each VM is consuming 1024 huge
pages (of size 2MB), but is on nodeid 0 always. Also, it seems that when I
reach 1/2 of the total number of huge pages used, libvirt gives an error
saying there is not enough memory to create the VM. Is it expected that the
huge pages are "allocated" to each NUMA node?

I don't know why I cannot repeat what I did on 6/3, where I changed
 hw:mem_page_size from "large" to "2048"and it worked, allocation to each
of the two NUMA nodes. :(

Regards,

PCM


On Fri, Jun 10, 2016 at 9:16 AM Paul Michali <p...@michali.net> wrote:

> Actually, I had menm_page_size set to "large" and not "1024". However, it
> seemed like it was using 1024 pages per (small VM creation). Is there
> possibly some issue with large not using one of the supported values? I
> would have guessed it would have chosen 2M or 1G for the size.
>
> Any thoughts?
>
> PCM
>
> On Fri, Jun 10, 2016 at 9:05 AM Paul Michali <p...@michali.net> wrote:
>
>> Thanks Daniel and Chris! I think that was the problem, I had configured
>> Nova flavor with a mem_page_size of 1024, and it should have been one of
>> the supported values.
>>
>> I'll go through and check things out one more time, but I think that is
>> the problem. I still need to figure out what is going on with the neutron
>> port not being released - we have another person in my group who has seen
>> the same issue.
>>
>> Regards,
>>
>> PCM
>>
>> On Fri, Jun 10, 2016 at 4:41 AM Daniel P. Berrange <berra...@redhat.com>
>> wrote:
>>
>>> On Thu, Jun 09, 2016 at 12:35:06PM -0600, Chris Friesen wrote:
>>> > On 06/09/2016 05:15 AM, Paul Michali wrote:
>>> > > 1) On the host, I was seeing 32768 huge pages, of 2MB size.
>>> >
>>> > Please check the number of huge pages _per host numa node_.
>>> >
>>> > > 2) I changed mem_page_size from 1024 to 2048 in the flavor, and then
>>> when VMs
>>> > > were created, they were being evenly assigned to the two NUMA nodes.
>>> Each using
>>> > > 1024 huge pages. At this point I could create more than half, but
>>> when there
>>> > > were 1945 pages left, it failed to create a VM. Did it fail because
>>> the
>>> > > mem_page_size was 2048 and the available pages were 1945, even
>>> though we were
>>> > > only requesting 1024 pages?
>>> >
>>> > I do not think that "1024" is a valid page size (at least for x86).
>>>
>>> Correct, 4k, 2M and 1GB are valid page sizes.
>>>
>>> > Valid mem_page_size values are determined by the host CPU.  You do not
>>> need
>>> > a larger page size for flavors with larger memory sizes.
>>>
>>> Though note that page sizes should be a multiple of favour mem size
>>> unless you want to waste memory. eg if you have a flavour with 750MB
>>> RAM, then you probably don't want to use 1GB pages as it waste 250MB
>>>
>>> Regards,
>>> Daniel
>>> --
>>> |: http://berrange.com  -o-
>>> http://www.flickr.com/photos/dberrange/ :|
>>> |: http://libvirt.org  -o-
>>> http://virt-manager.org :|
>>> |: http://autobuild.org   -o-
>>> http://search.cpan.org/~danberr/ :|
>>> |: http://entangle-photo.org   -o-
>>> http://live.gnome.org/gtk-vnc :|
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Actually, I had menm_page_size set to "large" and not "1024". However, it
seemed like it was using 1024 pages per (small VM creation). Is there
possibly some issue with large not using one of the supported values? I
would have guessed it would have chosen 2M or 1G for the size.

Any thoughts?

PCM

On Fri, Jun 10, 2016 at 9:05 AM Paul Michali <p...@michali.net> wrote:

> Thanks Daniel and Chris! I think that was the problem, I had configured
> Nova flavor with a mem_page_size of 1024, and it should have been one of
> the supported values.
>
> I'll go through and check things out one more time, but I think that is
> the problem. I still need to figure out what is going on with the neutron
> port not being released - we have another person in my group who has seen
> the same issue.
>
> Regards,
>
> PCM
>
> On Fri, Jun 10, 2016 at 4:41 AM Daniel P. Berrange <berra...@redhat.com>
> wrote:
>
>> On Thu, Jun 09, 2016 at 12:35:06PM -0600, Chris Friesen wrote:
>> > On 06/09/2016 05:15 AM, Paul Michali wrote:
>> > > 1) On the host, I was seeing 32768 huge pages, of 2MB size.
>> >
>> > Please check the number of huge pages _per host numa node_.
>> >
>> > > 2) I changed mem_page_size from 1024 to 2048 in the flavor, and then
>> when VMs
>> > > were created, they were being evenly assigned to the two NUMA nodes.
>> Each using
>> > > 1024 huge pages. At this point I could create more than half, but
>> when there
>> > > were 1945 pages left, it failed to create a VM. Did it fail because
>> the
>> > > mem_page_size was 2048 and the available pages were 1945, even though
>> we were
>> > > only requesting 1024 pages?
>> >
>> > I do not think that "1024" is a valid page size (at least for x86).
>>
>> Correct, 4k, 2M and 1GB are valid page sizes.
>>
>> > Valid mem_page_size values are determined by the host CPU.  You do not
>> need
>> > a larger page size for flavors with larger memory sizes.
>>
>> Though note that page sizes should be a multiple of favour mem size
>> unless you want to waste memory. eg if you have a flavour with 750MB
>> RAM, then you probably don't want to use 1GB pages as it waste 250MB
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com  -o-
>> http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org  -o-
>> http://virt-manager.org :|
>> |: http://autobuild.org   -o-
>> http://search.cpan.org/~danberr/ :|
>> |: http://entangle-photo.org   -o-
>> http://live.gnome.org/gtk-vnc :|
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Thanks Daniel and Chris! I think that was the problem, I had configured
Nova flavor with a mem_page_size of 1024, and it should have been one of
the supported values.

I'll go through and check things out one more time, but I think that is the
problem. I still need to figure out what is going on with the neutron port
not being released - we have another person in my group who has seen the
same issue.

Regards,

PCM

On Fri, Jun 10, 2016 at 4:41 AM Daniel P. Berrange <berra...@redhat.com>
wrote:

> On Thu, Jun 09, 2016 at 12:35:06PM -0600, Chris Friesen wrote:
> > On 06/09/2016 05:15 AM, Paul Michali wrote:
> > > 1) On the host, I was seeing 32768 huge pages, of 2MB size.
> >
> > Please check the number of huge pages _per host numa node_.
> >
> > > 2) I changed mem_page_size from 1024 to 2048 in the flavor, and then
> when VMs
> > > were created, they were being evenly assigned to the two NUMA nodes.
> Each using
> > > 1024 huge pages. At this point I could create more than half, but when
> there
> > > were 1945 pages left, it failed to create a VM. Did it fail because the
> > > mem_page_size was 2048 and the available pages were 1945, even though
> we were
> > > only requesting 1024 pages?
> >
> > I do not think that "1024" is a valid page size (at least for x86).
>
> Correct, 4k, 2M and 1GB are valid page sizes.
>
> > Valid mem_page_size values are determined by the host CPU.  You do not
> need
> > a larger page size for flavors with larger memory sizes.
>
> Though note that page sizes should be a multiple of favour mem size
> unless you want to waste memory. eg if you have a flavour with 750MB
> RAM, then you probably don't want to use 1GB pages as it waste 250MB
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/
> :|
> |: http://libvirt.org  -o- http://virt-manager.org
> :|
> |: http://autobuild.org   -o- http://search.cpan.org/~danberr/
> :|
> |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc
> :|
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

See PCM: Inline...


On Thu, Jun 9, 2016 at 11:42 AM Steve Gordon <sgor...@redhat.com> wrote:

> - Original Message -
> > From: "Paul Michali" <p...@michali.net>
> > To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> > Sent: Tuesday, June 7, 2016 11:00:30 AM
> > Subject: Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling
> >
> > Anyone have any thoughts on the two questions below? Namely...
> >
> > If the huge pages are 2M, we are creating a 2GB VM, have 1945 huge pages,
> > should the allocation fail (and if so why)?
>
> Were enough pages (1024) available in a single NUMA node? Which release
> are you using? There was a bug where node 0 would always be picked (and
> eventually exhausted) but that was - theoretically - fixed under
> https://bugs.launchpad.net/nova/+bug/1386236


PCM: This is on LIberty, so it sounds like the bugfix was in there.  It's
possible that there was not 1024 left, on a single NUMA node.

Regards,

PCM


>
>
> > Why do all the 2GB VMs get created on the same NUMA node, instead of
> > getting evenly assigned to each of the two NUMA nodes that are available
> on
> > the compute node (as a result, allocation fails, when 1/2 the huge pages
> > are used)? I found that increasing mem_page_size to 2048 resolves the
> > issue, but don't know why.
>
> What was the mem_page_size before it was 2048? I didn't think any smaller
> value was supported.
>
> > ANother thing I was seeing, when the VM create failed due to not enough
> > huge pages available and was in error state, I could delete the VM, but
> the
> > Neutron port was still there.  Is that correct?
> >
> > I didn't see any log messages in neutron, requesting to unbind and delete
> > the port.
> >
> > Thanks!
> >
> > PCM
> >
> > .
> >
> > On Fri, Jun 3, 2016 at 2:03 PM Paul Michali <p...@michali.net> wrote:
> >
> > > Thanks for the link Tim!
> > >
> > > Right now, I have two things I'm unsure about...
> > >
> > > One is that I had 1945 huge pages left (of size 2048k) and tried to
> create
> > > a VM with a small flavor (2GB), which should need 1024 pages, but Nova
> > > indicated that it wasn't able to find a host (and QEMU reported an
> > > allocation issue).
> > >
> > > The other is that VMs are not being evenly distributed on my two NUMA
> > > nodes, and instead, are getting created all on one NUMA node. Not sure
> if
> > > that is expected (and setting mem_page_size to 2048 is the proper way).
> > >
> > > Regards,
> > >
> > > PCM
> > >
> > >
> > > On Fri, Jun 3, 2016 at 1:21 PM Tim Bell <tim.b...@cern.ch> wrote:
> > >
> > >> The documentation at
> > >> http://docs.openstack.org/admin-guide/compute-flavors.html is
> gradually
> > >> improving. Are there areas which were not covered in your
> clarifications ?
> > >> If so, we should fix the documentation too since this is a complex
> area to
> > >> configure and good documentation is a great help.
> > >>
> > >>
> > >>
> > >> BTW, there is also an issue around how the RAM for the BIOS is
> shadowed.
> > >> I can’t find the page from a quick google but we found an imbalance
> when
> > >> we
> > >> used 2GB pages as the RAM for BIOS shadowing was done by default in
> the
> > >> memory space for only one of the NUMA spaces.
> > >>
> > >>
> > >>
> > >> Having a look at the KVM XML can also help a bit if you are debugging.
> > >>
> > >>
> > >>
> > >> Tim
> > >>
> > >>
> > >>
> > >> *From: *Paul Michali <p...@michali.net>
> > >> *Reply-To: *"OpenStack Development Mailing List (not for usage
> > >> questions)" <openstack-dev@lists.openstack.org>
> > >> *Date: *Friday 3 June 2016 at 15:18
> > >> *To: *"Daniel P. Berrange" <berra...@redhat.com>, "OpenStack
> Development
> > >> Mailing List (not for usage questions)" <
> > >> openstack-dev@lists.openstack.org>
> > >> *Subject: *Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling
> > >>
> > >>
> > >>
> > >> See PCM inline...
> > >>
> > >> On Fri, Jun 3, 2016 at 8:44 AM Daniel P. Berrange &

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

I'll try to reproduce and collect logs for a bug report.

Thanks for the info.

PCM


On Thu, Jun 9, 2016 at 9:43 AM Matt Riedemann <mrie...@linux.vnet.ibm.com>
wrote:

> On 6/9/2016 6:15 AM, Paul Michali wrote:
> >
> >
> > On Wed, Jun 8, 2016 at 11:21 PM Chris Friesen
> > <chris.frie...@windriver.com <mailto:chris.frie...@windriver.com>>
> wrote:
> >
> > On 06/03/2016 12:03 PM, Paul Michali wrote:
> > > Thanks for the link Tim!
> > >
> > > Right now, I have two things I'm unsure about...
> > >
> > > One is that I had 1945 huge pages left (of size 2048k) and tried
> > to create a VM
> > > with a small flavor (2GB), which should need 1024 pages, but Nova
> > indicated that
> > > it wasn't able to find a host (and QEMU reported an allocation
> issue).
> > >
> > > The other is that VMs are not being evenly distributed on my two
> > NUMA nodes, and
> > > instead, are getting created all on one NUMA node. Not sure if
> > that is expected
> > > (and setting mem_page_size to 2048 is the proper way).
> >
> >
> > Just in case you haven't figured out the problem...
> >
> > Have you checked the per-host-numa-node 2MB huge page availability
> > on your host?
> >   If it's uneven then that might explain what you're seeing.
> >
> >
> > These are the observations/questions I have:
> >
> > 1) On the host, I was seeing 32768 huge pages, of 2MB size. When I
> > created VMs (Cirros) using small flavor, each VM was getting created on
> > NUMA nodeid 0. When it hit half of the available pages, I could no
> > longer create any VMs (QEMU saying no space). I'd like to understand why
> > the assignment was always going two nodeid 0, and to confirm that the
> > huge pages are divided among the number of NUMA nodes available.
> >
> > 2) I changed mem_page_size from 1024 to 2048 in the flavor, and then
> > when VMs were created, they were being evenly assigned to the two NUMA
> > nodes. Each using 1024 huge pages. At this point I could create more
> > than half, but when there were 1945 pages left, it failed to create a
> > VM. Did it fail because the mem_page_size was 2048 and the available
> > pages were 1945, even though we were only requesting 1024 pages?
> >
> > 3) Related to #2, is there a relationship between mem_page_size, the
> > allocation of VMs to NUMA nodes, and the flavor size? IOW, if I use the
> > medium flavor (4GB), will I need a larger mem_page_size? (I'll play with
> > this variation, as soon as I can). Gets back to understanding how the
> > scheduling determines how to assign the VMs.
> >
> > 4) When the VM create failed due to QEMU failing allocation, the VM went
> > to error state. I deleted the VM, but the neutron port was still there,
> > and there were no log messages indicating that a request was made to
> > delete the port. Is this expected (that the user would have to manually
> > clean up the port)?
>
> When you hit this case, can you check if instance.host is set in the
> database before deleting the instance? I'm guessing what's happening is
> the instance didn't get assigned a host since it eventually ended up
> with NoValidHost, so when you go to delete it doesn't have a compute to
> send it to for delete, so it deletes from the compute API, and we don't
> have the host binding details to delete the port.
>
> Although, when the spawn failed in the compute to begin with we should
> have deallocated any networking that was created before kicking back to
> the scheduler - unless we don't go back to the scheduler if the instance
> is set to ERROR state.
>
> A bug report with stacktrace of the failure scenario when the instance
> goes to error state bug n-cpu logs would probably help.
>
> >
> > 5) A coworker had hit the problem mentioned in #1, with exhaustion at
> > the halfway point. If she delete's a VM, and then changes the flavor to
> > change the mem_page_size to 2048, should Nova start assigning all new
> > VMs to the other NUMA node, until the pool of huge pages is down to
> > where the huge pages are for NUMA node 0, or will it alternate between
> > the available NUMA nodes (and run out when node 0's pool is exhausted)?
> >
> > Thanks in advance!
> >
> > PCM
> >
> >
> >
> >
> > Chris
> >
> >
>  __
> > OpenStack Development Mailing List (not for usage questions)
> >   

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

On Wed, Jun 8, 2016 at 11:21 PM Chris Friesen <chris.frie...@windriver.com>
wrote:

> On 06/03/2016 12:03 PM, Paul Michali wrote:
> > Thanks for the link Tim!
> >
> > Right now, I have two things I'm unsure about...
> >
> > One is that I had 1945 huge pages left (of size 2048k) and tried to
> create a VM
> > with a small flavor (2GB), which should need 1024 pages, but Nova
> indicated that
> > it wasn't able to find a host (and QEMU reported an allocation issue).
> >
> > The other is that VMs are not being evenly distributed on my two NUMA
> nodes, and
> > instead, are getting created all on one NUMA node. Not sure if that is
> expected
> > (and setting mem_page_size to 2048 is the proper way).
>
>
> Just in case you haven't figured out the problem...
>
> Have you checked the per-host-numa-node 2MB huge page availability on your
> host?
>   If it's uneven then that might explain what you're seeing.
>

These are the observations/questions I have:

1) On the host, I was seeing 32768 huge pages, of 2MB size. When I created
VMs (Cirros) using small flavor, each VM was getting created on NUMA nodeid
0. When it hit half of the available pages, I could no longer create any
VMs (QEMU saying no space). I'd like to understand why the assignment was
always going two nodeid 0, and to confirm that the huge pages are divided
among the number of NUMA nodes available.

2) I changed mem_page_size from 1024 to 2048 in the flavor, and then when
VMs were created, they were being evenly assigned to the two NUMA nodes.
Each using 1024 huge pages. At this point I could create more than half,
but when there were 1945 pages left, it failed to create a VM. Did it fail
because the mem_page_size was 2048 and the available pages were 1945, even
though we were only requesting 1024 pages?

3) Related to #2, is there a relationship between mem_page_size, the
allocation of VMs to NUMA nodes, and the flavor size? IOW, if I use the
medium flavor (4GB), will I need a larger mem_page_size? (I'll play with
this variation, as soon as I can). Gets back to understanding how the
scheduling determines how to assign the VMs.

4) When the VM create failed due to QEMU failing allocation, the VM went to
error state. I deleted the VM, but the neutron port was still there, and
there were no log messages indicating that a request was made to delete the
port. Is this expected (that the user would have to manually clean up the
port)?

5) A coworker had hit the problem mentioned in #1, with exhaustion at the
halfway point. If she delete's a VM, and then changes the flavor to change
the mem_page_size to 2048, should Nova start assigning all new VMs to the
other NUMA node, until the pool of huge pages is down to where the huge
pages are for NUMA node 0, or will it alternate between the available NUMA
nodes (and run out when node 0's pool is exhausted)?

Thanks in advance!

PCM




> Chris
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Anyone have any thoughts on the two questions below? Namely...

If the huge pages are 2M, we are creating a 2GB VM, have 1945 huge pages,
should the allocation fail (and if so why)?

Why do all the 2GB VMs get created on the same NUMA node, instead of
getting evenly assigned to each of the two NUMA nodes that are available on
the compute node (as a result, allocation fails, when 1/2 the huge pages
are used)? I found that increasing mem_page_size to 2048 resolves the
issue, but don't know why.

ANother thing I was seeing, when the VM create failed due to not enough
huge pages available and was in error state, I could delete the VM, but the
Neutron port was still there.  Is that correct?

I didn't see any log messages in neutron, requesting to unbind and delete
the port.

Thanks!

PCM

.

On Fri, Jun 3, 2016 at 2:03 PM Paul Michali <p...@michali.net> wrote:

> Thanks for the link Tim!
>
> Right now, I have two things I'm unsure about...
>
> One is that I had 1945 huge pages left (of size 2048k) and tried to create
> a VM with a small flavor (2GB), which should need 1024 pages, but Nova
> indicated that it wasn't able to find a host (and QEMU reported an
> allocation issue).
>
> The other is that VMs are not being evenly distributed on my two NUMA
> nodes, and instead, are getting created all on one NUMA node. Not sure if
> that is expected (and setting mem_page_size to 2048 is the proper way).
>
> Regards,
>
> PCM
>
>
> On Fri, Jun 3, 2016 at 1:21 PM Tim Bell <tim.b...@cern.ch> wrote:
>
>> The documentation at
>> http://docs.openstack.org/admin-guide/compute-flavors.html is gradually
>> improving. Are there areas which were not covered in your clarifications ?
>> If so, we should fix the documentation too since this is a complex area to
>> configure and good documentation is a great help.
>>
>>
>>
>> BTW, there is also an issue around how the RAM for the BIOS is shadowed.
>> I can’t find the page from a quick google but we found an imbalance when we
>> used 2GB pages as the RAM for BIOS shadowing was done by default in the
>> memory space for only one of the NUMA spaces.
>>
>>
>>
>> Having a look at the KVM XML can also help a bit if you are debugging.
>>
>>
>>
>> Tim
>>
>>
>>
>> *From: *Paul Michali <p...@michali.net>
>> *Reply-To: *"OpenStack Development Mailing List (not for usage
>> questions)" <openstack-dev@lists.openstack.org>
>> *Date: *Friday 3 June 2016 at 15:18
>> *To: *"Daniel P. Berrange" <berra...@redhat.com>, "OpenStack Development
>> Mailing List (not for usage questions)" <
>> openstack-dev@lists.openstack.org>
>> *Subject: *Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling
>>
>>
>>
>> See PCM inline...
>>
>> On Fri, Jun 3, 2016 at 8:44 AM Daniel P. Berrange <berra...@redhat.com>
>> wrote:
>>
>> On Fri, Jun 03, 2016 at 12:32:17PM +, Paul Michali wrote:
>> > Hi!
>> >
>> > I've been playing with Liberty code a bit and had some questions that
>> I'm
>> > hoping Nova folks may be able to provide guidance on...
>> >
>> > If I set up a flavor with hw:mem_page_size=2048, and I'm creating
>> (Cirros)
>> > VMs with size 1024, will the scheduling use the minimum of the number of
>>
>> 1024 what units ? 1024 MB, or 1024 huge pages aka 2048 MB ?
>>
>>
>>
>> PCM: I was using small flavor, which is 2 GB. So that's 2048 MB and the
>> page size is 2048K, so 1024 pages? Hope I have the units right.
>>
>>
>>
>>
>>
>>
>> > huge pages available and the size requested for the VM, or will it base
>> > scheduling only on the number of huge pages?
>> >
>> > It seems to be doing the latter, where I had 1945 huge pages free, and
>> > tried to create another VM (1024) and Nova rejected the request with "no
>> > hosts available".
>>
>> From this I'm guessing you're meaning 1024 huge pages aka 2 GB earlier.
>>
>> Anyway, when you request huge pages to be used for a flavour, the
>> entire guest RAM must be able to be allocated from huge pages.
>> ie if you have a guest with 2 GB of RAM, you must have 2 GB worth
>> of huge pages available. It is not possible for a VM to use
>> 1.5 GB of huge pages and 500 MB of normal sized pages.
>>
>>
>>
>> PCM: Right, so, with 2GB of RAM, I need 1024 huge pages of size 2048K. In
>> this case, there are 1945 huge pages available, so I was wondering why it
>> failed. Maybe I'm confusing sizes/pages?
&g

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Thanks for the link Tim!

Right now, I have two things I'm unsure about...

One is that I had 1945 huge pages left (of size 2048k) and tried to create
a VM with a small flavor (2GB), which should need 1024 pages, but Nova
indicated that it wasn't able to find a host (and QEMU reported an
allocation issue).

The other is that VMs are not being evenly distributed on my two NUMA
nodes, and instead, are getting created all on one NUMA node. Not sure if
that is expected (and setting mem_page_size to 2048 is the proper way).

Regards,

PCM


On Fri, Jun 3, 2016 at 1:21 PM Tim Bell <tim.b...@cern.ch> wrote:

> The documentation at
> http://docs.openstack.org/admin-guide/compute-flavors.html is gradually
> improving. Are there areas which were not covered in your clarifications ?
> If so, we should fix the documentation too since this is a complex area to
> configure and good documentation is a great help.
>
>
>
> BTW, there is also an issue around how the RAM for the BIOS is shadowed. I
> can’t find the page from a quick google but we found an imbalance when we
> used 2GB pages as the RAM for BIOS shadowing was done by default in the
> memory space for only one of the NUMA spaces.
>
>
>
> Having a look at the KVM XML can also help a bit if you are debugging.
>
>
>
> Tim
>
>
>
> *From: *Paul Michali <p...@michali.net>
> *Reply-To: *"OpenStack Development Mailing List (not for usage
> questions)" <openstack-dev@lists.openstack.org>
> *Date: *Friday 3 June 2016 at 15:18
> *To: *"Daniel P. Berrange" <berra...@redhat.com>, "OpenStack Development
> Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org
> >
> *Subject: *Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling
>
>
>
> See PCM inline...
>
> On Fri, Jun 3, 2016 at 8:44 AM Daniel P. Berrange <berra...@redhat.com>
> wrote:
>
> On Fri, Jun 03, 2016 at 12:32:17PM +, Paul Michali wrote:
> > Hi!
> >
> > I've been playing with Liberty code a bit and had some questions that I'm
> > hoping Nova folks may be able to provide guidance on...
> >
> > If I set up a flavor with hw:mem_page_size=2048, and I'm creating
> (Cirros)
> > VMs with size 1024, will the scheduling use the minimum of the number of
>
> 1024 what units ? 1024 MB, or 1024 huge pages aka 2048 MB ?
>
>
>
> PCM: I was using small flavor, which is 2 GB. So that's 2048 MB and the
> page size is 2048K, so 1024 pages? Hope I have the units right.
>
>
>
>
>
>
> > huge pages available and the size requested for the VM, or will it base
> > scheduling only on the number of huge pages?
> >
> > It seems to be doing the latter, where I had 1945 huge pages free, and
> > tried to create another VM (1024) and Nova rejected the request with "no
> > hosts available".
>
> From this I'm guessing you're meaning 1024 huge pages aka 2 GB earlier.
>
> Anyway, when you request huge pages to be used for a flavour, the
> entire guest RAM must be able to be allocated from huge pages.
> ie if you have a guest with 2 GB of RAM, you must have 2 GB worth
> of huge pages available. It is not possible for a VM to use
> 1.5 GB of huge pages and 500 MB of normal sized pages.
>
>
>
> PCM: Right, so, with 2GB of RAM, I need 1024 huge pages of size 2048K. In
> this case, there are 1945 huge pages available, so I was wondering why it
> failed. Maybe I'm confusing sizes/pages?
>
>
>
>
>
>
> > Is this still the same for Mitaka?
>
> Yep, this use of huge pages has not changed.
>
> > Where could I look in the code to see how the scheduling is determined?
>
> Most logic related to huge pages is in nova/virt/hardware.py
>
> > If I use mem_page_size=large (what I originally had), should it evenly
> > assign huge pages from the available NUMA nodes (there are two in my
> case)?
> >
> > It looks like it was assigning all VMs to the same NUMA node (0) in this
> > case. Is the right way to change to 2048, like I did above?
>
> Nova will always avoid spreading your VM across 2 host NUMA nodes,
> since that gives bad performance characteristics. IOW, it will always
> allocate huge pages from the NUMA node that the guest will run on. If
> you explicitly want your VM to spread across 2 host NUMA nodes, then
> you must tell nova to create 2 *guest* NUMA nodes for the VM. Nova
> will then place each guest NUMA node, on a separate host NUMA node
> and allocate huge pages from node to match. This is done using
> the hw:numa_nodes=2 parameter on the flavour
>
>
>
> PCM: Gotcha, but that was not the issue I'm seeing. With this small flavor
> (2GB = 1024 pages)

Re: [openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

See PCM inline...

On Fri, Jun 3, 2016 at 8:44 AM Daniel P. Berrange <berra...@redhat.com>
wrote:

> On Fri, Jun 03, 2016 at 12:32:17PM +0000, Paul Michali wrote:
> > Hi!
> >
> > I've been playing with Liberty code a bit and had some questions that I'm
> > hoping Nova folks may be able to provide guidance on...
> >
> > If I set up a flavor with hw:mem_page_size=2048, and I'm creating
> (Cirros)
> > VMs with size 1024, will the scheduling use the minimum of the number of
>
> 1024 what units ? 1024 MB, or 1024 huge pages aka 2048 MB ?
>

PCM: I was using small flavor, which is 2 GB. So that's 2048 MB and the
page size is 2048K, so 1024 pages? Hope I have the units right.



> > huge pages available and the size requested for the VM, or will it base
> > scheduling only on the number of huge pages?
> >
> > It seems to be doing the latter, where I had 1945 huge pages free, and
> > tried to create another VM (1024) and Nova rejected the request with "no
> > hosts available".
>
> From this I'm guessing you're meaning 1024 huge pages aka 2 GB earlier.
>
> Anyway, when you request huge pages to be used for a flavour, the
> entire guest RAM must be able to be allocated from huge pages.
> ie if you have a guest with 2 GB of RAM, you must have 2 GB worth
> of huge pages available. It is not possible for a VM to use
> 1.5 GB of huge pages and 500 MB of normal sized pages.
>

PCM: Right, so, with 2GB of RAM, I need 1024 huge pages of size 2048K. In
this case, there are 1945 huge pages available, so I was wondering why it
failed. Maybe I'm confusing sizes/pages?



>
> > Is this still the same for Mitaka?
>
> Yep, this use of huge pages has not changed.
>
> > Where could I look in the code to see how the scheduling is determined?
>
> Most logic related to huge pages is in nova/virt/hardware.py
>
> > If I use mem_page_size=large (what I originally had), should it evenly
> > assign huge pages from the available NUMA nodes (there are two in my
> case)?
> >
> > It looks like it was assigning all VMs to the same NUMA node (0) in this
> > case. Is the right way to change to 2048, like I did above?
>
> Nova will always avoid spreading your VM across 2 host NUMA nodes,
> since that gives bad performance characteristics. IOW, it will always
> allocate huge pages from the NUMA node that the guest will run on. If
> you explicitly want your VM to spread across 2 host NUMA nodes, then
> you must tell nova to create 2 *guest* NUMA nodes for the VM. Nova
> will then place each guest NUMA node, on a separate host NUMA node
> and allocate huge pages from node to match. This is done using
> the hw:numa_nodes=2 parameter on the flavour
>

PCM: Gotcha, but that was not the issue I'm seeing. With this small flavor
(2GB = 1024 pages), I had 13107 huge pages initially. As I created VMs,
they were *all* placed on the same NUMA node (0). As a result, when I got
to more than have the available pages, Nova failed to allow further VMs,
even though I had 6963 available on one compute node, and 5939 on another.

It seems that all the assignments were to node zero. Someone suggested to
me to set mem_page_size to 2048, and at that point it started assigning to
both NUMA nodes evenly.

Thanks for the help!!!


Regards,

PCM


>
> > Again, has this changed at all in Mitaka?
>
> Nope. Well aside from random bug fixes.
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/
> :|
> |: http://libvirt.org  -o- http://virt-manager.org
> :|
> |: http://autobuild.org   -o- http://search.cpan.org/~danberr/
> :|
> |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc
> :|
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] NUMA, huge pages, and scheduling

[Paul Michali]

Hi!

I've been playing with Liberty code a bit and had some questions that I'm
hoping Nova folks may be able to provide guidance on...

If I set up a flavor with hw:mem_page_size=2048, and I'm creating (Cirros)
VMs with size 1024, will the scheduling use the minimum of the number of
huge pages available and the size requested for the VM, or will it base
scheduling only on the number of huge pages?

It seems to be doing the latter, where I had 1945 huge pages free, and
tried to create another VM (1024) and Nova rejected the request with "no
hosts available".

Is this still the same for Mitaka?

Where could I look in the code to see how the scheduling is determined?

If I use mem_page_size=large (what I originally had), should it evenly
assign huge pages from the available NUMA nodes (there are two in my case)?

It looks like it was assigning all VMs to the same NUMA node (0) in this
case. Is the right way to change to 2048, like I did above?

Again, has this changed at all in Mitaka?

Lastly, I had a case where there was not enough huge pages, so the create
failed and the VM was in ERROR state. It had created and bound a neutron
port.  I then deleted the VM. The VM disappeared from the list of VMs, but
the Neutron port was still there. I don't see anything in the neutron log
to request deleting the port.  Shouldn't the port have been unbound/deleted?

Any thoughts on how to figure out why not?


Thanks in advance!

PCM
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Social at the summit

[Paul Michali]

+1

On Mon, Apr 25, 2016, 5:32 PM Armando M.  wrote:

> On 25 April 2016 at 10:01, Ihar Hrachyshka  wrote:
>
>> WAT???
>>
>> It was never supposed to be core only. Everyone is welcome!
>>
>
> In fact this should be cross-posted the other openstack ML too.
>
>
>> Sent from my iPhone
>>
>> > On 25 Apr 2016, at 11:56, Edgar Magana 
>> wrote:
>> >
>> > Would you extend it to ex-cores?
>> >
>> > Edgar
>> >
>> >
>> >
>> >
>> >> On 4/25/16, 10:55 AM, "Kyle Mestery"  wrote:
>> >>
>> >> Ihar, Henry and I were talking and we thought Thursday night makes
>> sense for a Neutron social in Austin. If others agree, reply on this thread
>> and we'll find a place.
>> >>
>> >> Thanks!
>> >> Kyle
>> >>
>> >>
>> __
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] VPNaaS tox api failure

[Paul Michali]

Are you running the test locally?

IIRC, the tempest based API tests for VPN have been (are being) moved to
the VPN repo, but I don't know if a job was ever created for this so that
the tests actually run. You may want to check with the author who moved the
tests (madhusudan-kandadai) under commit 3cae934a, to see if the CI job was
ever created. It is possible that it was never completed.

Likewise, I don't know if support was added to tox.ini for the api test.
I've never run the test, granted I've been away from VPN for a while and
only involved intermittently since Liberty, so I may be a bit out of touch.

Regards,

PCM


On Thu, Apr 7, 2016 at 2:23 PM bharath  wrote:

>
> Hi,
>
> While running "tox -e api" i hitting " tempest.lib.NotFound" Error.
>
> Is it a known issue ? failures expected?
>
> https://review.openstack.org/#/c/291949/1  ==> In this commit messages it
> says it kindof expected failure.
>
> Can someone provide some clarification on this
>
> Thanks,
> bharath
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Ihar as *-aas core reviewer

[Paul Michali]

+1 Great addition!



On Thu, Mar 10, 2016 at 12:14 PM Doug Wiegley 
wrote:

> The cleanup was my fault. I had removed folks that were added initially
> just for the initial *aas split. Welcome back.  :-)
>
> Thanks,
> doug
>
> > On Mar 10, 2016, at 2:33 AM, Ihar Hrachyshka 
> wrote:
> >
> > Sean M. Collins  wrote:
> >
> >> I probably speak for all FwaaS cores when I say - "Welcome!”
> >
> > …back! Thanks.
> >
> >>
> >> Frankly I had just assumed he had core privileges for our repo anyway
> >> via an inherited ACL.
> >
> > Full disclosure: I had all -*aas core privileges before [not thru
> inherited ACLs though]. It’s just that lately I lost some of them, probably
> due to some cleanup.
> >
> > To all: it’s not my plan to abuse the powers for anything that is not
> required for general success of the current stadium. If I ever decide to
> get more involved into a specific *-aas subproject strategically, I will
> get back to the corresponding *-aas team for additional approval before I
> use the powers for anything beyond those immediate goals set by PTL.
> >
> > Ihar
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Announcing a simple new tool: git-restack

[Paul Michali]

Sounds interesting... the link
https://docs.openstack.org/infra/git-restack/ referenced
as the home page in PyPI is a broken link.

Regards,

PCM


On Tue, Feb 2, 2016 at 12:54 PM James E. Blair  wrote:

> Hi,
>
> I'm pleased to announce a new and very simple tool to help with managing
> large patch series with our Gerrit workflow.
>
> In our workflow we often find it necessary to create a series of
> dependent changes in order to make a larger change in manageable chunks,
> or because we have a series of related changes.  Because these are part
> of larger efforts, it often seems like they are even more likely to have
> to go through many revisions before they are finally merged.  Each step
> along the way reviewers look at the patches in Gerrit and leave
> comments.  As a reviewer, I rely heavily on looking at the difference
> between patchsets to see how the series evolves over time.
>
> Occasionally we also find it necessary to re-order the patch series, or
> to include or exclude a particular patch from the series.  Of course the
> interactive git rebase command makes this easy -- but in order to use
> it, you need to supply a base upon which to "rebase".  A simple choice
> would be to rebase the series on master, however, that creates
> difficulties for reviewers if master has moved on since the series was
> begun.  It is very difficult to see any actual intended changes between
> different patch sets when they have different bases which include
> unrelated changes.
>
> The best thing to do to make it easy for reviewers (and yourself as you
> try to follow your own changes) is to keep the same "base" for the
> entire patch series even as you "rebase" it.  If you know how long your
> patch series is, you can simply run "git rebase -i HEAD~N" where N is
> the patch series depth.  But if you're like me and have trouble with
> numbers other than 0 and 1, then you'll like this new command.
>
> The git-restack command is very simple -- it looks for the most recent
> commit that is both in your current branch history and in the branch it
> was based on.  It uses that as the base for an interactive rebase
> command.  This means that any time you are editing a patch series, you
> can simply run:
>
>   git restack
>
> and you will be placed in an interactive rebase session with all of the
> commits in that patch series staged.  Git-restack is somewhat
> branch-aware as well -- it will read a .gitreview file to find the
> remote branch to compare against.  If your stack was based on a
> different branch, simply run:
>
>   git restack 
>
> and it will use that branch for comparison instead.
>
> Git-restack is on pypi so you can install it with:
>
>   pip install git-restack
>
> The source code is based heavily on git-review and is in Gerrit under
> openstack-infra/git-restack.
>
> https://pypi.python.org/pypi/git-restack/1.0.0
> https://git.openstack.org/cgit/openstack-infra/git-restack
>
> I hope you find this useful,
>
> Jim
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] Gate is broken

[Paul Michali]

Thanks for hopping on it quickly!



On Fri, Jan 29, 2016 at 10:41 AM Michael Krotscheck 
wrote:

> Hey everyone!
>
> Since the summit submission deadline is this weekend, we (the infra team)
> have decided that it's an excellent time to break all of our slaves, so you
> can go and submit your talks for Austin!
>
> That certainly sounds better than "We misconfigured pip.conf and now none
> of our nodes know how to pip install.". A fix has been merged, however it
> will take some time to propagate. We will keep you updated on this thread
> and on IRC.
>
> Michael
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][neutron-lib] Proposal for callback mechanism migration

[Paul Michali]

Reposted: as I had a typo in subject line that affected filtering of
message...

I wanted to float two ideas related to the neutron callback mechanism that
is being moved to neutron-lib.

1) API
The current API uses kwargs as a way for the notifier to pass information
to the subscribers listening (callbacks). One issue with this, is that the
actual keyword arguments used, could clash with the positional argument
names.

To address this, I'm proposing that we do the same as is done in the
Requests package to pass a payload to the get() method, where a 'params'
positional argument is used to hold a dict with the arguments to be passed
to the callback.

I've pushed a commit to neutron-lib for review
https://review.openstack.org/265997. Please provide your comments on that
as a proposed solution.

2) Migrating callbacks in neutron to use neutron-lib
I was thinking that the following plan (A) could work, as a way to migrate
to using the callback mechanism in neutron-lib:

   1. In neutron, where callback notifications are performed, add a
   duplicate notification to the neutron-lib callback notification.
   2. In each client repo, change the subscription to subscribe to the
   neutron-lib version of the resource/event tuple. At this time, the clients
   could be altered to use the new 'params' positional argument
   3. Once all the client repos have been updated, remove the old
   notification calls from neutron, the callback code, and callback UTs.

An alternative proposal (B), *may* be to:

   1. Change the notification wrapper method in registry.py to call both
   the existing callback notify() and the one in neutron_lib. For the latter,
   the kwargs would need to be stored in the params dict.
   2. This and next step are the same as in proposal (A).

I think plan A gives more flexibility in converting kwargs into a param
dict, at the expense of more of a change impact (32 places/9 files).

Looking forward to community feedback on this...

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutorn][neutron-lib] Proposal for callback mechanism migration

[Paul Michali]

I wanted to float two ideas related to the neutron callback mechanism that
is being moved to neutron-lib.

1) API
The current API uses kwargs as a way for the notifier to pass information
to the subscribers listening (callbacks). One issue with this, is that the
actual keyword arguments used, could clash with the positional argument
names.

To address this, I'm proposing that we do the same as is done in the
Requests package to pass a payload to the get() method, where a 'params'
positional argument is used to hold a dict with the arguments to be passed
to the callback.

I've pushed a commit to neutron-lib for review
https://review.openstack.org/265997. Please provide your comments on that
as a proposed solution.

2) Migrating callbacks in neutron to use neutron-lib
I was thinking that the following plan (A) could work, as a way to migrate
to using the callback mechanism in neutron-lib:

   1. In neutron, where callback notifications are performed, add a
   duplicate notification to the neutron-lib callback notification.
   2. In each client repo, change the subscription to subscribe to the
   neutron-lib version of the resource/event tuple. At this time, the clients
   could be altered to use the new 'params' positional argument
   3. Once all the client repos have been updated, remove the old
   notification calls from neutron, the callback code, and callback UTs.

An alternative proposal (B), *may* be to:

   1. Change the notification wrapper method in registry.py to call both
   the existing callback notify() and the one in neutron_lib. For the latter,
   the kwargs would need to be stored in the params dict.
   2. This and next step are the same as in proposal (A).

I think plan A gives more flexibility in converting kwargs into a param
dict, at the expense of more of a change impact (32 places/9 files).

Looking forward to community feedback on this...

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] FYI: Updated wiki on creating functional jobs

[Paul Michali]

I revised the wiki page that I created previously on how to add functional
jobs to the gate. I added more details, indicated about getting liaison
review, and added some miscellaneous information on templates, skipping
tests, and making tests release based.

Ref: https://wiki.openstack.org/wiki/Neutron/FunctionalGateSetup
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Gate failure with grenade

[Paul Michali]

Thanks Sean!

On Mon, Dec 14, 2015 at 12:58 PM Armando M.  wrote:

> On 14 December 2015 at 09:51, Sean Dague  wrote:
>
>> On 12/14/2015 12:32 PM, Armando M. wrote:
>> > Hi folks,
>> >
>> > Something snuck in past the gate last night [1]. Please stop rechecking
>> > and pushing in the merge queue until the matter is resolved.
>> >
>> > I will follow up with details, if someone knows more, please find me on
>> IRC.
>> >
>> > Thanks,
>> > Armando
>> >
>> > [1]
>> >
>> http://logs.openstack.org/00/254900/4/gate/gate-grenade-dsvm-neutron/a9216c9/logs/grenade.sh.txt.gz#_2015-12-14_12_24_12_561
>>
>> https://review.openstack.org/#/c/257303/ is the fix, it's top of gate
>> right now. Apparently it wasn't noticed that those were deprecated
>> during the liberty cycle, and fixed accordingly.
>>
>
> Thanks Sean!
>
> Cheers,
> Armando
>
>
>>
>> -Sean
>>
>> --
>> Sean Dague
>> http://dague.net
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Announcing the OpenStack Health Dashboard

[Paul Michali]

Sweet!


On Fri, Dec 4, 2015 at 3:40 PM Matthew Treinish 
wrote:

> Hi Everyone,
>
> As some people may have seen already we've been working on creating a test
> results dashboard up and running to visualize the state of the tests
> running in
> the gate. You can get to the dashboard here:
>
> http://status.openstack.org/openstack-health/#/
>
> It's still early for this project (we only started on it back in Sept.),
> so not
> everything is really polished yet and there are still a couple of issues
> and
> limitations.
>
> The biggest current limitation is based on the data store. We're using
> subunit2sql as the backend for all the data and right now we only are
> collecting
> results for tempest and grenade runs in the gate and periodic queues. This
> is
> configurable, as any job that emits a subunit stream can use the same
> mechanism,
> and it is something we will likely change in the future.
>
> We also don't have any results for runs that fail before tempest starts
> running,
> since we need a subunit stream to populate the DB with results. However,
> we have
> several proposed paths to fix this, so it's just a matter of time. But for
> right
> now if a job fails before tests start running it isn't counted on the
> dashboard.
>
> The code for everything lives here:
>
> http://git.openstack.org/cgit/openstack/openstack-health/
>
> If you find an issue feel free to file a bug here:
>
> https://bugs.launchpad.net/openstack-health
>
> We're eager to see this grow to enable the dashboard to suit the needs of
> anyone
> looking at the gate results.
>
> We're tracking work items that need to be done here:
>
> https://etherpad.openstack.org/p/openstack-health-tracking
>
> Please feel free to contribute if you have an idea on how to improve the
> dashboard, or want to take on one of the existing TODO items. The only way
> we'll
> be able to grow this into something that fits everyone's needs is if more
> people
> get involved in improving it.
>
> Thanks,
>
> Matthew Treinish
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][neutron-*] Notice! pylint breakage

[Paul Michali]

Some additional info...

astroid upstream folks are going to try to push for pylint 1.4.5 that pins
to astroid 1.3.8. If that happens, we could just pin pylint at 1.4.5. Ref:
https://bitbucket.org/logilab/astroid/issues/275/140-and-141-fail-to-work-with-pylint-144

It sounds like we don't need to pin versions for Juno. My attempt seems to
be failing tests (badly) https://review.openstack.org/#/c/251865.

The neutron kilo patch seems ready to go, once the infra commit is
upstreamed: https://review.openstack.org/#/c/251827/

Infra commits are: https://review.openstack.org/#/c/251599/ (juno) and
https://review.openstack.org/#/c/251600/ (kilo). Need to (at least) get
kilo upstreamed.

For master, LB and VPN repos are broken. I can see these options

   1. implement pep8 constraints (takes time)
   2. ignore pylint errors/warnings until do #1  (less coverage)
   3. disable pylint until do #1 (no coverage)
   4. Fix the pylint 1.5.0 errors (will be an issue when go to pylint 1.4.4
   as part of #1)

I'm thinking #2 is the least intrusive, and will try that for VPN repo.

BTW: FW repo does not do 'pylint' as part of pep8 (essentially #3 option)
currently, so they don't see this carnage.


Regards,

Paul Michali (pc_m)

On Tue, Dec 1, 2015 at 6:44 AM Paul Michali <p...@michali.net> wrote:

> I found a problem yesterday running pep8 locally in neutron-lbaas. After
> discussing with LBaaS team, we identified that there is a problem with
> pylint.  The same issues were seen, when hecking in neutron and
> neutron-vpnaas repos (need to check neutron-fwaas). There are two issues
> seen.
>
> First, the new version of pylint (1.5.0) is finding additional
> warnings/errors. This will require updates to code, to be compliant with
> the new pylint (assuming we move up to that version). I did see one case
> where the changes needed for pylint 1.5.0 are backward incompatible with
> pylint 1.4.4, which raises another concern (how to migrate to newer pylint).
>
> Second, pylint uses the astroid package, which was recently update from
> 1.3.8 to 1.4.1. When used with pylint 1.4.4 (the currently used version in
> neutron), we get all sorts of false positive errors. For example, use of
> each imported module shows a "undefined variable" error.
>
> In neutron-vpnaas, this issue is worse, as pylint iand astroid are not
> pinned to any version, so it can update at unexpected times.
>
> After talking to infra, here are the proposed solutions...
>
> Neutron - In pretty good shape
>
> The pep8-constraints job currently works, as global requirements in pylint
> to 1.4.4 and upper constraints pin astroid to 1.3.8 - both work together
> well.  Locally, one needs to use pep8-constraints and not pep8, as the
> latter will pull in the latest astroid (1.4.1) and cause havoc.
>
> Infra is pushing up two commits to global requirements to pin astroid to
> 1.3.8 for kilo (251600) and juno (251599). We need to pin astroid to 1.3.8
> in test-requirements.txt for those branches. I'll start on that in a few
> minutes.
>
> LBaaS/VPNaaS/FWaas? - Needs constraints
>
> For pep8 jobs, these repos need to use the new pep8-constraints style job
> and tox.ini should also use the same target, instead of pep8. Like neutron,
> kilo and juno branches need to pin astroid to 1.3.8.
>
> neutron-vpnaas should also pin pylint to 1.4.4 in test-requirements.txt,
> to prevent it from floating to 1.5.0.
>
> All repos will need a plan for updating code to conform to pylint 1.5.0,
> if/when we upgrade to this.
>
> Note: I have not looked at neutron-fwaas, so we need to confirm the above
> issue is present, but it likely is (even if there is not a current pylint
> failure).
>
> One concern I have, that infra hasn't figured out how it can be addressed,
> is how we'll update to pylint 1.5.0. If were are using pep8-constraints,
> the constraints file is in a different repo. Updating, to say, pylint 1.5.0
> and astroid 1.4.1, would cause breakage until both the neutron* repos and
> requirements repo are updated.  This is complicated with backward
> incompatible changes needed.
>
> Thanks to blogan, ZZelle, fungi, anteaya, lifeless, ajmiller and others
> for helping investigate and come up with the approach on this issue!
>
> Regards,
>
> Paul Michali (pc_m)
>
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][neutron-*] Notice! pylint breakage

[Paul Michali]

Infra did not want to update requirements for master.

I found a solution for VPN on master, namely pinning pylint and astroid in
tox.ini (we already have a dependency for pylint in tox.ini).

I'm thinking that for VPN on Kilo, we could cherry pick that change, rather
than employ https://review.openstack.org/#/c/251874/, which needs the fix
to requirements project.

Thoughts?

PCM

On Tue, Dec 1, 2015 at 10:29 AM Gary Kotton <gkot...@vmware.com> wrote:

> Should we not be updating this in the requirements project?
>
> From: Paul Michali <p...@michali.net>
> Reply-To: OpenStack List <openstack-dev@lists.openstack.org>
> Date: Tuesday, December 1, 2015 at 4:50 PM
> To: OpenStack List <openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [neutron][neutron-*] Notice! pylint breakage
>
> Some additional info...
>
> astroid upstream folks are going to try to push for pylint 1.4.5 that pins
> to astroid 1.3.8. If that happens, we could just pin pylint at 1.4.5. Ref:
> https://bitbucket.org/logilab/astroid/issues/275/140-and-141-fail-to-work-with-pylint-144
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__bitbucket.org_logilab_astroid_issues_275_140-2Dand-2D141-2Dfail-2Dto-2Dwork-2Dwith-2Dpylint-2D144=BQMFaQ=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs=VlZxHpZBmzzkWT5jqz9JYBk8YTeq9N3-diTlNj4GyNc=gl1v9FdSQ7rYUO4HlbGDJ_28lT07a9Thah_ecaHzIbw=jvBDSbNi3dDG_83LBVa3sOpfq49Rk0gnSAzzsxGLD2U=>
>
> It sounds like we don't need to pin versions for Juno. My attempt seems to
> be failing tests (badly) https://review.openstack.org/#/c/251865.
>
> The neutron kilo patch seems ready to go, once the infra commit is
> upstreamed: https://review.openstack.org/#/c/251827/
>
> Infra commits are: https://review.openstack.org/#/c/251599/ (juno) and
> https://review.openstack.org/#/c/251600/ (kilo). Need to (at least) get
> kilo upstreamed.
>
> For master, LB and VPN repos are broken. I can see these options
>
>1. implement pep8 constraints (takes time)
>2. ignore pylint errors/warnings until do #1  (less coverage)
>3. disable pylint until do #1 (no coverage)
>4. Fix the pylint 1.5.0 errors (will be an issue when go to pylint
>1.4.4 as part of #1)
>
> I'm thinking #2 is the least intrusive, and will try that for VPN repo.
>
> BTW: FW repo does not do 'pylint' as part of pep8 (essentially #3 option)
> currently, so they don't see this carnage.
>
>
> Regards,
>
> Paul Michali (pc_m)
>
> On Tue, Dec 1, 2015 at 6:44 AM Paul Michali <p...@michali.net> wrote:
>
>> I found a problem yesterday running pep8 locally in neutron-lbaas. After
>> discussing with LBaaS team, we identified that there is a problem with
>> pylint.  The same issues were seen, when hecking in neutron and
>> neutron-vpnaas repos (need to check neutron-fwaas). There are two issues
>> seen.
>>
>> First, the new version of pylint (1.5.0) is finding additional
>> warnings/errors. This will require updates to code, to be compliant with
>> the new pylint (assuming we move up to that version). I did see one case
>> where the changes needed for pylint 1.5.0 are backward incompatible with
>> pylint 1.4.4, which raises another concern (how to migrate to newer pylint).
>>
>> Second, pylint uses the astroid package, which was recently update from
>> 1.3.8 to 1.4.1. When used with pylint 1.4.4 (the currently used version in
>> neutron), we get all sorts of false positive errors. For example, use of
>> each imported module shows a "undefined variable" error.
>>
>> In neutron-vpnaas, this issue is worse, as pylint iand astroid are not
>> pinned to any version, so it can update at unexpected times.
>>
>> After talking to infra, here are the proposed solutions...
>>
>> Neutron - In pretty good shape
>>
>> The pep8-constraints job currently works, as global requirements in
>> pylint to 1.4.4 and upper constraints pin astroid to 1.3.8 - both work
>> together well.  Locally, one needs to use pep8-constraints and not pep8, as
>> the latter will pull in the latest astroid (1.4.1) and cause havoc.
>>
>> Infra is pushing up two commits to global requirements to pin astroid to
>> 1.3.8 for kilo (251600) and juno (251599). We need to pin astroid to 1.3.8
>> in test-requirements.txt for those branches. I'll start on that in a few
>> minutes.
>>
>> LBaaS/VPNaaS/FWaas? - Needs constraints
>>
>> For pep8 jobs, these repos need to use the new pep8-constraints style job
>> and tox.ini should also use the same target, instead of pep8. Like neutron,
>> kilo and juno branches need to pin astroid to 1.3.8.
>>
>> neutron-vpnaas should also pin py

Re: [openstack-dev] [OpenStack-Infra] IRC Bot issues

[Paul Michali]

Check out https://freenode.net/irc_servers.shtml which lists the servers. I
was using irc.freenode.net. Switched to weber.freenode.net and able to
connect.

(now everyone will hop on that one and I'll have to pick another :)



On Mon, Nov 30, 2015 at 2:46 PM Clark, Jay  wrote:

> Can't connect either. Dead in the water
>
> Regards,
> Jay Clark
> Sr. OpenStack Deployment Engineer
> E: jason.t.cl...@hpe.com
> H: 919.341.4670
> M: 919.345.1127
> IRC (freenode): jasondotstar
>
> 
> From: lichen.hangzhou [lichen.hangz...@gmail.com]
> Sent: Monday, November 30, 2015 9:17 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Cc: openstack-in...@lists.openstack.org
> Subject: Re: [openstack-dev] [OpenStack-Infra] IRC Bot issues
>
> Can't connect +1 and web client do not work  :(
>
> -chen
>
> At 2015-11-30 22:08:12, "Hinds, Luke (Nokia - GB/Bristol)" <
> luke.hi...@nokia.com> wrote:
> Me too.  It is possible to get on using the web client though;
> https://webchat.freenode.net/ .
>
> On Mon, 2015-11-30 at 14:00 +, EXT Dugger, Donald D wrote:
> I can’t even connect to the IRC server at all, can others get it?
>
> --
> Don Dugger
> "Censeo Toto nos in Kansa esse decisse." - D. Gale
> Ph: 303/443-3786
>
> From: Joshua Hesketh [mailto:joshua.hesk...@gmail.com joshua.hesk...@gmail.com>]
> Sent: Monday, November 30, 2015 2:50 AM
> To: OpenStack Development Mailing List (not for usage questions) <
> openstack-dev@lists.openstack.org>;
> openstack-infra >
> Subject: [openstack-dev] IRC Bot issues
>
> Hi all,
> Freenode are currently experiencing a severe DDoS attack that are having
> an effect on our bots. As such the meetbot, irc logging and gerrit watcher
> are interminably available.
>
> We expect the bots to resume their normal function once Freenode has
> recovered. For now, meetings may have to be postponed or minuted by hand.
> Cheers,
> Josh
>
> ___
> OpenStack-Infra mailing list
> openstack-in...@lists.openstack.org openstack-in...@lists.openstack.org>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][vpnaas] "SKIPPED: Neutron support is required" while running tox

[Paul Michali]

There is an effort to move the VPN API tests from neutron to neutron-vpnaas
repo under https://review.openstack.org/#/c/211381.  You may want to help
the submitter getting those tests running as an alternative to trying to
get this working (unless there is some need to run this specific test -
older release or something).

Regards,

Paul Michali (pc_m)


On Mon, Nov 30, 2015 at 4:33 AM Ihar Hrachyshka <ihrac...@redhat.com> wrote:

> bharath <bhar...@brocade.com> wrote:
>
> > Hi,
> >
> > when running tox "sudo -u stack -H tox -e api
> > neutron.tests.api.test_vpnaas_extensions"
> > test cases are failing with Error " setUpClass
> > (neutron.tests.api.test_vpnaas_extensions.VPNaaSTestJSON) ... SKIPPED:
> > Neutron support is required"
> >
> > I even tried setting tempest_config_dir as below, still i am hitting the
> > issue.
> >
> > [testenv:api]
> > basepython = python2.7
> > passenv = {[testenv]passenv} TEMPEST_CONFIG_DIR
> > setenv = {[testenv]setenv}
> >  OS_TEST_PATH=./neutron/tests/api
> >
> TEMPEST_CONFIG_DIR={env:TEMPEST_CONFIG_DIR:/opt/stack/tempest/etc}
> >  OS_TEST_API_WITH_REST=1
> >
> >
> > Can someone help me out
> >
>
> Does your /opt/stack/tempest/etc have tempest configured to run tests for
> neutron?
>
> Ihar
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Horizon][Neutron] dashboard repository for neutron subprojects

[Paul Michali]

I like the idea of doing a Horizon plugin, similar to a devstack plugin...
So choice (c) is my preference, given my current understanding. May be good
to try it on one subproject and see how it works out. Would give a concrete
example to discuss.

Regards,

Paul Michali (pc_m)

On Wed, Nov 25, 2015 at 2:13 AM Fawad Khaliq <fa...@plumgrid.com> wrote:

> On Wed, Nov 25, 2015 at 12:06 PM, Armando M. <arma...@gmail.com> wrote:
>
>>
>>
>> On 24 November 2015 at 21:46, Akihiro Motoki <amot...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Neutron has now various subprojects and some of them would like to
>>> implement Horizon supports. Most of them are additional features.
>>> I would like to start the discussion where we should have horizon
>>> support.
>>>
>>> [Background]
>>> Horizon team introduced a plugin mechanism and we can add horizon panels
>>> from external repositories. Horizon team is recommending external repos
>>> for
>>> additional services for faster iteration and features.
>>> We have various horizon related repositories now [1].
>>>
>>> In Neutron related world, we have neutron-lbaas-dashboard and
>>> horizon-cisco-ui repos.
>>>
>>> [Possible options]
>>> There are several possible options for neutron sub-projects.
>>> My current vote is (b), and the next is (a). It looks a good balance to
>>> me.
>>> I would like to gather broader opinions,
>>>
>>> (a) horizon in-tree repo
>>> - [+] It was a legacy approach and there is no initial effort to setup a
>>> repo.
>>> - [+] Easy to share code conventions.
>>> - [-] it does not scale. Horizon team can be a bottleneck.
>>>
>>> (b) a single dashboard repo for all neutron sub-projects
>>> - [+] No need to set up a repo by each sub-project
>>> - [+] Easier to share the code convention. Can get horizon reviewers.
>>> - [-] who will be a core reviewer of this repo?
>>>
>>> (c) neutron sub-project repo
>>>
>>
>> All circumstances considered, I think c) is the only viable one.
>>
> +1
>
>>
>>
>>> - [+] Each sub-project can develop a dashboard fast.
>>> - [-] It is doable, but the directory tree can be complicated.
>>>
>>
>> why? do you envision something else other than /horizon directory in the
>> tree?
>>
>>
>>> - [-] Lead to too many repos and the horizon team/liaison cannot cover
>>> all.
>>>
>>
>> If that's true for horizon, shouldn't the same be true for the neutron
>> team :)? IMO, the level of feedback/oversight provided is always going to
>> be constant (you can't clone people) no matter how the efforts are
>> distributed. I'd rather empower the individual projects.
>>
> Agree. +1
>
>>
>>
>>>
>>> (d) a separate repo per neutron sub-project
>>> Similar to (c)
>>> - [+] A dedicate repo for dashboard simplifies the directory tree.
>>> - [-] Need to setup a separate repo.
>>> - [-] Lead to too many repos and the horizon team/liaison cannot cover
>>> all.
>>>
>>
>>> Note that this mail is not intended to move the current neutron
>>> support in horizon
>>> to outside of horizon tree. I would like to discuss Horizon support of
>>> additional features.
>>>
>>> Akihiro
>>>
>>> [1] http://docs.openstack.org/developer/horizon/plugins.html
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] VPNaaS project status

[Paul Michali]

Thanks folks.

I have gone through all of the "open" VPN bugs and commented/updated them.
The Wiki was updated with references to some of the LP bugs of particular
interest.

Regards,

PCM (pc_m)

On Fri, Nov 13, 2015 at 3:42 AM Elena Ezhova <eezh...@mirantis.com> wrote:

> Thanks for everything you've done for VPNaaS, Paul! Your help and guidance
> was (and is) always very helpful.
>
> On Thu, Nov 12, 2015 at 5:56 PM, Paul Michali <p...@michali.net> wrote:
>
>> Neutron community,
>>
>> During the past several releases, while leading the VPNaaS project, I've
>> seen many great enhancements and features added to the VPNaaS project by
>> the community, including support for StrongSwan, Libreswan, completion of
>> the project split out, functional and rally tests, endpoint groups,
>> multiple local subnets, vendor drivers, etc.
>>
>> There is still work needed (certificate support the most important,
>> followed by documentation and scale testing), but there is a solid (in my
>> bias and subjective opinion :) foundation in place for people to play with
>> this capability.
>>
>> As I mentioned to Armando at the summit, it's time for me to move on to
>> other areas of Neutron, and as such, I'm stepping down as VPNaaS chair and
>> wrapping up work on the project over the next few weeks. I'll still try to
>> review VPNaaS commits as much as possible, and be available to advise in
>> this area.
>>
>> Towards that end, I've updated the VPNaaS wiki page (
>> https://wiki.openstack.org/wiki/Meetings/VPNaaS) to list what I think
>> are outstanding work that can be done in this area, from important to wish
>> items.  Meetings have transitioned to on-demand, and future meetings can
>> either be done as an on-demand topic in the Neutron IRC meeting, or as an
>> on-demand special meeting.
>>
>> I'll go through the VPNaaS bugs in Launchpad and comment on them, as to
>> my opinion of importance, priority, relevance, etc.
>>
>> Regards,
>>
>> PCM (pc_m)
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] VPNaaS project status

[Paul Michali]

Neutron community,

During the past several releases, while leading the VPNaaS project, I've
seen many great enhancements and features added to the VPNaaS project by
the community, including support for StrongSwan, Libreswan, completion of
the project split out, functional and rally tests, endpoint groups,
multiple local subnets, vendor drivers, etc.

There is still work needed (certificate support the most important,
followed by documentation and scale testing), but there is a solid (in my
bias and subjective opinion :) foundation in place for people to play with
this capability.

As I mentioned to Armando at the summit, it's time for me to move on to
other areas of Neutron, and as such, I'm stepping down as VPNaaS chair and
wrapping up work on the project over the next few weeks. I'll still try to
review VPNaaS commits as much as possible, and be available to advise in
this area.

Towards that end, I've updated the VPNaaS wiki page (
https://wiki.openstack.org/wiki/Meetings/VPNaaS) to list what I think are
outstanding work that can be done in this area, from important to wish
items.  Meetings have transitioned to on-demand, and future meetings can
either be done as an on-demand topic in the Neutron IRC meeting, or as an
on-demand special meeting.

I'll go through the VPNaaS bugs in Launchpad and comment on them, as to my
opinion of importance, priority, relevance, etc.

Regards,

PCM (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Stepping Down from Neutron Core Responsibilities

[Paul Michali]

Appreciate all the work Edgar!

Regards,

PCM

On Thu, Nov 5, 2015 at 11:15 AM Miguel Lavalle  wrote:

> Hey Paisano,
>
> Thanks for your great contributions.
>
> Un abrazo
>
> On Wed, Nov 4, 2015 at 6:28 PM, Edgar Magana 
> wrote:
>
>> Dear Colleagues,
>>
>> I have been part of this community from the very beginning when in Santa
>> Clara, CA back in 2011 a bunch of we crazy people decided to work on this
>> networking project.
>> Neutron has become is a very unique piece of code and it requires an
>> approval team that will always be on the top of everything, this is why I
>> would like to communicate you that I decided to step down as Neutron Core.
>>
>> These are not breaking news for many of you because I shared this thought
>> during the summit in Tokyo and now it is a commitment. I want to let you
>> know that I learnt a lot from you and I hope my comments and reviews never
>> offended you.
>>
>> I will be around of course. I will continue my work on code reviews and
>> coordination on the Networking Guide.
>>
>> Thank you all for your support and good feedback,
>>
>> Edgar
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] HenryG addition to the Neutron Drivers team

[Paul Michali]

Congratulations Henry!


On Wed, Oct 21, 2015 at 5:03 AM Ihar Hrachyshka  wrote:

>
> > On 21 Oct 2015, at 05:14, Armando M.  wrote:
> >
> > Hi folks,
> >
> > Henry has been instrumental in many areas of the projects and his crazy
> working hours makes even Kevin and I bow in awe.
> >
> > Jokes aside, I would like to announce HenryG as a new member of the
> Neutron Drivers team.
> >
> > Having a propension to attendance, and desire to review of RFEs puts you
> on the right foot to join the group, whose members are rotated regularly so
> that everyone is given the opportunity to grow, and no-one burns out.
> >
> > The team [1] meets regularly on Tuesdays [2], and anyone is welcome to
> attend.
> >
> > Please, join me in welcome Henry to the team.
>
> Nice addition. :)
>
> Do we have criteria for neutron-drivers team members documented? Or is it
> a mere ‘regularly attending the meetings, be mindful and apply common
> sense’?
>
> Ihar
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Stumped...need help with neutronclient job failure

[Paul Michali]

Hi,

I created a pair of experimental jobs for python-neutronclient that will
run functional tests on core and advanced services, respectively. In the
python-neutronclient repo, I have a commit [1] that splits the tests into
two directories for core/adv-svcs, enables the VPN devstack plugin for the
advanced services tests, and removes the skip decorator for the VPN tests.

When these two jobs run, the core job pass (as expected). The advanced
services job shows all four advanced services tests (testing REST LIST
requests for IKE policy, IPSec policy, IPSec site-to-site connection, and
VPN service resources) failing, with this T/B:

ft1.1: 
neutronclient.tests.functional.adv-svcs.test_readonly_neutron_vpn.SimpleReadOnlyNeutronVpnClientTest.test_neutron_vpn_*ipsecpolicy_list*_StringException:
Empty attachments:
  pythonlogging:''
  stderr
  stdout

Traceback (most recent call last):
  File "neutronclient/tests/functional/adv-svcs/test_readonly_neutron_vpn.py",
line 37, in test_neutron_vpn_ipsecpolicy_list
ipsecpolicy = self.parser.listing(self.neutron('vpn-ipsecpolicy-list'))
  File "neutronclient/tests/functional/base.py", line 78, in neutron
**kwargs)
  File 
"/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
line 292, in neutron
'neutron', action, flags, params, fail_ok, merge_stderr)
  File 
"/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
line 361, in cmd_with_auth
self.cli_dir)
  File 
"/opt/stack/new/python-neutronclient/.tox/functional-adv-svcs/local/lib/python2.7/site-packages/tempest_lib/cli/base.py",
line 61, in execute
proc = subprocess.Popen(cmd, stdout=stdout, stderr=stderr)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory


When I look at the other logs on this run [2], I see these things:
- The VPN agent is running (so the DevStack plugin started up VPN)
- screen-q-svc.log shows only two of the four REST GET requests
- Initially there was no testr results, but I modified post test hook
script similar to what Neutron does (so it shows results now)
- No other errors seen, including nothing on the StringException

When I run this locally, all four tests pass, and I see four REST requests
in the screen-q-svc.log.

I tried a hack to enable NEUTRONCLIENT_DEBUG environment variable, but no
additional information was shown.

Does anyone have any thoughts on what may be going wrong here?
Any ideas on how to troubleshoot this issue?

Thanks in advance!

Paul Michali (pc_m)

Refs
[1] https://review.openstack.org/#/c/214587/
[2]
http://logs.openstack.org/87/214587/8/experimental/gate-neutronclient-test-dsvm-functional-adv-svcs/5dfa152/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] PTL Non-Candidacy

[Paul Michali]

You've done (are doing) a great job as PTL Kyle! Many thanks for all your
hard work in leaving the camp-site in better shape than when you got there
:)



On Fri, Sep 11, 2015 at 6:12 PM Eichberger, German <
german.eichber...@hpe.com> wrote:

> I am with Kevin — we will just write you into the ballot!
>
> Kyle, you rock! Thanks for all the support and help — and hit me up if you
> are short on gin :-)
>
> German
>
>
> From: Kevin Benton >
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org >>
> Date: Friday, September 11, 2015 at 2:35 PM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org >>
> Subject: Re: [openstack-dev] [neutron] PTL Non-Candidacy
>
> This has the works "PTL" and "Candidacy" in the subject. I think that's
> enough to make it on the ballot!
>
> On Fri, Sep 11, 2015 at 2:12 PM, Kyle Mestery > wrote:
> I'm writing to let everyone know that I do not plan to run for Neutron PTL
> for a fourth cycle. Being a PTL is a rewarding but difficult job, as Morgan
> recently put it in his non-candidacy email [1]. But it goes further than
> that for me. As Flavio put it in his post about "Being a PTL" [2], it's a
> full time job. In the case of Neutron, it's more than a full time job, it's
> literally an always on job.
>
> I've tried really hard over my three cycles as PTL to build a stronger web
> of trust so the project can grow, and I feel that's been accomplished. We
> have a strong bench of future PTLs and leaders ready to go, I'm excited to
> watch them lead and help them in anyway I can.
>
> As was said by Zane in a recent email [3], while Heat may have pioneered
> the concept of rotating PTL duties with each cycle, I'd like to highly
> encourage Neutron and other projects to do the same. Having a deep bench of
> leaders supporting each other is important for the future of all projects.
>
> See you all in Tokyo!
> Kyle
>
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2015-September/074157.html
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2015-September/073986.html
> [2]
> http://lists.openstack.org/pipermail/openstack-dev/2015-September/074242.html
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> --
> Kevin Benton
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][gate] please stop approving patches until netaddr issue is solved

[Paul Michali]

For all VPN commits that I had previously gave a +2, I added a WIP (not
ready for review).

Regards,


On Mon, Aug 31, 2015 at 11:34 AM Kevin Benton  wrote:

> Thanks for the clarification. I had thought that, after a short period of
> time, changes would always have to go back through the check tests before
> hitting the gate.
>
> On Mon, Aug 31, 2015 at 8:27 AM, Clark Boylan 
> wrote:
>
>> If the check tests ran prior to the netaddr release and got a +1 any
>> approvals will put them straight into the gate. They will only go
>> through check first (and fail there) if they do not already have a +1
>> from Jenkins.
>>
>> When they fail in the gate they reset all the changes behind them. When
>> you have several changes that will fail like this you end up reseting
>> over and over until the queue processes each one at the head of the
>> queue, removes them, and leaves a -2 vote for the failed gate run.
>>
>> On Mon, Aug 31, 2015, at 08:20 AM, Kevin Benton wrote:
>> > Why would they reset the gate? Shouldn't they just fail the check test?
>> >
>> > On Mon, Aug 31, 2015 at 6:18 AM, Ihar Hrachyshka 
>> > wrote:
>> >
>> > > Hi all,
>> > >
>> > > I see neutron folks pushing more neutron patches into the gate. They
>> are
>> > > all doomed to fail until [1] is resolved. So please stop approving
>> patches,
>> > > we only make harm by resetting the gate, with no chance to pass it.
>> > >
>> > > PS: it is the same for all neutron stadium projects, so *aas and
>> > > python-networking-* folks, please also avoid +W until the situation is
>> > > cleared.
>> > >
>> > > [1]: https://bugs.launchpad.net/neutron/+bug/1490380
>> > >
>> > > Ihar
>> > >
>> __
>> > > OpenStack Development Mailing List (not for usage questions)
>> > > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> > >
>> >
>> >
>> >
>> > --
>> > Kevin Benton
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Kevin Benton
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

Thanks for the links and thoughtful comments James!

In the Heat documentation, is the subnet ID being treated as optional (or
is the documentation not correct)? I think it is a required argument in the
REST API. Ref:
http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Neutron::VPNService

It really sounds like we need to support both versions simultaneously, to
avoid some of the complexity that you are mentioning (I didn't know about
the Heat aspects).

We (development) should focus on formulating a plan for supporting both
versions. Hopefully my proposal in https://review.openstack.org/#/c/191944/ is
a good starting point.  I'll respond to comments on that today.

Regards,

Paul Michali (pc_m)

P.S. BTW: No offense taken in comments - email is such an imprecise
communication medium.



On Wed, Aug 26, 2015 at 7:11 PM James Dempsey jam...@catalyst.net.nz
wrote:

 On 26/08/15 23:43, Paul Michali wrote:
  James,
 
  Great stuff! Please see @PCM in-line...
 
  On Tue, Aug 25, 2015 at 6:26 PM James Dempsey jam...@catalyst.net.nz

 SNIP

  1) Horizon compatibility
 
  We run a newer version of horizon than we do neutron.  If Horizon
  version X doesn't work with Neutron version X-1, this is a very big
  problem for us.
 
 
  @PCM Interesting. I always thought that Horizon updates lagged Neutron
  changes, and this wouldn't be a concern.
 

 @JPD
 Our Installed Neutron typically lags Horizon by zero or one release.  My
 concern is how will Horizon version X cope with a point-in-time API
 change?  Worded slightly differently: We rarely update Horizon and
 Neutron at the same time so there would need to be a version(or
 versions) of Horizon that could detect a Neutron upgrade and start using
 the new API.  (I'm fine if there is a Horizon config option to select
 old/new VPN API usage.)

 
 
 
  2) Service interruption
 
  How much of a service interruption would the 'migration path' cause?
 
 
  @PCM The expectation of the proposal is that the migration would occur as
  part of the normal OpenStack upgrade process (new services installed,
  current services stopped, database migration occurs, new services are
  started).
 
  It would have the same impact as what would happen today, if you update
  from one release to another. I'm sure you folks have a much better handle
  on that impact and how to handle it (maintenance windows, scheduled
  updates, etc).
 

 @JPD This seems fine.

 
  We
  all know that IPsec VPNs can be fragile...  How much of a guarantee will
  we have that migration doesn't break a bunch of VPNs all at the same
  time because of some slight difference in the way configurations are
  generated?
 
 
  @PCM I see the risk as extremely low. With the migration, the end result
 is
  really just moving/copying fields from one table to another. The
 underlying
  configuration done to *Swan would be the same.
 
  For example, the subnet ID, which is specified in the VPN service API and
  stored in the vpnservices table, would be stored in a new vpn_endpoints
  table, and the ipsec_site_connections table would reference that entry
  (rather than looking up the subnet in the vpnservices table).
 

 @JPD This makes me feel more comfortable; thanks for explaining.

 
 
  3) Heat compatibility
 
  We don't always run the same version of Heat and Neutron.
 
 
  @PCM I must admit, I've never used Heat, and am woefully ignorant about
 it.
  Can you elaborate on Heat concerns as may be related to VPN API
 differences?
 
  Is Heat being used to setup VPN connections, as part of orchestration?
 

 @JPD
 My concerns are two-fold:

 1) Because Heat makes use of the VPNaaS API, it seems like the same
 situation exists as with Horizon.  Some version or versions of Heat will
 need to be able to make use of both old and new VPNaaS APIs in order to
 cope with a Neutron upgrade.

 2) Because we use Heat resource types like
 OS::Neutron::IPsecSiteConnection [1], we may lose the ability to
 orchestrate VPNs if endpoint groups are not added to Heat at the same time.


 Number 1 seems like a real problem that needs a fix.  Number 2 is a fact
 of life that I am not excited about, but am prepared to deal with.

 Yes, Heat is being used to build VPNs, but I am prepared to make the
 decision on behalf of my users... VPN creation via Heat is probably less
 important than the new VPNaaS features, but it would be really great if
 we could work on the updated heat resource types in parallel.

 [1]

 http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Neutron::IPsecSiteConnection

 
 
 
 
  Is there pain for the customers beyond learning about the new API
  changes
  and capabilities (something that would apply whether there is backward
  compatibility or not)?
 
 
  See points 1,2, and 3 above.
 
 
 
  Another implication of not having backwards compatibility would be that
  end-users would need to immediately switch to using the new API, once
 the
  migration occurs, versus doing so

Re: [openstack-dev] [heat][horizon] Backward-incompatible changes to the Neutron API

[Paul Michali]

Akihiro, can you look at the developer's reference I posted (191944), where
there is the overall API plan and a proposal for handling backward
compatibility.

Thanks!

Paul Michali (pc_m)

On Thu, Aug 27, 2015 at 11:12 AM Akihiro Motoki amot...@gmail.com wrote:

 As Mathias said, Horizon worked (and in many cases works) cross releases.

 Horizon determines supported features based on keystone catalogs,
 extension list from back-end services (like nova, neutron).
 Micro-versioning support may come in future (though it is not supported).

 For backward incompatible API change in VPNaaS, Horizon can determine
 if Neutron (including VPNaaS) provides a way to determines which version
 is available.
 At now, the only way is to expose it through the extension list.

 On the other hand, it is tough to maintain multiple versions of
 implementations.
 It is reasonable to me that Horizon supports two implementation in one or
 two
 release cycle(s) and drop older implementation later.

 Akihiro


 2015-08-27 16:29 GMT+09:00 Matthias Runge mru...@redhat.com:

 On 26/08/15 23:55, James Dempsey wrote:
  Greetings Heat/Horizon Devs,
 
  There is some talk about possibly backward-incompatible changes to the
  Neutron VPNaaS API and I'd like to better understand what that means for
  Heat and Horizon.
 
  It has been proposed to change Neutron VPNService objects such that they
  reference a new resource type called an Endpoint Group instead of
  simply a Subnet.
 
  Does this mean that any version of Heat/Horizon would only be able to
  support either the old or new Neutron API, or is there some way to allow
  a version of Heat/Horizon to support both?
 
 In the past, Horizon worked cross releases.

 The way horizon works is, it looks out for a networking endpoint in
 keystone catalog. We don't really care, if it's nova or neutron
 answering. The rest should be discoverable via API.
 Horizon uses neutronclient rather than directly talking to neutron by
 using its API interface.

 If you make it discoverable, and you'd add that to neutronclient,
 horizon could support both.

 Matthias




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] Need community guidance please...

[Paul Michali]

See @PCM inline...


On Wed, Aug 26, 2015 at 4:44 AM Germy Lure germy.l...@gmail.com wrote:

 Hi,

 Maybe I missed some key points. But why we introduced vpn-endpoint groups
 here?


@PCM For the multiple local subnet capabilities for IPSec, the existing API
would need to be changed, so that we can specify 1+ local subnets, as part
of the VPN connection. This would have been the simplest approach for
updating IPSec connections, however, it makes it a point solution.

Other VPN flavors would also have similar endpoint specifications in
their connection APIs.

The approach that I'm advocating, is to extract out some of the commonality
between VPN flavors such that we can have some reuse.

Essentially, the idea is to break VPN into two parts. One is what is
connected and the other is how the connection is made.

For the former, the idea of an endpoint group is introduced. It provides a
collection of endpoints that can be identified by a type (e.g. subnet,
CIDR, network, vlan, ...) and an ID.

The latter would be VPN flavor specific, having all of the details needed
for that type of VPN connection and would reference the needed endpoint
group(s) by ID.

This separates the what from the how.



 ipsec-site-connection for IPSec VPN only, gre-connection for GRE VPN
 only, and mpls-connection for MPLS VPN only. You see, different
 connections for different vpn types. Indeed, We can't reuse connection API.


@PCM Correct. The how is VPN type specific. But we can have a common API
for the what.




 Piece of the ref document(https://review.openstack.org/#/c/191944/) like
 this:
 allowing subnets (local) and CIDRs (peer) to be used for IPSec, but
 routers, networks, and VLANs to be used for other VPN types (BGP, L2,
 direct connection)

 You see, different epg types for different vpn types. We can't reuse epg.


@PCM We're not reusing the endpoint group itself for different types of
VPN, we're reusing the API for different types of VPN. A common API that
holds a collection of endpoints of a specific type.

You can look at the code out for review, for a feel for the implementation
being worked on:  https://review.openstack.org/#/c/212692/3



 So, how we meet The third goal, is to do this in a manner that the code
 can be reused for other flavors of VPN.?


@PCM The code for the endpoint group API could be used for other VPN types.
Instead of them creating table fields (and the corresponding db logic) for
the endpoints of their connection, they can refer to the ID(s) from the
endpoint groups table, and can add additional validation based on the VPN
type.

FYI, I pushed up version 7 of the dev ref document yesterday.

Regards,

PCM


 Thanks.


 On Tue, Aug 25, 2015 at 1:54 AM, Madhusudhan Kandadai 
 madhusudhan.openst...@gmail.com wrote:

 My two cents..

 On Mon, Aug 24, 2015 at 8:48 AM, Jay Pipes jaypi...@gmail.com wrote:

 Hi Paul, comments inline...

 On 08/24/2015 07:02 AM, Paul Michali wrote:

 Hi,

 I'm working on the multiple local subnet feature for VPN (RFE
 https://bugs.launchpad.net/neutron/+bug/1459423), with a developer
 reference document detailing the proposed process
 (https://review.openstack.org/#/c/191944/). The plan is to do this in
 two steps. The first is to add new APIs and database support for
 endpoint groups (see dev ref for details). The second is to modify the
 IPSec/VPN APIs to make use of the new information (and no longer use
 some older, but equivalent info that is being extended).

 I have a few process/procedural questions for the community...

 Q1) Should I do this all as one huge commit, as two commits (one for
 endpoint groups and one for modification to support multiple local
 subnets), or multiple (chained) commits (e.g. commit for each API for
 the endpoint groups and for each part of the multiple subnet change)?

 My thought (now) is to do this as two commits, with the endpoint groups
 as one, and multiple subnet groups as a second. I started with a commit
 for create API of endpoint (212692), and then did a chained commit for
 delete/show/list (215717), thinking they could be reviewed in pieces,
 but they are not that large and could be easily merged.


 My advice would be 2 commits, as you have split them out.


 I would prefer to have two commits with end-point groups as one and
 modification to support multiple local subnets as another. This will be
 easy to troubleshoot when in need.


 Q2) If the two parts are done separately, should the endpoint group
 portion, which adds a table and API calls, be done as part of the
 existing version (v2) of VPN, instead of introducing a new version at
 that step?


 Is the Neutron VPN API microversioned? If not, then I suppose your only
 option is to modify the existing v2 API. These seem to be additive changes,
 not modifications to existing API calls, in which case they are
 backwards-compatible (just not discoverable via an API microversion).

 I suggest to be done as part of the existing version v2 API . As the api
 tests

Re: [openstack-dev] [Openstack-operators] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

James,

Great stuff! Please see @PCM in-line...



On Tue, Aug 25, 2015 at 6:26 PM James Dempsey jam...@catalyst.net.nz
wrote:

 Oops, sorry about the blank email.  Answers/Questions in-line.

 On 26/08/15 07:46, Paul Michali wrote:
  Previous post only went to dev list. Ensuring both and adding a bit
 more...
 
 
 
  On Tue, Aug 25, 2015 at 8:37 AM Paul Michali p...@michali.net wrote:
 
  Xav,
 
  The discussion is very important, and hence why both Kyle and I have
 been
  posting these questions on the operator (and dev) lists. Unfortunately,
 I
  wasn't subscribed to the operator's list and missed some responses to
  Kyle's message, which were posted only to that list.
 
  As a result, I had an incomplete picture and posted this thread to see
 if
  it was OK to do this without backward compatibility, based on the
  (incorrect) assumption that there was no production use. That is
 corrected
  now, and I'm getting all the messages and thanks to everyone, have
 input on
  messages I missed.
 
  So given that, let's try a reset on the discussion, so that I can better
  understand the issues...
 

 Great!  Thanks very much for expanding the scope.  We really appreciate it.

  Do you feel that not having backward compatibility (but having a
 migration
  path) would seriously affect you or would it be manageable?

 Currently, this feels like it would seriously affect us.  I don't feel
 confident that the following concerns won't cause us big problems.


 As Xav mentioned previously, we have a few major concerns:


@PCM Thanks! It's good to know what things we need to be aware of.




 1) Horizon compatibility

 We run a newer version of horizon than we do neutron.  If Horizon
 version X doesn't work with Neutron version X-1, this is a very big
 problem for us.


@PCM Interesting. I always thought that Horizon updates lagged Neutron
changes, and this wouldn't be a concern.




 2) Service interruption

 How much of a service interruption would the 'migration path' cause?


@PCM The expectation of the proposal is that the migration would occur as
part of the normal OpenStack upgrade process (new services installed,
current services stopped, database migration occurs, new services are
started).

It would have the same impact as what would happen today, if you update
from one release to another. I'm sure you folks have a much better handle
on that impact and how to handle it (maintenance windows, scheduled
updates, etc).


We
 all know that IPsec VPNs can be fragile...  How much of a guarantee will
 we have that migration doesn't break a bunch of VPNs all at the same
 time because of some slight difference in the way configurations are
 generated?


@PCM I see the risk as extremely low. With the migration, the end result is
really just moving/copying fields from one table to another. The underlying
configuration done to *Swan would be the same.

For example, the subnet ID, which is specified in the VPN service API and
stored in the vpnservices table, would be stored in a new vpn_endpoints
table, and the ipsec_site_connections table would reference that entry
(rather than looking up the subnet in the vpnservices table).



 3) Heat compatibility

 We don't always run the same version of Heat and Neutron.


@PCM I must admit, I've never used Heat, and am woefully ignorant about it.
Can you elaborate on Heat concerns as may be related to VPN API differences?

Is Heat being used to setup VPN connections, as part of orchestration?




 
  Is there pain for the customers beyond learning about the new API
 changes
  and capabilities (something that would apply whether there is backward
  compatibility or not)?
 

 See points 1,2, and 3 above.


 
  Another implication of not having backwards compatibility would be that
  end-users would need to immediately switch to using the new API, once the
  migration occurs, versus doing so on their own time frame.  Would this
 be a
  concern for you (customers not having the convenience of delaying their
  switch to the new API)?
 
 
  I was thinking that backward incompatible changes would adversely affect
  people who were using client scripts/apps to configure (a large number
 of)
  IPsec connections, where they'd have to have client scripts/apps
 in-place
  to support the new API.
 

 This is actually less of a concern.  We have found that VPN creation is
 mostly done manually and anyone who is clever enough to make IPsec go is
 clever enough to learn a new API/horizon interface.


@PCM Do you see much reliance on tooling to setup VPN (such that having to
update the tooling would be a concern for end-users), or is this something
that could be managed through process/preparation?




 
  Which is more of a logistics issue, and could be managed, IMHO.
 
 
 
 
  Would there be customers that would fall into that category, or are
  customers manually configuring IPSec connections in that they could just
  use the new API?
 

 Most customers could easily adapt to a new API

Re: [openstack-dev] [heat][horizon] Backward-incompatible changes to the Neutron API

[Paul Michali]

For details on the API/resource change, see the developer's reference
document that is under review: https://review.openstack.org/#/c/191944/

There is a section at the end, where I'm proposing a possible way to
support both versions of the API and provide backward compatibility. Feel
free to comment on the review.

Regards,

Paul Michali (pc_m)


On Wed, Aug 26, 2015 at 6:10 PM James Dempsey jam...@catalyst.net.nz
wrote:

 Greetings Heat/Horizon Devs,

 There is some talk about possibly backward-incompatible changes to the
 Neutron VPNaaS API and I'd like to better understand what that means for
 Heat and Horizon.

 It has been proposed to change Neutron VPNService objects such that they
 reference a new resource type called an Endpoint Group instead of
 simply a Subnet.

 Does this mean that any version of Heat/Horizon would only be able to
 support either the old or new Neutron API, or is there some way to allow
 a version of Heat/Horizon to support both?


 Thanks,
 James

 --
 James Dempsey
 Senior Cloud Engineer
 Catalyst IT Limited
 +64 4 803 2264
 --

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

Previous post only went to dev list. Ensuring both and adding a bit more...



On Tue, Aug 25, 2015 at 8:37 AM Paul Michali p...@michali.net wrote:

 Xav,

 The discussion is very important, and hence why both Kyle and I have been
 posting these questions on the operator (and dev) lists. Unfortunately, I
 wasn't subscribed to the operator's list and missed some responses to
 Kyle's message, which were posted only to that list.

 As a result, I had an incomplete picture and posted this thread to see if
 it was OK to do this without backward compatibility, based on the
 (incorrect) assumption that there was no production use. That is corrected
 now, and I'm getting all the messages and thanks to everyone, have input on
 messages I missed.

 So given that, let's try a reset on the discussion, so that I can better
 understand the issues...

 Do you feel that not having backward compatibility (but having a migration
 path) would seriously affect you or would it be manageable?

 Is there pain for the customers beyond learning about the new API changes
 and capabilities (something that would apply whether there is backward
 compatibility or not)?


Another implication of not having backwards compatibility would be that
end-users would need to immediately switch to using the new API, once the
migration occurs, versus doing so on their own time frame.  Would this be a
concern for you (customers not having the convenience of delaying their
switch to the new API)?


 I was thinking that backward incompatible changes would adversely affect
 people who were using client scripts/apps to configure (a large number of)
 IPsec connections, where they'd have to have client scripts/apps in-place
 to support the new API.


Which is more of a logistics issue, and could be managed, IMHO.




 Would there be customers that would fall into that category, or are
 customers manually configuring IPSec connections in that they could just
 use the new API?


Are there other adverse effects of not having backward compatibility that
 need to be considered?


So far, I'm identifying one effect that is more of a convenience (although
nice one at that), and one effect that can be avoided by planning for the
upgrade.  I'd like to know if I'm missing something more important to
operators.

I'd also like to know if we thing there is a user base large enough (and
how large is large?0 that would warrant going through the complexity and
risk to support both API versions simultaneously?

Regards,

Paul Michali (pc_m)


 Specifically, we're talking about the VPN service create API no longer
 taking a subnet ID (instead an endpoint group is create that contains the
 subnet ID), and the IPSec site-to-site connection create API would no
 longer take a list of peer CIDRs, but instead would take a pair of endpoint
 group IDs (one for the local subnet(s) formally specified by the service
 API, and one for peer CIDRs).





 Regards,

 Paul Michali (pc_m)


 On Mon, Aug 24, 2015 at 5:32 PM Xav Paice xavpa...@gmail.com wrote:

 I'm sure I'm not the only one that finds the vast amount of traffic in
 the dev list to be completely unmanageable to catch the important messages
 - the ops list is much lower traffic, and as an operator I pay a bunch more
 attention to it.  The discussion of deprecating an API is something that
 HAS to be discussed with operators, on the operators list or highlighted
 somehow so that people get attention drawn to the message.

 Let's be clear - I fully appreciate the extra effort that would be
 required in supporting both the new and the old APIs, and also would
 absolutely love to see the new feature.  I do think we need to be able to
 support our customers in the transition, and extra pain for them results in
 lower uptake of the services we provide.

 On 25 August 2015 at 09:27, Xav Paice xavpa...@gmail.com wrote:

 Also:


 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007928.html

 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007891.html


 On 25 August 2015 at 09:09, Kevin Benton blak...@gmail.com wrote:

 It sounds like you might have missed a couple responses:


 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007903.html

 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007910.html


 On Mon, Aug 24, 2015 at 1:53 PM, Paul Michali p...@michali.net wrote:

 Xav,

 In the email, there were no responses of anyone using VPNaaS *in a
 production environment*. Summary from responders:

 Erik M - Tried in Juno with no success. Will retry.
 Edgar M - said no reports from operators about VPNaaS code
 Sam S - Using VPN in VMs and not VPNaaS
 Kevin B - Not used. Use VMs instead
 Sriram S - Indicating not used.

 If I misread the responses, or if someone has not spoken up, right now
 is the time to let us know of your situation and the impact this proposal
 would have on your use of VPNaaS IPSec site-to-site connections.

 The request here

Re: [openstack-dev] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

Xav,

The discussion is very important, and hence why both Kyle and I have been
posting these questions on the operator (and dev) lists. Unfortunately, I
wasn't subscribed to the operator's list and missed some responses to
Kyle's message, which were posted only to that list.

As a result, I had an incomplete picture and posted this thread to see if
it was OK to do this without backward compatibility, based on the
(incorrect) assumption that there was no production use. That is corrected
now, and I'm getting all the messages and thanks to everyone, have input on
messages I missed.

So given that, let's try a reset on the discussion, so that I can better
understand the issues...

Do you feel that not having backward compatibility (but having a migration
path) would seriously affect you or would it be manageable?

Is there pain for the customers beyond learning about the new API changes
and capabilities (something that would apply whether there is backward
compatibility or not)?

I was thinking that backward incompatible changes would adversely affect
people who were using client scripts/apps to configure (a large number of)
IPsec connections, where they'd have to have client scripts/apps in-place
to support the new API.

Would there be customers that would fall into that category, or are
customers manually configuring IPSec connections in that they could just
use the new API?

Are there other adverse effects of not having backward compatibility that
need to be considered?

Specifically, we're talking about the VPN service create API no longer
taking a subnet ID (instead an endpoint group is create that contains the
subnet ID), and the IPSec site-to-site connection create API would no
longer take a list of peer CIDRs, but instead would take a pair of endpoint
group IDs (one for the local subnet(s) formally specified by the service
API, and one for peer CIDRs).


Regards,

Paul Michali (pc_m)


On Mon, Aug 24, 2015 at 5:32 PM Xav Paice xavpa...@gmail.com wrote:

 I'm sure I'm not the only one that finds the vast amount of traffic in the
 dev list to be completely unmanageable to catch the important messages -
 the ops list is much lower traffic, and as an operator I pay a bunch more
 attention to it.  The discussion of deprecating an API is something that
 HAS to be discussed with operators, on the operators list or highlighted
 somehow so that people get attention drawn to the message.

 Let's be clear - I fully appreciate the extra effort that would be
 required in supporting both the new and the old APIs, and also would
 absolutely love to see the new feature.  I do think we need to be able to
 support our customers in the transition, and extra pain for them results in
 lower uptake of the services we provide.

 On 25 August 2015 at 09:27, Xav Paice xavpa...@gmail.com wrote:

 Also:


 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007928.html

 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007891.html


 On 25 August 2015 at 09:09, Kevin Benton blak...@gmail.com wrote:

 It sounds like you might have missed a couple responses:


 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007903.html

 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007910.html


 On Mon, Aug 24, 2015 at 1:53 PM, Paul Michali p...@michali.net wrote:

 Xav,

 In the email, there were no responses of anyone using VPNaaS *in a
 production environment*. Summary from responders:

 Erik M - Tried in Juno with no success. Will retry.
 Edgar M - said no reports from operators about VPNaaS code
 Sam S - Using VPN in VMs and not VPNaaS
 Kevin B - Not used. Use VMs instead
 Sriram S - Indicating not used.

 If I misread the responses, or if someone has not spoken up, right now
 is the time to let us know of your situation and the impact this proposal
 would have on your use of VPNaaS IPSec site-to-site connections.

 The request here, is that if operators are not using this in a
 production deployment where they need backward compatibility, then we'd
 like to avoid having to provide the complexity needed to support backward
 compatibility.

 In the proposal, there are two APIs that would be changed with this
 enhancement. It's detailed in reference [1], and I can elaborate, if 
 needed.

 Please keep in mind, that users/operators using previous versions, can
 upgrade to the new version, and any existing VPNaaS configuration will be
 automatically migrated to the new table structures, so that existing IPSec
 connections would continue to operate with the new release.

 The proposal would not support using the older APIs, once the new APIs
 are available. Client apps/scripts would need to be updated to use the new
 API (neutron client and the Horizon dashboard will be updated as part of
 the overall effort).

 I hope that clarifies the discussion.

 Regards,

 Paul Michali (pc_m)


 On Mon, Aug 24, 2015 at 3:50 PM Xav Paice xavpa...@gmail.com wrote:

 On 25/08

[openstack-dev] [neutron][vpnaas] Need community guidance please...

[Paul Michali]

Hi,

I'm working on the multiple local subnet feature for VPN (RFE
https://bugs.launchpad.net/neutron/+bug/1459423), with a developer
reference document detailing the proposed process (
https://review.openstack.org/#/c/191944/). The plan is to do this in two
steps. The first is to add new APIs and database support for endpoint
groups (see dev ref for details). The second is to modify the IPSec/VPN
APIs to make use of the new information (and no longer use some older, but
equivalent info that is being extended).

I have a few process/procedural questions for the community...

Q1) Should I do this all as one huge commit, as two commits (one for
endpoint groups and one for modification to support multiple local
subnets), or multiple (chained) commits (e.g. commit for each API for the
endpoint groups and for each part of the multiple subnet change)?

My thought (now) is to do this as two commits, with the endpoint groups as
one, and multiple subnet groups as a second. I started with a commit for
create API of endpoint (212692), and then did a chained commit for
delete/show/list (215717), thinking they could be reviewed in pieces, but
they are not that large and could be easily merged.

Q2) If the two parts are done separately, should the endpoint group
portion, which adds a table and API calls, be done as part of the existing
version (v2) of VPN, instead of introducing a new version at that step?

Q3) For the new API additions, do I create a new subclass for the
interface that includes all the existing APIs, introduce a new class that
is used together with the existing class, or do I add this to the existing
API?

Q4) With the final multiple local subnet changes, there will be changes to
the VPN service API (delete subnet_id arg) and IPSec connection API (delete
peer_cidrs arg, and add local_endpoints and peer_endpoints args). Do we
modify the URI so that it calls out v3 (versus v2)? Where do we do that?

I'm unsure of the mechanism of increasing the version.

Thanks in advance for any guidance here on how this should be rolled out...

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] Need community guidance please...

[Paul Michali]

Had a discussion today on Neutron IRC with Salvatore (thanks!) and here is
the thoughts forward going...

1) Will do two commits, one for endpoint groups, which is a new API and
stands independently, and one for multiple local subnets, which will alter
existing APIs. This should prevent any test job breakage, as the API change
is rolled out in the second commit.

2) Will email to operators, indicating that we'll institute a backward
incompatible change, given there is little, possibly none, production use
(from previous e-mail sent out). If that is OK with them, will proceed,
otherwise, we'll look at applying effort to make the change backward
compatible. It's not a trial amount of work, so avoiding it, if not needed.

3) Based on the assumption that we'll not provide backward compatibility,
given the current usage, the endpoint groups API will be added to the
existing extension, as part of v2.0. We can, at a later time, break it out
into a new extension module, if needed.

Jay, there isn't any micro-versioning yet for neutron-vpnaas repo.


Regards,

PCM

On Mon, Aug 24, 2015 at 11:49 AM Jay Pipes jaypi...@gmail.com wrote:

 Hi Paul, comments inline...

 On 08/24/2015 07:02 AM, Paul Michali wrote:
  Hi,
 
  I'm working on the multiple local subnet feature for VPN (RFE
  https://bugs.launchpad.net/neutron/+bug/1459423), with a developer
  reference document detailing the proposed process
  (https://review.openstack.org/#/c/191944/). The plan is to do this in
  two steps. The first is to add new APIs and database support for
  endpoint groups (see dev ref for details). The second is to modify the
  IPSec/VPN APIs to make use of the new information (and no longer use
  some older, but equivalent info that is being extended).
 
  I have a few process/procedural questions for the community...
 
  Q1) Should I do this all as one huge commit, as two commits (one for
  endpoint groups and one for modification to support multiple local
  subnets), or multiple (chained) commits (e.g. commit for each API for
  the endpoint groups and for each part of the multiple subnet change)?
 
  My thought (now) is to do this as two commits, with the endpoint groups
  as one, and multiple subnet groups as a second. I started with a commit
  for create API of endpoint (212692), and then did a chained commit for
  delete/show/list (215717), thinking they could be reviewed in pieces,
  but they are not that large and could be easily merged.

 My advice would be 2 commits, as you have split them out.

  Q2) If the two parts are done separately, should the endpoint group
  portion, which adds a table and API calls, be done as part of the
  existing version (v2) of VPN, instead of introducing a new version at
  that step?

 Is the Neutron VPN API microversioned? If not, then I suppose your only
 option is to modify the existing v2 API. These seem to be additive
 changes, not modifications to existing API calls, in which case they are
 backwards-compatible (just not discoverable via an API microversion).

  Q3) For the new API additions, do I create a new subclass for the
  interface that includes all the existing APIs, introduce a new class
  that is used together with the existing class, or do I add this to the
  existing API?

 Until microversioning is introduced to the Neutron VPN API, it should
 probably be a change to the existing v2 API.

  Q4) With the final multiple local subnet changes, there will be changes
  to the VPN service API (delete subnet_id arg) and IPSec connection API
  (delete peer_cidrs arg, and add local_endpoints and peer_endpoints
  args). Do we modify the URI so that it calls out v3 (versus v2)? Where
  do we do that?

 Hmm, with the backwards-incompatible API changes like the above, your
 only option is to increment the major version number. The alternative
 would be to add support for microversioning as a prerequisite to the
 patch that adds backwards-incompatible changes, and then use a
 microversion to introduce those changes.

 Best,
 -jay

  I'm unsure of the mechanism of increasing the version.
 
  Thanks in advance for any guidance here on how this should be rolled
 out...
 
  Regards,
 
  Paul Michali (pc_m)
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http

[openstack-dev] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

As part of the effort to provide support for multiple local subnets for
VPNaaS IPSec connections, there are three API changes planned [1].

One is to add a new endpoint groups API that will describe what is being
connected. This is the place were local and peer subnets will be specified.
It will allow future VPN flavors to reuse this API for other types of VPN
endpoints.

The second is to modify the IPSec site connection API to make use of this
new endpoint groups API and no longer use the peer CIDRs arguments.

The third is to remove the subnet ID from the VPN service API, and again,
use the new endpoint groups API for this information (and to allow 1).

A previous email send out from Kyle Mestery, asking about the production
usage of VPNaaS, did not indicate that this service was used in production
by operators.

As a result, we'd like to propose making this change as a non-backward
compatible change, which greatly reduces the effort.

The change WILL support migration, so older databases can be migrated to
the new schema and continue to operate, with future changes using the new
API. This gives a smooth upgrade path to the new capability.

The change would NOT support the older API in parallel with the new API, so
users upgrading would need to also update any client scripts/tools to use
the revised/new VPN API.

If this is acceptable to operators, we'd like to go forward with these
plans. Please respond within 7 days of this email, if you have concerns
about the proposal.

Regards,

Paul Michali (IRC pc_m)
VPNaaS Core

Ref: https://review.openstack.org/#/c/191944/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] How do we handle test jobs for neutron client?

[Paul Michali]

Could use some consensus on how to proceed here... Can the cores weigh in
here?

Regards,

PCM


On Thu, Aug 20, 2015 at 10:54 AM Paul Michali p...@michali.net wrote:

 I'm trying to update the DSVM test for neutron client so that it uses the
 new VPN devstack plugin. In this process, however, I want to have a
 solution that will work for other advanced services and for networking-*
 projects. I could use some guidance here please.

 The approach I'm taking (and I need some guidance here), is to create a
 gate-hook for neutron client [1], so that we can apply any needed devstack
 plugin enables (not just for VPN). Then, add a job template that takes
 variable args, and define (experimental) jobs to project-config that use
 the gate-hook [2]. Finally, modify neutron client to make use of the gate
 hook to do the needed testing.

 There are several ways forward, and I'd like to here what the community
 consensus is to handle this.

 The way I was originally heading, was to have a job that replaces the
 original job to run the core test cases, and another job for running the
 VPN test cases (enabling the VPN devstack plugin). Later, jobs could be
 defined for LB and FW, as needed/desired.

 Another possibility, would be to just have two jobs, one that handles
 core tests, and one that handles non-core (advanced services, etc) test
 cases and which enables the needed plugin(s).

 A third, and maybe simple case, is to just have a single job that replaces
 the original job and runs all the tests and enables all the needed devstack
 plugins.

 The obvious difference here is in the granularity of what gets tested for
 a job (and the isolation provided).

 Some questions...

 - Opinions on the approaches suggested?
 - Do we need separate jobs?
 - Do networking-* projects have their own CLI client or do they use
 neutron client?
 - If they use neutron client project for CLI, how do we handle enabling of
 any devstack plugins (without causing dependencies on external projects)?

 Thanks in advance!

 Paul Michali (pc_m)

 [1] https://review.openstack.org/#/c/210021/
 https://review.openstack.org/#/c/209887/ - upstreamed
 [2] https://review.openstack.org/#/c/209887/ - in review
 [3] https://review.openstack.org/#/c/214587/
 https://review.openstack.org/#/c/209887/ - in review

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

Xav,

In the email, there were no responses of anyone using VPNaaS *in a
production environment*. Summary from responders:

Erik M - Tried in Juno with no success. Will retry.
Edgar M - said no reports from operators about VPNaaS code
Sam S - Using VPN in VMs and not VPNaaS
Kevin B - Not used. Use VMs instead
Sriram S - Indicating not used.

If I misread the responses, or if someone has not spoken up, right now is
the time to let us know of your situation and the impact this proposal
would have on your use of VPNaaS IPSec site-to-site connections.

The request here, is that if operators are not using this in a production
deployment where they need backward compatibility, then we'd like to avoid
having to provide the complexity needed to support backward compatibility.

In the proposal, there are two APIs that would be changed with this
enhancement. It's detailed in reference [1], and I can elaborate, if needed.

Please keep in mind, that users/operators using previous versions, can
upgrade to the new version, and any existing VPNaaS configuration will be
automatically migrated to the new table structures, so that existing IPSec
connections would continue to operate with the new release.

The proposal would not support using the older APIs, once the new APIs are
available. Client apps/scripts would need to be updated to use the new API
(neutron client and the Horizon dashboard will be updated as part of the
overall effort).

I hope that clarifies the discussion.

Regards,

Paul Michali (pc_m)


On Mon, Aug 24, 2015 at 3:50 PM Xav Paice xavpa...@gmail.com wrote:

 On 25/08/15 06:32, Paul Michali wrote:

 As part of the effort to provide support for multiple local subnets for
 VPNaaS IPSec connections, there are three API changes planned [1].

 One is to add a new endpoint groups API that will describe what is being
 connected. This is the place were local and peer subnets will be specified.
 It will allow future VPN flavors to reuse this API for other types of VPN
 endpoints.

 The second is to modify the IPSec site connection API to make use of this
 new endpoint groups API and no longer use the peer CIDRs arguments.

 The third is to remove the subnet ID from the VPN service API, and again,
 use the new endpoint groups API for this information (and to allow 1).

 A previous email send out from Kyle Mestery, asking about the production
 usage of VPNaaS, did not indicate that this service was used in production
 by operators.


 Not true.  There were replies indicating that it is indeed in use, but
 perhaps not from operators of installations deemed significant enough -
 what's the criteria here?


 As a result, we'd like to propose making this change as a non-backward
 compatible change, which greatly reduces the effort.

 The change WILL support migration, so older databases can be migrated to
 the new schema and continue to operate, with future changes using the new
 API. This gives a smooth upgrade path to the new capability.

 The change would NOT support the older API in parallel with the new API,
 so users upgrading would need to also update any client scripts/tools to
 use the revised/new VPN API.


 How will this integrate with Horizon?  The majority of our users don't use
 the API directly, but use Horizon for the config, does this mean we'll be
 tied in to using a particular version of Horizon to match Neutron?


 If this is acceptable to operators, we'd like to go forward with these
 plans. Please respond within 7 days of this email, if you have concerns
 about the proposal.

 Regards,

 Paul Michali (IRC pc_m)
 VPNaaS Core

 Ref: https://review.openstack.org/#/c/191944/


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] PLEASE READ: VPNaaS API Change - not backward compatible

[Paul Michali]

Thanks Kevin, I think I was only seeing messages that had also replied all
to openstack-dev mailing list (I wasn't on openstack-operators, until
today).

Regards,

Paul Michali (pc_m)


On Mon, Aug 24, 2015 at 5:11 PM Kevin Benton blak...@gmail.com wrote:

 It sounds like you might have missed a couple responses:


 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007903.html

 http://lists.openstack.org/pipermail/openstack-operators/2015-August/007910.html


 On Mon, Aug 24, 2015 at 1:53 PM, Paul Michali p...@michali.net wrote:

 Xav,

 In the email, there were no responses of anyone using VPNaaS *in a
 production environment*. Summary from responders:

 Erik M - Tried in Juno with no success. Will retry.
 Edgar M - said no reports from operators about VPNaaS code
 Sam S - Using VPN in VMs and not VPNaaS
 Kevin B - Not used. Use VMs instead
 Sriram S - Indicating not used.

 If I misread the responses, or if someone has not spoken up, right now is
 the time to let us know of your situation and the impact this proposal
 would have on your use of VPNaaS IPSec site-to-site connections.

 The request here, is that if operators are not using this in a production
 deployment where they need backward compatibility, then we'd like to avoid
 having to provide the complexity needed to support backward compatibility.

 In the proposal, there are two APIs that would be changed with this
 enhancement. It's detailed in reference [1], and I can elaborate, if needed.

 Please keep in mind, that users/operators using previous versions, can
 upgrade to the new version, and any existing VPNaaS configuration will be
 automatically migrated to the new table structures, so that existing IPSec
 connections would continue to operate with the new release.

 The proposal would not support using the older APIs, once the new APIs
 are available. Client apps/scripts would need to be updated to use the new
 API (neutron client and the Horizon dashboard will be updated as part of
 the overall effort).

 I hope that clarifies the discussion.

 Regards,

 Paul Michali (pc_m)


 On Mon, Aug 24, 2015 at 3:50 PM Xav Paice xavpa...@gmail.com wrote:

 On 25/08/15 06:32, Paul Michali wrote:

 As part of the effort to provide support for multiple local subnets for
 VPNaaS IPSec connections, there are three API changes planned [1].

 One is to add a new endpoint groups API that will describe what is
 being connected. This is the place were local and peer subnets will be
 specified. It will allow future VPN flavors to reuse this API for other
 types of VPN endpoints.

 The second is to modify the IPSec site connection API to make use of
 this new endpoint groups API and no longer use the peer CIDRs arguments.

 The third is to remove the subnet ID from the VPN service API, and
 again, use the new endpoint groups API for this information (and to allow
 1).

 A previous email send out from Kyle Mestery, asking about the production
 usage of VPNaaS, did not indicate that this service was used in production
 by operators.


 Not true.  There were replies indicating that it is indeed in use, but
 perhaps not from operators of installations deemed significant enough -
 what's the criteria here?


 As a result, we'd like to propose making this change as a non-backward
 compatible change, which greatly reduces the effort.

 The change WILL support migration, so older databases can be migrated to
 the new schema and continue to operate, with future changes using the new
 API. This gives a smooth upgrade path to the new capability.

 The change would NOT support the older API in parallel with the new API,
 so users upgrading would need to also update any client scripts/tools to
 use the revised/new VPN API.


 How will this integrate with Horizon?  The majority of our users don't
 use the API directly, but use Horizon for the config, does this mean we'll
 be tied in to using a particular version of Horizon to match Neutron?


 If this is acceptable to operators, we'd like to go forward with these
 plans. Please respond within 7 days of this email, if you have concerns
 about the proposal.

 Regards,

 Paul Michali (IRC pc_m)
 VPNaaS Core

 Ref: https://review.openstack.org/#/c/191944/


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: 
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org

Re: [openstack-dev] [neutron] I am pleased to propose two new Neutron API/DB/RPC core reviewers!

[Paul Michali]

Congratulations Brandon and Russell!

On Sat, Aug 22, 2015 at 6:36 PM Doug Wiegley doug...@parksidesoftware.com
wrote:

 New guys have to plan the liberty summit get together, right? :)


 Doug


 On Aug 22, 2015, at 2:06 PM, Kyle Mestery mest...@mestery.com wrote:

 It's been over a week, so I'd like to welcome Brandon and Russell to the
 API/DB/RPC team in Neutron!

 Kyle

 On Wed, Aug 12, 2015 at 6:45 AM, Kyle Mestery mest...@mestery.com wrote:

 It gives me great pleasure to propose Russell Bryant and Brandon Logan as
 core reviewers in the API/DB/RPC area of Neutron. Russell and Brandon have
 both been incredible contributors to Neutron for a while now. Their
 expertise has been particularly helpful in the area they are being proposed
 in. Their review stats [1] place them both comfortably in the range of
 existing Neutron core reviewers. I expect them to continue working with all
 community members to drive Neutron forward for the rest of Liberty and into
 Mitaka.

 Existing DB/API/RPC core reviewers (and other Neutron core reviewers),
 please vote +1/-1 for the addition of Russell and Brandon.

 Thanks!
 Kyle

 [1] http://stackalytics.com/report/contribution/neutron-group/90


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] Next IRC meeting - Tuesday, August 4th.

[Paul Michali]

Will hold another meeting tomorrow. If you have items for the agenda,
please update the Wiki.

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

[Paul Michali]

Maybe I'm not explaining myself well (sorry)...

For VPN commits, there are functional jobs that (now) enable the devstack
plugin for neutron-vpnaas as needed (and grenade job will do the same).
From the neutron-vpnaas repo standpoint everything is in place.

Now that there is a devstack plugin for neutron-vpnaas, I want to remove
all the VPN setup from the *DevStack* repo's setup, as the user of DevStack
can specify the enable_plugin in their local.conf file now. The commit is
https://review.openstack.org/#/c/201119/.

The issue I see though, is that the DevStack repo's jobs are failing,
because they are using devstack, are relying on VPN being set up, and the
enable_plugin line for VPN isn't part of any of the jobs shown in my last
post.

How do we resolve that issue?

Regards,

PCM


On Mon, Jul 27, 2015 at 8:09 AM Sean Dague s...@dague.net wrote:

 You would build variants of the jobs you want that specifically enable
 your plugin.

 That being said, you should focus on jobs that substantially test your
 component, not just the giant list of all jobs. Part of our focus in on
 decoupling so that for something like vpnaas you can start with the
 assumption that neutron base services are sufficiently tested elsewhere,
 and the only thing you should test is the additional function and
 complexity that your component brings to the mix.

 -Sean

 On 07/27/2015 07:44 AM, Paul Michali wrote:
  Yes, the plugin enables the service, and for the neutron-vpnaas DSVM
  based jobs, I have the enable_plugin line added to the job so that
  everything works.
 
  However, for the DevStack repo, which runs a bunch of other DSVM jobs,
  this fails, as there is (obviously) no enable_plugin line.:
 
* gate-tempest-dsvm-full
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-full/98be491/
 SUCCESS in
  58m 37s
* gate-tempest-dsvm-postgres-full
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-postgres-full/85c5b92/
 SUCCESS in
  50m 45s
* gate-tempest-dsvm-neutron-full
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-full/0050bfe/
 FAILURE in
  1h 25m 30s
* gate-grenade-dsvm
  
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm/b224606/
 SUCCESS in
  44m 23s
* gate-tempest-dsvm-large-ops
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-large-ops/a250cf5/
 SUCCESS in
  26m 49s
* gate-tempest-dsvm-neutron-large-ops
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-large-ops/6faa1be/
 SUCCESS in
  25m 51s
* gate-devstack-bashate
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-bashate/65ad952/
 SUCCESS in
  13s
* gate-devstack-unit-tests
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-unit-tests/ccdbe4e/
 SUCCESS in
  1m 02s
* gate-devstack-dsvm-cells
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-cells/a6ca00c/
 SUCCESS in
  24m 08s
* gate-grenade-dsvm-partial-ncpu
  
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-partial-ncpu/744deb8/
 SUCCESS in
  48m 36s
* gate-tempest-dsvm-ironic-pxe_ssh
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-ironic-pxe_ssh/8eb4315/
 FAILURE in
  40m 10s
* gate-devstack-dsvm-updown
  
 http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-updown/85f1de5/
 SUCCESS in
  21m 12s
* gate-tempest-dsvm-f21
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-f21/35a04c4/
 FAILURE in
  51m 01s (non-voting)
* gate-tempest-dsvm-centos7
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-centos7/b9c99c9/
 SUCCESS in
  30m 23s (non-voting)
* gate-devstack-publish-docs
  
 http://docs-draft.openstack.org/19/201119/1/check/gate-devstack-publish-docs/f794b1c//doc/build/html/
 SUCCESS in
  2m 23s
* gate-swift-dsvm-functional-nv
  
 http://logs.openstack.org/19/201119/1/check/gate-swift-dsvm-functional-nv/13d2c58/
 SUCCESS in
  27m 12s (non-voting)
* gate-grenade-dsvm-neutron
  
 http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-neutron/8675f0c/
 FAILURE in
  47m 49s
* gate-tempest-dsvm-multinode-smoke
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-multinode-smoke/bd69c45/
 SUCCESS in
  36m 53s (non-voting)
* gate-tempest-dsvm-neutron-multinode-smoke
  
 http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-multinode-smoke/01e1d45/
 FAILURE in
  44m 16s (non-voting)
 
 
  I'm wondering what's the best way to modify those jobs... is there some
  common location where I can enable the plugin to handle all DSVM based
  jobs, do I just update the 5 failing tests, do I update just the 3
  voting tests, or do I update all 16 DSVM based jobs?
 
  Regards,
  PCM
 
  On Fri, Jul 24, 2015 at 5:12 PM Clark Boylan cboy

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

[Paul Michali]

Yes, the plugin enables the service, and for the neutron-vpnaas DSVM based
jobs, I have the enable_plugin line added to the job so that everything
works.

However, for the DevStack repo, which runs a bunch of other DSVM jobs, this
fails, as there is (obviously) no enable_plugin line.:


   - gate-tempest-dsvm-full
   http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-full/98be491/
SUCCESS in 58m 37s
   - gate-tempest-dsvm-postgres-full
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-postgres-full/85c5b92/
SUCCESS in 50m 45s
   - gate-tempest-dsvm-neutron-full
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-full/0050bfe/
FAILURE in 1h 25m 30s
   - gate-grenade-dsvm
   http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm/b224606/
   SUCCESS in 44m 23s
   - gate-tempest-dsvm-large-ops
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-large-ops/a250cf5/
SUCCESS in 26m 49s
   - gate-tempest-dsvm-neutron-large-ops
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-large-ops/6faa1be/
SUCCESS in 25m 51s
   - gate-devstack-bashate
   http://logs.openstack.org/19/201119/1/check/gate-devstack-bashate/65ad952/
SUCCESS in 13s
   - gate-devstack-unit-tests
   
http://logs.openstack.org/19/201119/1/check/gate-devstack-unit-tests/ccdbe4e/
SUCCESS in 1m 02s
   - gate-devstack-dsvm-cells
   
http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-cells/a6ca00c/
SUCCESS in 24m 08s
   - gate-grenade-dsvm-partial-ncpu
   
http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-partial-ncpu/744deb8/
SUCCESS in 48m 36s
   - gate-tempest-dsvm-ironic-pxe_ssh
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-ironic-pxe_ssh/8eb4315/
FAILURE in 40m 10s
   - gate-devstack-dsvm-updown
   
http://logs.openstack.org/19/201119/1/check/gate-devstack-dsvm-updown/85f1de5/
SUCCESS in 21m 12s
   - gate-tempest-dsvm-f21
   http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-f21/35a04c4/
FAILURE in 51m 01s (non-voting)
   - gate-tempest-dsvm-centos7
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-centos7/b9c99c9/
SUCCESS in 30m 23s (non-voting)
   - gate-devstack-publish-docs
   
http://docs-draft.openstack.org/19/201119/1/check/gate-devstack-publish-docs/f794b1c//doc/build/html/
SUCCESS in 2m 23s
   - gate-swift-dsvm-functional-nv
   
http://logs.openstack.org/19/201119/1/check/gate-swift-dsvm-functional-nv/13d2c58/
SUCCESS in 27m 12s (non-voting)
   - gate-grenade-dsvm-neutron
   
http://logs.openstack.org/19/201119/1/check/gate-grenade-dsvm-neutron/8675f0c/
FAILURE in 47m 49s
   - gate-tempest-dsvm-multinode-smoke
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-multinode-smoke/bd69c45/
SUCCESS in 36m 53s (non-voting)
   - gate-tempest-dsvm-neutron-multinode-smoke
   
http://logs.openstack.org/19/201119/1/check/gate-tempest-dsvm-neutron-multinode-smoke/01e1d45/
FAILURE in 44m 16s (non-voting)


I'm wondering what's the best way to modify those jobs... is there some
common location where I can enable the plugin to handle all DSVM based
jobs, do I just update the 5 failing tests, do I update just the 3 voting
tests, or do I update all 16 DSVM based jobs?

Regards,
PCM

On Fri, Jul 24, 2015 at 5:12 PM Clark Boylan cboy...@sapwetik.org wrote:

 On Fri, Jul 24, 2015, at 02:05 PM, Paul Michali wrote:
  Hi,
 
  I've created a DevStack plugin for the neutron-vpnaas repo. Now, I'm
  trying
  to remove the q-vpn service setup from the DevStack repo (
  https://review.openstack.org/#/c/201119/).
 
  However, I'm hitting an issue in that (almost) every test that uses
  DevStack fails, because it is no longer setting up q-vpn.
 
  How should I modify the tests, so that they setup the q-vpn service, in
  light of the fact that there is a DevStack plugin available for it. Is
  there some common place that I can do the enable_plugin
  neutron-vpnaas...
  line?
 
 Your devstack plugin should enable the service. Then in your jobs you
 just need to enable the plugin which will then enable the vpn service.
 There should be plenty of prior art with the ec2api plugin, glusterfs
 plugin, and others.

 Clark

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] No VPNaaS IRC meeting Tuesday, July 27th.

[Paul Michali]

I'm on vacation tomorrow (yeah!), and there wasn't much new to discuss, so
I was planning on canceling the meeting this week. If you have something
pressing and want to host the meeting, let everyone know, by updating the
agenda and responding to this message. Otherwise you can use neutron IRC
channel or ML to discuss items.

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

[Paul Michali]

Not being very familiar with how this all works, can someone provide a bit
more hand holding here?

The overall question is, do we remove VPN from all the DevStack based tests
(except for those run by VPN repo)?

Thanks,

PCM


On Mon, Jul 27, 2015 at 8:26 AM Sean Dague s...@dague.net wrote:

 On 07/27/2015 08:21 AM, Paul Michali wrote:
  Maybe I'm not explaining myself well (sorry)...
 
  For VPN commits, there are functional jobs that (now) enable the
  devstack plugin for neutron-vpnaas as needed (and grenade job will do
  the same). From the neutron-vpnaas repo standpoint everything is in
 place.
 
  Now that there is a devstack plugin for neutron-vpnaas, I want to remove
  all the VPN setup from the *DevStack* repo's setup, as the user of
  DevStack can specify the enable_plugin in their local.conf file now. The
  commit is https://review.openstack.org/#/c/201119/.
 
  The issue I see though, is that the DevStack repo's jobs are failing,
  because they are using devstack, are relying on VPN being set up, and
  the enable_plugin line for VPN isn't part of any of the jobs shown in my
  last post.
 
  How do we resolve that issue?

 Presumably there is a flag in Tempest for whether or not this service
 should be tested? That would be where I'd look.

 -Sean

 --
 Sean Dague
 http://dague.net

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][testing] How to modify DSVM tests to use a DevStack plugin?

[Paul Michali]

Hi,

I've created a DevStack plugin for the neutron-vpnaas repo. Now, I'm trying
to remove the q-vpn service setup from the DevStack repo (
https://review.openstack.org/#/c/201119/).

However, I'm hitting an issue in that (almost) every test that uses
DevStack fails, because it is no longer setting up q-vpn.

How should I modify the tests, so that they setup the q-vpn service, in
light of the fact that there is a DevStack plugin available for it. Is
there some common place that I can do the enable_plugin neutron-vpnaas...
line?

Thanks!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Cedric Brandily to Neutron Core Reviewer Team

[Paul Michali]

Great to see you added as a core!


On Thu, Jul 23, 2015 at 3:02 AM Somanchi Trinath 
trinath.soman...@freescale.com wrote:

  Congratulations Cedric. J



 *From:* Miguel Lavalle [mailto:mig...@mlavalle.com]
 *Sent:* Wednesday, July 22, 2015 10:32 PM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [Neutron] Proposing Cedric Brandily to
 Neutron Core Reviewer Team



 Congrats Cedric!



 On Wed, Jul 22, 2015 at 10:37 AM, Carl Baldwin c...@ecbaldwin.net wrote:

 It has been a week since this nomination.  I'm pleased to confirm
 Cedric as a core reviewer for these areas of focus.  We look forward
 to your continued contribution to the project!

 Carl

 On Wed, Jul 15, 2015 at 12:47 PM, Carl Baldwin c...@ecbaldwin.net wrote:
  As the Neutron L3 Lieutenant along with Kevin Benton for control
  plane, and Assaf Muller for testing, I would like to propose Cedric
  Brandily as a member of the Neutron core reviewer team under these
  areas of focus.
 
  Cedric has been a long time contributor to Neutron showing expertise
  particularly in these areas.  His knowledge and involvement will be
  very important to the project.  He is a trusted member of our
  community.  He has been reviewing consistently [1][2] and community
  feedback that I've received indicates that he is a solid reviewer.
 
  Existing Neutron core reviewers from these areas of focus, please vote
  +1/-1 for the addition of Cedric to the team.
 
  Thanks!
  Carl Baldwin
 
  [1] https://review.openstack.org/#/q/reviewer:zzelle%2540gmail.com,n,z
  [2] http://stackalytics.com/report/contribution/neutron-group/90

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Plethora of dbase migration questions...

[Paul Michali]

Thanks Salvatore for the responses. See @PCM in-line...



On Tue, Jul 7, 2015 at 6:14 AM Salvatore Orlando sorla...@nicira.com
wrote:

 Some comments inline.

 Salvatore

 On 6 July 2015 at 20:00, Paul Michali p...@michali.net wrote:

 Hi,

 I have some urgent requests about migration that I'm hoping to get some
 info on. I'm working on a bug where I need to add two (related) fields to a
 table for VPNaaS. Here's the objectives related to migration...

 1) create local_v4_ip and lcoal_v6_ip fields in the vpnservice table
 2) for each entry in the vpnservice table:
 2.1) Get the router.gw_port.fixed_ips list
 2.2) Determine the version of each fixed IP and store the first of
 each version (if any) into the appropriate new field.

 I have created a migration file, and I changed the down_revision to be
 the number of the revision that is the first in the migration chain in the
 VPN repo.

 Here are the many questions I have...

 When I look in the VPN repo, the HEAD file has the version 'kilo', which
 is not the current head.


 Shouldn't it the version number of the first file in the migration chain?


 It should indeed. How are you generating the revision script? Using
 neutron-db-manage it should be updated automatically [1]


@PCM I ran neutron-db-manage, when in the neutron repo, and it assigned
some version, but it was not the latest in the neutron-vpnaas repo.

I checked the VPN repo and there were a chain of versions, which I used to
determine what the head should be and have set the version accordingly.
However, in the current repo, head is set to kilo, which appears to be
incorrect.  The versions are:

5689aa52
kiloHEAD
3ea02b2a773e
start_neutron_vpnaas
None

Should I do a separate commit that fixes the HEAD file, or just fix it as
part of the bug fix I'm working on.

BTW, at one point, after having correctly set the HEAD and versions in my
new migration file, I think I ran neutron-db-manage check_migration, and I
think it set the HEAD to my version, but it did that in the neutron repo,
and not the VPN repo.  I might have been running from the wrong repo?



 For my commit, I'm assuming I change the HEAD file to use my migration
 file's version?


 You can do that manually too, yes.



 I set HEAD to my migration file, and my file has a down revision of the
 previous head's revision. If I run 'neutron-db-manage --config-file
 ../neutron/etc/neutron.conf --config-file
 ../neutron/etc/neutron/plugins/ml2/ml2_conf.ini check_migration' there is
 no output so I guess that is OK.

 As I develop my new migration file, is there a way that I can test it
 (running neutron-db-migration, maybe)?


 When I test migrations I usually dump the database, run the migration with
 neutron-db-manage upgrade HEAD (I think it's not necessary to specify
 HEAD), and restore the db from the dump if the migration fails.


 Is there a way to run the migration file under the debugger, as well
 (importing pdb, for example)?


 The migration process is just like any python application, so I guess you
 can debug it with pdb.


@PCM Ah, so use neutron-db-manage upgrade HEAD. That was the piece that
was missing. I take it there are no specific unit tests of the migration
files?





 In the migration, I can add the columns needed. What's the best way to
 fill out those fields - using raw SQL queries or create a Session object
 and access the VpnService object's router object?


 If the default value for the column is not enough, and you need to specify
 a value which depends on other values in the same row I would prefer plain
 SQL statements, but if that become cumbersome I guess it's ok to use
 sqlalchemy's session.


 I see there is some op.bind() call and then engine.execute(), but could
 use some help on the best way to extract the needed queries (I need to
 access the vpnservice's router, and then access the (Port) gw_port
 relationship, and from that access the (IPAllocation) fixed_ips list).


 Perhaps you can point us to the review pages on gerrit, and we can provide
 detailed comments there.


@PCM Yeah, I haven't pushed it up yet. I have a few more changes to make,
and should be able to get it up in a few days. The LP bug is 1464387.

Essentially, in the vpnservices table, I'm adding an IPv4 and/or IPv6
addresses for the  local end of VPN connections that will be established.
This is to allow alternative VPN implementation (appliances, separate S/W,
H/W, VM based VPN, etc) to specify addresses different than what is
available on the Neutron router.

However, for the reference implementation, we'll use the Neutron router's
fixed_ips list (as is done today), and to handle the migration, I'm
thinking the following is needed:

1) create the new columns.
2) Identify the router for that service and obtain it's GW fixed_ips list.
3) Pick first IPv4 address (if any) and IPv6 address (if any), and store in
new columns.

So I need to form a query and code to do this.





 Appreciate any advise here on how

Re: [openstack-dev] [neutron] Plethora of dbase migration questions...

[Paul Michali]

And for the query, it involves several table references (Router, Port,
IPAllocation).



On Tue, Jul 7, 2015 at 8:00 AM Paul Michali p...@michali.net wrote:

 Thanks Salvatore for the responses. See @PCM in-line...



 On Tue, Jul 7, 2015 at 6:14 AM Salvatore Orlando sorla...@nicira.com
 wrote:

 Some comments inline.

 Salvatore

 On 6 July 2015 at 20:00, Paul Michali p...@michali.net wrote:

 Hi,

 I have some urgent requests about migration that I'm hoping to get some
 info on. I'm working on a bug where I need to add two (related) fields to a
 table for VPNaaS. Here's the objectives related to migration...

 1) create local_v4_ip and lcoal_v6_ip fields in the vpnservice table
 2) for each entry in the vpnservice table:
 2.1) Get the router.gw_port.fixed_ips list
 2.2) Determine the version of each fixed IP and store the first of
 each version (if any) into the appropriate new field.

 I have created a migration file, and I changed the down_revision to be
 the number of the revision that is the first in the migration chain in the
 VPN repo.

 Here are the many questions I have...

 When I look in the VPN repo, the HEAD file has the version 'kilo', which
 is not the current head.


 Shouldn't it the version number of the first file in the migration chain?


 It should indeed. How are you generating the revision script? Using
 neutron-db-manage it should be updated automatically [1]


 @PCM I ran neutron-db-manage, when in the neutron repo, and it assigned
 some version, but it was not the latest in the neutron-vpnaas repo.

 I checked the VPN repo and there were a chain of versions, which I used to
 determine what the head should be and have set the version accordingly.
 However, in the current repo, head is set to kilo, which appears to be
 incorrect.  The versions are:

 5689aa52
 kiloHEAD
 3ea02b2a773e
 start_neutron_vpnaas
 None

 Should I do a separate commit that fixes the HEAD file, or just fix it as
 part of the bug fix I'm working on.

 BTW, at one point, after having correctly set the HEAD and versions in my
 new migration file, I think I ran neutron-db-manage check_migration, and I
 think it set the HEAD to my version, but it did that in the neutron repo,
 and not the VPN repo.  I might have been running from the wrong repo?



 For my commit, I'm assuming I change the HEAD file to use my migration
 file's version?


 You can do that manually too, yes.



 I set HEAD to my migration file, and my file has a down revision of the
 previous head's revision. If I run 'neutron-db-manage --config-file
 ../neutron/etc/neutron.conf --config-file
 ../neutron/etc/neutron/plugins/ml2/ml2_conf.ini check_migration' there is
 no output so I guess that is OK.

 As I develop my new migration file, is there a way that I can test it
 (running neutron-db-migration, maybe)?


 When I test migrations I usually dump the database, run the migration
 with neutron-db-manage upgrade HEAD (I think it's not necessary to specify
 HEAD), and restore the db from the dump if the migration fails.


 Is there a way to run the migration file under the debugger, as well
 (importing pdb, for example)?


 The migration process is just like any python application, so I guess you
 can debug it with pdb.


 @PCM Ah, so use neutron-db-manage upgrade HEAD. That was the piece that
 was missing. I take it there are no specific unit tests of the migration
 files?





 In the migration, I can add the columns needed. What's the best way to
 fill out those fields - using raw SQL queries or create a Session object
 and access the VpnService object's router object?


 If the default value for the column is not enough, and you need to
 specify a value which depends on other values in the same row I would
 prefer plain SQL statements, but if that become cumbersome I guess it's ok
 to use sqlalchemy's session.


 I see there is some op.bind() call and then engine.execute(), but could
 use some help on the best way to extract the needed queries (I need to
 access the vpnservice's router, and then access the (Port) gw_port
 relationship, and from that access the (IPAllocation) fixed_ips list).


 Perhaps you can point us to the review pages on gerrit, and we can
 provide detailed comments there.


 @PCM Yeah, I haven't pushed it up yet. I have a few more changes to make,
 and should be able to get it up in a few days. The LP bug is 1464387.

 Essentially, in the vpnservices table, I'm adding an IPv4 and/or IPv6
 addresses for the  local end of VPN connections that will be established.
 This is to allow alternative VPN implementation (appliances, separate S/W,
 H/W, VM based VPN, etc) to specify addresses different than what is
 available on the Neutron router.

 However, for the reference implementation, we'll use the Neutron router's
 fixed_ips list (as is done today), and to handle the migration, I'm
 thinking the following is needed:

 1) create the new columns.
 2) Identify the router for that service and obtain it's GW

[openstack-dev] [neutron] Plethora of dbase migration questions...

[Paul Michali]

Hi,

I have some urgent requests about migration that I'm hoping to get some
info on. I'm working on a bug where I need to add two (related) fields to a
table for VPNaaS. Here's the objectives related to migration...

1) create local_v4_ip and lcoal_v6_ip fields in the vpnservice table
2) for each entry in the vpnservice table:
2.1) Get the router.gw_port.fixed_ips list
2.2) Determine the version of each fixed IP and store the first of each
version (if any) into the appropriate new field.

I have created a migration file, and I changed the down_revision to be the
number of the revision that is the first in the migration chain in the VPN
repo.

Here are the many questions I have...

When I look in the VPN repo, the HEAD file has the version 'kilo', which is
not the current head.

Shouldn't it the version number of the first file in the migration chain?
For my commit, I'm assuming I change the HEAD file to use my migration
file's version?

I set HEAD to my migration file, and my file has a down revision of the
previous head's revision. If I run 'neutron-db-manage --config-file
../neutron/etc/neutron.conf --config-file
../neutron/etc/neutron/plugins/ml2/ml2_conf.ini check_migration' there is
no output so I guess that is OK.

As I develop my new migration file, is there a way that I can test it
(running neutron-db-migration, maybe)?
Is there a way to run the migration file under the debugger, as well
(importing pdb, for example)?

In the migration, I can add the columns needed. What's the best way to fill
out those fields - using raw SQL queries or create a Session object and
access the VpnService object's router object?

I see there is some op.bind() call and then engine.execute(), but could use
some help on the best way to extract the needed queries (I need to access
the vpnservice's router, and then access the (Port) gw_port relationship,
and from that access the (IPAllocation) fixed_ips list).

Appreciate any advise here on how to debug the migration stuff...

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][lbaas] Proposing Al Miller for neutron-lbaas core team

[Paul Michali]

Great to have you as a LBass core Al!

On Mon, Jul 6, 2015 at 1:29 PM Doug Wiegley doug...@parksidesoftware.com
wrote:

 Since all cores have responded, I’m going to end this early.  Welcome to
 the core team, Al.

 Thanks,
 doug

 On Jul 5, 2015, at 6:27 PM, Kyle Mestery mest...@mestery.com wrote:

 +1 for Al!

 On Thu, Jul 2, 2015 at 5:16 PM, Doug Wiegley doug...@parksidesoftware.com
  wrote:

 Hi all,

 As the Lieutenant of the advanced services, I would like to nominate Al
 Miller to be a member of the neutron-lbaas core reviewer team.

 Review stats are in line with other cores[2] and feedback on patches has
 been great. Additionally, he has been instrumental in our devstack support
 and octavia work.

 Existing cores, please vote +1/-1 for his addition to the team (that’s
 Brandon, Phil, and Kyle.)

 Thanks,
 doug

 1.
 http://docs.openstack.org/developer/neutron/policies/core-reviewers.html#core-review-hierarchy
 2. http://stackalytics.com/report/contribution/neutron-lbaas/90
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [VPNaas]How to load kernel module with IPSec?

[Paul Michali]

Curious as to what operating system you are using and which release?

Are you running under DevStack or doing an OpenStack install?

Regards,

Paul Michali (pc_m)

On Mon, Jun 29, 2015 at 6:31 AM Zhi Chang chang...@unitedstack.com wrote:

 Hi, all
 I have some questions about how to load kernel module of IPSec. I'm
 using Openswan to build VPNaas and there is a error message says no kernel
 code presently loaded when I run ipsec verify. My solution is running
 service ipsec start on host to load kernel module. Everything goes okay
 when I run it. But I think the solution is too ungraceful. Does anyone
 have a simple solution to resolve this problem instead of run service
 ipsec start?

 Thx.
 Zhi Chang
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] Scenario test for VPN getting socket error

[Paul Michali]

For review 159746, we are seeing that a traceback is occurring (PS26) that
appears to be caused by two rootwrap commands running at the same time and
trying to use the same socket for communication with the daemon. This
happens in one functional job, and not the other. The difference being that
the failing (dsvm-functional-sswan) has it's own rootwrap function, instead
of  using ip_lib.IPWrapper().

There are two main questions here. One is whether or not we need the custom
rootwrap logic, or if ip_lib methods can be used to mount the desired paths
and run the commands, as needed. There is a mounting of /etc and /var/run.
I'm guessing that the former is to allow customizing of ipsec config files
for the connection being created. I'm not sure why /var/run is mounted (and
if that is interfering with the rootwrap daemon operation).

The other is why this issue is happening with the one job and not the other
(or general use of IPWrapper where there are long running and short running
commands happening at once (and how to resolve this issue). Both jobs do
the same tasks, only with different (external) VPN driver processes.

In the failure case, an execute() is done from ip_lib's get_devices() for a
find operation, and a second execute() is done by send_ip_addr_adv_notif()
for a arping operation (long). In the working case, these two operations
seem to happen simultaneously w/o incident.

Any thoughts on what may be happening, would be appreciated!

Regards,

Paul Michali (pc_m)

Ref:
https://review.openstack.org/#/c/159746/28/neutron_vpnaas/tests/functional/common/test_scenario.py
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [VPNaas]How to load kernel module with IPSec?

[Paul Michali]

Ah, so Icehouse... From what I recall, there were two problems running RHEL
type operating systems with *Swan. First, they use LibSwan instead of
OpenSwan. Second, there were some config/setup problems with StrongSwan
based connections.

Recently, there were some commits to resolve these issues. For the kernel
issue that you have, see commit 72e1f670, which creates a LibSwan driver
and deals with the kernel module loading.  You may need to backport that
fix to run VPN under CentOS.

Regards,

Paul Michali (pc_m)


On Mon, Jun 29, 2015 at 8:26 AM Zhi Chang chang...@unitedstack.com wrote:

 Hi, thanks for you reply.
 My OS is CentOS 6.5 and doing an OpenStack install, and my OpenStack
 verison is I.

 Regards,
 Zhi Chang


 -- Original --
 *From: * Paul Michalip...@michali.net;
 *Date: * Mon, Jun 29, 2015 06:37 PM
 *To: * OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org;
 *Subject: * Re: [openstack-dev] [VPNaas]How to load kernel module with
 IPSec?

 Curious as to what operating system you are using and which release?

 Are you running under DevStack or doing an OpenStack install?

 Regards,

 Paul Michali (pc_m)

 On Mon, Jun 29, 2015 at 6:31 AM Zhi Chang chang...@unitedstack.com
 wrote:

 Hi, all
 I have some questions about how to load kernel module of IPSec. I'm
 using Openswan to build VPNaas and there is a error message says no kernel
 code presently loaded when I run ipsec verify. My solution is running
 service ipsec start on host to load kernel module. Everything goes okay
 when I run it. But I think the solution is too ungraceful. Does anyone
 have a simple solution to resolve this problem instead of run service
 ipsec start?

 Thx.
 Zhi Chang
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Rossella Sblendido for the Control Plane core team

[Paul Michali]

Congratulations Rossella! Great addition to the team!
On Sat, Jun 20, 2015 at 12:13 PM Miguel Lavalle mig...@mlavalle.com wrote:

 Complimenti per il nuovo lavoro!

 On Fri, Jun 19, 2015 at 5:50 PM, Kevin Benton blak...@gmail.com wrote:

 It's been a week with no negative feedback. Welcome to the team Rossella!

 On Mon, Jun 15, 2015 at 2:53 AM, Oleg Bondarev obonda...@mirantis.com
 wrote:

 +1

 On Mon, Jun 15, 2015 at 12:14 PM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256

 No doubt +1.

 On 06/12/2015 09:44 PM, Kevin Benton wrote:
  Hello!
 
  As the Lieutenant of the built-in control plane[1], I would like
  Rossella Sblendido to be a member of the control plane core
  reviewer team.
 
  Her review stats are in line with other cores[2] and her feedback
  on patches related to the agents has been great. Additionally, she
  has been working quite a bit on the blueprint to restructure the L2
  agent code so she is very familiar with the agent code and the APIs
  it leverages.
 
  Existing cores that have spent time working on the reference
  implementation (agents and AMQP code), please vote +1/-1 for her
  addition to the team. Aaron, Gary, Assaf, Maru, Kyle, Armando, Carl
  and Oleg; you have all been reviewing things in these areas
  recently so I would like to hear from you specifically.
 
  1.
 
 http://docs.openstack.org/developer/neutron/policies/core-reviewers.ht
 ml#core-review-hierarchy
 http://docs.openstack.org/developer/neutron/policies/core-reviewers.html#core-review-hierarchy
 
 
 2. http://stackalytics.com/report/contribution/neutron-group/30
 
  Cheers -- Kevin Benton
 
 
  __
 
 
 
 OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v2

 iQEcBAEBCAAGBQJVfpdmAAoJEC5aWaUY1u574skIAOm15mNKujH3X3XSpwtmy+V4
 cWNvjYZy0lCG2lbyAwGwgQD0Ybs88/RKOZPPpu9gugAuWcFQRHMSMcsahaTLcH5B
 6imK0tEUUn2HdCd9522kEWtf7Hkpux6p5TLQQo2tucvIBQohJuU42EidfKs9Peat
 ed/2kiXm+TGRSnZHAYwzJIrttux3ntTLQeTvElo+4vUQEQJNwm8eguXiAPENEjr9
 7Ou3TgnYeLXsfVopHXNV+jtkHDr92giB4joODqwc8RNDKE2+dhaqDxCrXq6ksYq7
 RFiyVy1qve394ebP0Ta0dq6LKmAYZCnRC9i928GSK5RvQBvcnJRiyFGIgtn4Qu0=
 =59av
 -END PGP SIGNATURE-


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Kevin Benton

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposing Brian Haley to Neutron L3 Core Reviewer Team

[Paul Michali]

Congratulations Brian! Great addition to the L3 team!

On Wed, Jun 17, 2015 at 11:14 PM Edgar Magana edgar.mag...@workday.com
wrote:

 Congratulations Brian!  Welcome to the team!

 Edgar




 On 6/17/15, 3:59 PM, Carl Baldwin c...@ecbaldwin.net wrote:

 It has been a week and feedback has been positive and supportive of
 Brian's nomination.  Welcome to the L3 core reviewer team, Brian.
 
 Carl
 
 On Wed, Jun 10, 2015 at 1:11 PM, Carl Baldwin c...@ecbaldwin.net wrote:
  Folks,
 
  As the Neutron L3 Lieutenant [1] under the PTL, Kyle, I'd like to
  propose Brian Haley as a member of the Neutron L3 core reviewer team.
  Brian has been a long time contributor in Neutron showing expertise
  particularly in IPv6, iptables, and Linux kernel matters.  His
  knowledge and involvement will be very important especially in this
  area.  Brian has become a trusted member of our community.  His review
  stats [2][3][4] place him comfortably with other Neutron core
  reviewers.  He regularly runs proposed patches himself and gives
  insightful feedback.  He has shown a lot of interest in the success of
  Neutron.
 
  Existing Neutron core reviewers from the L3 area of focus, please vote
  +1/-1 for the addition of Brian to the core reviewer team.
  Specifically, I'm looking for votes from Henry, Assaf, and Mark.
 
  Thanks!
  Carl
 
  [1]
 http://docs.openstack.org/developer/neutron/policies/core-reviewers.html#adding-or-removing-core-reviewers
  [2]
 https://review.openstack.org/#/q/reviewer:%22Brian+Haley+%253Cbrian.haley%2540hp.com%253E%22,n,z
  [3] http://stackalytics.com/report/contribution/neutron-group/90
  [4] http://stackalytics.com/?user_id=brian-haleymetric=marks
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing Ann Kamyshnikova for the API DB core reviewer team

[Paul Michali]

Congratulations Ann!

On Thu, Jun 18, 2015 at 12:22 PM Edgar Magana edgar.mag...@workday.com
wrote:

 Congratulations Ann and welcome to the team!

 Edgar




 On 6/18/15, 8:56 AM, Henry Gessau ges...@cisco.com wrote:

 It has been a week and feedback has been positive and supportive of
 Ann's nomination.  Welcome to the Neutron DB core reviewer team, Ann.
 
 --
 Henry
 
 On Thu, Jun 11, 2015, Henry Gessau ges...@cisco.com wrote:
  As one of the Lieutenants [1] for the API and DB areas under the PTL, I
 would
  like to propose Ann Kamyshnikova as a member of the Neutron API and DB
 core
  reviewer team.
 
  Ann has been a long time contributor in Neutron showing expertise
 particularly
  in database matters. She has also worked with and contributed code to
 the
  oslo.db and sqlalchemy/alembic projects. Ann was a critical contributor
 to the
  Neutron database healing effort that was completed in the Juno cycle.
 
  Her deep knowledge of databases and backends, and her expertise with
 oslo.db,
  sqlalchemy and alembic will be very important in this area. Ann is a
 trusted
  member of our community and her review stats [2][3][4] place her
 comfortably
  with other Neutron core reviewers. She consistently catches database
 issues
  early when patches are submitted for review, and shows dedication to
 helping
  developers understand and perfect their database-related changes.
 
  Existing Neutron core reviewers from the API and DB area of focus,
 please vote
  +1/-1 for the addition of Ann to the core reviewer team. Specifically,
 I'm
  looking for votes from Akihiro (co-Lieutenant), Mark, Maru, Armando and
 Carl.
 
 
  [1]
 
 http://docs.openstack.org/developer/neutron/policies/core-reviewers.html#adding-or-removing-core-reviewers
  [2]
 
 https://review.openstack.org/#/q/reviewer:%22Ann+Kamyshnikova+%253Cakamyshnikova%2540mirantis.com%253E%22,n,z
  [3] http://stackalytics.com/report/contribution/neutron-group/90
  [4] http://stackalytics.com/?user_id=akamyshnikovametric=marks
 
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Proposing YAMAMOTO Takashi for the Control Plane core team

[Paul Michali]

Congrats Yamamoto!
On Thu, Jun 18, 2015 at 6:23 PM Kevin Benton blak...@gmail.com wrote:

 It has been a week and I haven't heard any negative feedback.
 Welcome to the control plane core reviewer team YAMAMOTO Takashi!

 On Mon, Jun 15, 2015 at 2:52 AM, Oleg Bondarev obonda...@mirantis.com
 wrote:

 +1


 On Mon, Jun 15, 2015 at 12:16 PM, Ihar Hrachyshka ihrac...@redhat.com
 wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256

 Not on the list either, but I want to +1 what Henry said. Yamamoto's
 reviews expand to the whole code base and are pretty always *very* usefu
 l.

 On 06/12/2015 11:39 PM, Henry Gessau wrote:
  Although I am not on your list I would like to add my +1! Yamamoto
  shows great attention to detail in code reviews and frequently
  finds real issues that were not spotted by others.
 
  On Thu, Jun 11, 2015, Kevin Benton blak...@gmail.com wrote:
  Hello all!
 
  As the Lieutenant of the built-in control plane[1], I would like
  YAMAMOTO Takashi to be a member of the control plane core
  reviewer team.
 
  He has been extensively reviewing the entire codebase[2] and his
  feedback on patches related to the reference implementation has
  been very useful. This includes everything ranging from the AMPQ
  API to OVS flows.
 
  Existing cores that have spent time working on the reference
  implementation (agents and AMQP code), please vote +1/-1 for his
  addition to the team. Aaron, Gary, Assaf, Maru, Kyle, Armando,
  Carl and Oleg; you have all been reviewing things in these areas
  recently so I would like to hear from you specifically.
 
  1.
  http://docs.openstack.org/developer/neutron/policies/core-reviewers.h
 tml#core-review-hierarchy
 http://docs.openstack.org/developer/neutron/policies/core-reviewers.html#core-review-hierarchy
 
 
 2. http://stackalytics.com/report/contribution/neutron-group/90
 
 
  Cheers -- Kevin Benton
 
 
  __
 
 
 
 OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v2

 iQEcBAEBCAAGBQJVfpgHAAoJEC5aWaUY1u57z40IAL8HpGOEQY7aW1aa39C9ig3Y
 bNLEabeQ0K5a4+5ynVLIm1sEl4s3GF2r0gCXak9FrkqjHx2r8VeyOx8TqrCj8gX3
 +4aHI7n6rJoJtINuTd+bN7T65uaZE86erOcZN+yma5V69ObfIZhIfQKr3BMaBeZT
 ah5hkUKl88ckLL5zqc0HnT4wcFH/3Ved2fuhP7hw/IEGMFsqTDS8QUTdXg7/OF5t
 fLt0NluKoOuNMOk8dTqpqtQtiyS/E5TH+miVOzyrUeUmZOEayOO3O2b/9QRkX/hL
 ijv1j+clT69fhoVAWSaeR7IsXSfqMuKK/hrVtjnyDpBH4KuZnl3QbRpKJ6Yi3bw=
 =qnTt
 -END PGP SIGNATURE-


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Kevin Benton
  __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron[[vpnaas] Next meeting June 16th at 1600 UTC

[Paul Michali]

Planning on weekly meetings for a while, since this are several things to
discuss. See the agenda on the wiki page:
https://wiki.openstack.org/wiki/Meetings/VPNaaS

There are a bunch of questions to discuss.

See you Tuesday!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Looking for help getting git-review to work over https

[Paul Michali]

 to continue connecting (yes/no)? yes

 ssh://dk0...@review.openstack.org:29418/openstack/horizon.git did not
 work.

 Could not connect to gerrit.

 Enter your gerrit username:

 ---



 *From:* Paul Michali [mailto:p...@michali.net]
 *Sent:* Thursday, June 11, 2015 11:09 AM


 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] Looking for help getting git-review to
 work over https



 Do you know if you have SSH access to the outside world through the
 firewall?



 Did you setup a proxy? I setup 'corkscrew' under Ubuntu. After installing,
 created a .ssh/config file with:



 Host review.openstack.org

 ProxyCommand corkscrew proxy-host 80 %h %p



 The proxy host is one that allows HTTP/HTTPS to outside world and
 corkscrew tunnels the SSH through to port 80.



 HTHs,



 PCM



 On Thu, Jun 11, 2015 at 12:44 PM KARR, DAVID dk0...@att.com wrote:

   Thanks for replying.



 % git review -vs

 2015-06-11 09:30:38.396076 Running: git log --color=never --oneline
 HEAD^1..HEAD

 2015-06-11 09:30:38.399021 Running: git remote

 2015-06-11 09:30:38.401033 Running: git config --get gitreview.username

 No remote set, testing ssh://
 dk0...@review.openstack.org:29418/openstack/horizon.git

 2015-06-11 09:30:38.402988 Running: git push --dry-run ssh://
 dk0...@review.openstack.org:29418/openstack/horizon.git --all

 ssh://dk0...@review.openstack.org:29418/openstack/horizon.git did not
 work.

 Could not connect to gerrit.

 Enter your gerrit username:



 This output is interesting, because I followed the instructions to set the
 scheme and port to https and 443, which can be seen from:

 % git config --global -l

 user.name=David Karr

 user.email=dk0...@att.com

 gitreview.scheme=https

 gitreview.port=443



 Concerning the question ‘Do you have gerrit remote already configured?’,
 I guess I’d have to say I don’t know. I’ve followed instructions for
 setting up my pub key, but I’m not sure exactly what is entailed in “gerrit
 remote”.



 I can get to https://review.openstack.org/ from my browser and from the
 command line with curl.



 The “ls-remote” command returns without error (or any other output).



 *From:* Yuriy Taraday [mailto:yorik@gmail.com]
 *Sent:* Thursday, June 11, 2015 9:19 AM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] Looking for help getting git-review to
 work over https



 On Thu, Jun 11, 2015, 18:09 KARR, DAVID dk0...@att.com wrote:

 I could use some help with setting up git-review in a slightly unfriendly
 firewall situation.

 I'm trying to set up git-review on my CentOS7 VM, and our firewall blocks
 the non-standard ssh port.  I'm following the instructions at
 http://docs.openstack.org/infra/manual/developers.html#accessing-gerrit-over-https
 , for configuring git-review to use https on port 443, but this still isn't
 working (times out with Could not connect to gerrit).  I've confirmed
 that I can reach other external sites on port 443.

 Can someone give me a hand with this?





 Hello.



 - Can you please post all output from git review -vs?

 - Do you have gerrit remote already configured?

 - Do you have access to https://review.openstack.org/ from your browser?

 - Can you access it from command line (via curl -I
 https://review.openstack.org/; for example)?

 - Does git ls-remote https://review.openstack.org/openstack/nova 
 /dev/null produce and error?

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Looking for help getting git-review to work over https

[Paul Michali]

Do you know if you have SSH access to the outside world through the
firewall?

Did you setup a proxy? I setup 'corkscrew' under Ubuntu. After installing,
created a .ssh/config file with:

Host review.openstack.org
ProxyCommand corkscrew proxy-host 80 %h %p

The proxy host is one that allows HTTP/HTTPS to outside world and corkscrew
tunnels the SSH through to port 80.

HTHs,

PCM


On Thu, Jun 11, 2015 at 12:44 PM KARR, DAVID dk0...@att.com wrote:

   Thanks for replying.



 % git review -vs

 2015-06-11 09:30:38.396076 Running: git log --color=never --oneline
 HEAD^1..HEAD

 2015-06-11 09:30:38.399021 Running: git remote

 2015-06-11 09:30:38.401033 Running: git config --get gitreview.username

 No remote set, testing ssh://
 dk0...@review.openstack.org:29418/openstack/horizon.git

 2015-06-11 09:30:38.402988 Running: git push --dry-run ssh://
 dk0...@review.openstack.org:29418/openstack/horizon.git --all

 ssh://dk0...@review.openstack.org:29418/openstack/horizon.git did not
 work.

 Could not connect to gerrit.

 Enter your gerrit username:



 This output is interesting, because I followed the instructions to set the
 scheme and port to https and 443, which can be seen from:

 % git config --global -l

 user.name=David Karr

 user.email=dk0...@att.com

 gitreview.scheme=https

 gitreview.port=443



 Concerning the question ‘Do you have gerrit remote already configured?’,
 I guess I’d have to say I don’t know. I’ve followed instructions for
 setting up my pub key, but I’m not sure exactly what is entailed in “gerrit
 remote”.



 I can get to https://review.openstack.org/ from my browser and from the
 command line with curl.



 The “ls-remote” command returns without error (or any other output).



 *From:* Yuriy Taraday [mailto:yorik@gmail.com]
 *Sent:* Thursday, June 11, 2015 9:19 AM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] Looking for help getting git-review to
 work over https



 On Thu, Jun 11, 2015, 18:09 KARR, DAVID dk0...@att.com wrote:

 I could use some help with setting up git-review in a slightly unfriendly
 firewall situation.

 I'm trying to set up git-review on my CentOS7 VM, and our firewall blocks
 the non-standard ssh port.  I'm following the instructions at
 http://docs.openstack.org/infra/manual/developers.html#accessing-gerrit-over-https
 , for configuring git-review to use https on port 443, but this still isn't
 working (times out with Could not connect to gerrit).  I've confirmed
 that I can reach other external sites on port 443.

 Can someone give me a hand with this?





 Hello.



 - Can you please post all output from git review -vs?

 - Do you have gerrit remote already configured?

 - Do you have access to https://review.openstack.org/ from your browser?

 - Can you access it from command line (via curl -I
 https://review.openstack.org/; for example)?

 - Does git ls-remote https://review.openstack.org/openstack/nova 
 /dev/null produce and error?

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas][barbican] What types of authentication to support?

[Paul Michali]

Hi,

There is a Request for Feature Enhancement [1] to support authentication
certifications for VPNaaS IPSec site to site connections, by using
Barbican, in a manner similar to what was done for LBaaS listeners.

Currently, VPNaaS only supports pre-shared keys for authentication, but the
reference StrongSwan implementation of VPN supports several types of
authentication. [2]

Looking at IPsec site-to-site connections, there are examples [3] for PSK
and X.509 certificates.

Should we just do X.509 certificates for now?
Are there other methods that we should support?
Can Barbican support such methods?

The plan is to support other VPN types in the future (e.g. DM VPN), so we
want to make sure this will be extendable.

Suggestions/Comments/Concerns?

Thanks!

Paul Michali (pc_m)


[1] https://bugs.launchpad.net/neutron/+bug/1459427
[2] https://wiki.strongswan.org/projects/strongswan/wiki/IpsecSecrets
[3] https://wiki.strongswan.org/projects/strongswan/wiki/IKEv2Examples (see
site-2-site)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] Team meeting Tuesday, June 9th @ 1600 UTC

[Paul Michali]

Will hold weekly meetings to discuss VPNaaS topics.  Please check out the
meeting page for proposed agenda, time, and IRC channel (
https://wiki.openstack.org/wiki/Meetings/VPNaaS).

Also, there is an Etherpad for VPN info, where we hope to collect use-cases
and workflow information to (hopefully) create a shared understanding of
the various VPN proposals. Please contribute to the page (
https://etherpad.openstack.org/p/vpn-flavors).

Regards,

PCM
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Proposing Assaf Muller for the Neutron Core Reviewer Team

[Paul Michali]

+100 Great addition! Congratulations Assaf!

On Thu, Jun 4, 2015 at 11:41 AM Miguel Lavalle mig...@mlavalle.com wrote:

 Congrats! Well deserved

 On Thu, Jun 4, 2015 at 8:50 AM, Assaf Muller amul...@redhat.com wrote:

 Thank you.

 We have a lot of work ahead of us :)


 - Original Message -
  It's a been a week since I proposed this, with no objections. Welcome
 to the
  Neutron core reviewer team as the new QA Lieutenant Assaf!
 
  On Tue, Jun 2, 2015 at 12:35 PM, Maru Newby  ma...@redhat.com  wrote:
 
 
  +1 from me, long overdue!
 
 
   On May 28, 2015, at 9:42 AM, Kyle Mestery  mest...@mestery.com 
 wrote:
  
   Folks, I'd like to propose Assaf Muller to be a member of the Neutron
 core
   reviewer team. Assaf has been a long time contributor in Neutron, and
 he's
   also recently become my testing Lieutenant. His influence and
 knowledge in
   testing will be critical to the team in Liberty and beyond. In
 addition to
   that, he's done some fabulous work for Neutron around L3 HA and DVR.
 Assaf
   has become a trusted member of our community. His review stats place
 him
   in the pack with the rest of the Neutron core reviewers.
  
   I'd also like to take this time to remind everyone that reviewing
 code is a
   responsibility, in Neutron the same as other projects. And core
 reviewers
   are especially beholden to this responsibility. I'd also like to
 point out
   that +1/-1 reviews are very useful, and I encourage everyone to
 continue
   reviewing code even if you are not a core reviewer.
  
   Existing Neutron cores, please vote +1/-1 for the addition of Assaf
 to the
   core reviewer team.
  
   Thanks!
   Kyle
  
   [1] http://stackalytics.com/report/contribution/neutron-group/180
  
 __
   OpenStack Development Mailing List (not for usage questions)
   Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
   http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][bgpvpn] IRC meetings on BGP VPN interconnection API

[Paul Michali]

FYI, the channel is openstack-meeting-3.

On Sun, May 31, 2015 at 10:41 PM Mohammad Hanif mha...@brocade.com wrote:

 Hi Thomas,

 The time reserved for the weekly IRC is 1600 UTC on Tuesdays (
 http://eavesdrop.openstack.org/#Neutron_VPNaaS_Meeting).  The IRC channel
 might not be available on any other time (1500 UTC is taken by the libvirt
 team meeting).

 Thanks,
 —Hanif.



 On 5/29/15, 12:57 PM, Vikram Choudhary viks...@gmail.com wrote:

 Hi Thomos/Mathieu,
 
 Thanks for starting this mail thread. Let's discuss over the IRC as
 suggested by Paul.
 
 Thanks
 Vikram
 
 On 5/29/15, Paul Michali p...@michali.net wrote:
  You can use the VPNaaS IRC channel/time... we don't have much on the
 agenda
  right now, other than discussion VPN flavors for Liberty, in which it
 would
  be good to discuss BGP VPN and Edge VPN.
 
  Regards,
 
  Paul Michali (pc_m)
 
  On Fri, May 29, 2015 at 11:08 AM thomas.mo...@orange.com wrote:
 
  Hi everyone,
 
  As a follow-up to discussions last week on a BGP VPN interconnection
 API
  and the work started with the people already involved, we are going to
  hold IRC meetings to discuss how to progress the different pieces of
  work, in particular on the API itself [1] and its
 implementation+drivers
  [2].
 
  The slot we propose is ** Tuesday 15:00 UTC ** with the first meeting
  next Tuesday (June 2nd).
 
  Note that, based on last week feedback, we submitted the existing
  stackforge project for inclusion in the Neutron big tent earlier this
  week [3].
 
  We will do a proper meeting registration (patch to openstack-infra
  irc-meeting) and send meeting info with wiki and meeting room before
  next Tuesday.
 
  Looking forward to discussing with everyone interested!
 
  -Thomas  Mathieu
 
  [1] currently being discussed at
 https://review.openstack.org/#/c/177740
  [2] https://github.com/stackforge/networking-bgpvpn
  [3] https://review.openstack.org/#/c/186041
 
 
 
 
 
 _
 
  Ce message et ses pieces jointes peuvent contenir des informations
  confidentielles ou privilegiees et ne doivent donc
  pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
  recu ce message par erreur, veuillez le signaler
  a l'expediteur et le detruire ainsi que les pieces jointes. Les
 messages
  electroniques etant susceptibles d'alteration,
  Orange decline toute responsabilite si ce message a ete altere, deforme
  ou
  falsifie. Merci.
 
  This message and its attachments may contain confidential or privileged
  information that may be protected by law;
  they should not be distributed, used or copied without authorisation.
  If you have received this email in error, please notify the sender and
  delete this message and its attachments.
  As emails may be altered, Orange is not liable for messages that have
  been
  modified, changed or falsified.
  Thank you.
 
 
 
 __
  OpenStack Development Mailing List (not for usage questions)
  Unsubscribe:
  openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vnaas] IRC Meeting scheduled for June 2nd...

[Paul Michali]

Since we had a bunch of interest at the Summit in VPN, we'll hold a meeting
at our time-slot, Tuesday, 1600 UTC, openstack-meeting-3, tomorrow on June
2nd.

I updated the agenda. If you have other updates, please add them to the
wiki page:

https://wiki.openstack.org/wiki/Meetings/VPNaaS

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][bgpvpn] IRC meetings on BGP VPN interconnection API

[Paul Michali]

You can use the VPNaaS IRC channel/time... we don't have much on the agenda
right now, other than discussion VPN flavors for Liberty, in which it would
be good to discuss BGP VPN and Edge VPN.

Regards,

Paul Michali (pc_m)

On Fri, May 29, 2015 at 11:08 AM thomas.mo...@orange.com wrote:

 Hi everyone,

 As a follow-up to discussions last week on a BGP VPN interconnection API
 and the work started with the people already involved, we are going to
 hold IRC meetings to discuss how to progress the different pieces of
 work, in particular on the API itself [1] and its implementation+drivers
 [2].

 The slot we propose is ** Tuesday 15:00 UTC ** with the first meeting
 next Tuesday (June 2nd).

 Note that, based on last week feedback, we submitted the existing
 stackforge project for inclusion in the Neutron big tent earlier this
 week [3].

 We will do a proper meeting registration (patch to openstack-infra
 irc-meeting) and send meeting info with wiki and meeting room before
 next Tuesday.

 Looking forward to discussing with everyone interested!

 -Thomas  Mathieu

 [1] currently being discussed at https://review.openstack.org/#/c/177740
 [2] https://github.com/stackforge/networking-bgpvpn
 [3] https://review.openstack.org/#/c/186041




 _

 Ce message et ses pieces jointes peuvent contenir des informations
 confidentielles ou privilegiees et ne doivent donc
 pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
 recu ce message par erreur, veuillez le signaler
 a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
 electroniques etant susceptibles d'alteration,
 Orange decline toute responsabilite si ce message a ete altere, deforme ou
 falsifie. Merci.

 This message and its attachments may contain confidential or privileged
 information that may be protected by law;
 they should not be distributed, used or copied without authorisation.
 If you have received this email in error, please notify the sender and
 delete this message and its attachments.
 As emails may be altered, Orange is not liable for messages that have been
 modified, changed or falsified.
 Thank you.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] Etherpad for Friday Neutron Contributors' session

[Paul Michali]

I created an etherpad to help focus the discussion. We'll try to form a
collective in the morning and start talking through these items.

https://etherpad.openstack.org/p/YVR-neutron-vpnaas

Regards,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] Supporting multiple local subnets for VPN connections

[Paul Michali]

Loy, yeah, that is why I was thinking option B, as user has flexibility to
specify what subnet(s) are used for a connection.  Ideally, we want to
consider different VPN connection types.

Regards,
Paul Michali (pc_m)


On Wed, May 20, 2015 at 5:34 PM loy wolfe loywo...@gmail.com wrote:

 Extra subnets is suitable for attribute of IKE connections, but not
 the whole VPN service. Because customer usually want fine-grained
 control of which local subnet could communicate to remote subnets.

 On Wed, May 20, 2015 at 11:21 PM, Mathieu Rohon mathieu.ro...@gmail.com
 wrote:
  However after thinking about it more deeply, option A might be suitable
 if
  the vpn-service becomes more generic, and usable by other vpn objects
  (ipsec-site-connection, bgpvpn-connection).
  I would become an object that the tenant can update to attach CIDR it
 wants
  to export in its VPN.
  To transform the vpn-service object in a generic vpn description, we
 need to
  remove the mandatory ROUTER attribute, so that we can use it for l2vpn
  either.
 
  Hope we can discuss that on friday morning
 
  On Wed, May 20, 2015 at 5:12 PM, Paul Michali p...@michali.net wrote:
 
  Hi Mathieu,
 
  In Kilo, VPNaaS APIs were no longer marked as experimental. We need to
  understand the implication of changing the API. I'm not sure how much
 VPNaaS
  is being used by OpenStack users at this time, either.  I'm hoping to
 seek
  out answers to this at the summit.
 
  If anyone has suggestions, comments, information on this, please chime
 in.
  I'll likely make a BP for multiple subnets, when I get back from the
 summit.
 
  Lastly, I'm planning on trying to get people interested in VPN to meet
 on
  Friday morning at the summit to discuss all the VPN topics that have
 been
  coming up.
 
  Regards,
 
  Paul Michali (pc_m)
 
  On Wed, May 20, 2015 at 7:54 AM Mathieu Rohon mathieu.ro...@gmail.com
  wrote:
 
  Hi paul,
 
  this is also something that we would like to introduce for BGP/MPLS
 VPNs
  [2].
 
  We choose to allow tenants to attach existing networks ( it might
 evolve
  to subnets) to bgpvpn-connection objects, by updating the
 bgpvpn-connection
  object, which is the equivalent of the ipsec-site-connection object.
 
  So I think that Option B is suitable here.
 
  Concerning backward compatibility, I think VPNaas is still considered
 as
  experimental, am I wrong? Do you have to provide backward compatbility
 in
  this case?
 
  Mathieu
 
  [2] https://review.openstack.org/#/c/177740/
 
  On Wed, May 13, 2015 at 8:59 PM, Paul Michali p...@michali.net wrote:
 
  Hi,
 
  There has been, over the years, some mention about having VPN IPSec
  connections supporting multiple CIDRs for local (left side) private
  networks, in addition to the current support for multiple peer CIDRs
 (right
  side).
 
  I'm raising the question again with these goals:
 
  1) Determine if the reference IPSec implementations support this
  capability
  2) Determine if there is a community desire to enhance VPN to support
  this capability (for all VPN types)
  3) See what would be the best way to handle this (changes to API and
  model)
  4) Identify any consequences of making this change.
 
  Note: My assumption here is something that could be used for any type
 of
  VPN connection - current IPSec, future BGP/MPLS VPN, DM VPN, etc.
 
  Here is some information that was gathered from people on the VPN team
  so far. Please correct any inaccuracies and comment on the items...
 
  (1) It looks like OpenSwan and Libreswan will support this capability.
  StrongSwan will support this with IKEv2. For IKEv1, a Cisco Unity
 plugin
  extensions is needed. I'm not sure what that implies [1].
 
  (2) Do we, as a community, want to enhance VPNaaS to provide this
  capability of N:M subnets for VPN implementations? Putting on my
 vendor hat,
  I can see cases where customers want to be able to only create one
  connection and reference multiple subnets on each end. Is there a
 desire to
  do this and bake it into the reference implementation (thus making it
  available for other implementations)?
 
  (3) Currently, the vpn service API includes the router and subnet ID.
  The IPSec connection command includes the peer CIDR(s). For
 reference, here
  are two of the APIs:
 
  usage: neutron vpn-service-create [-h] [-f
  {html,json,shell,table,value,yaml}]
[-c COLUMN] [--max-width integer]
[--prefix PREFIX]
[--request-format {json,xml}]
[--tenant-id TENANT_ID]
  [--admin-state-down]
[--name NAME] [--description
  DESCRIPTION]
ROUTER SUBNET
 
  usage: neutron ipsec-site-connection-create [-h]
  [-f
  {html,json,shell,table,value,yaml}]
  [-c COLUMN

Re: [openstack-dev] [neutron][vpnaas] Supporting multiple local subnets for VPN connections

[Paul Michali]

Hi Mathieu,

In Kilo, VPNaaS APIs were no longer marked as experimental. We need to
understand the implication of changing the API. I'm not sure how much
VPNaaS is being used by OpenStack users at this time, either.  I'm hoping
to seek out answers to this at the summit.

If anyone has suggestions, comments, information on this, please chime in. I'll
likely make a BP for multiple subnets, when I get back from the summit.

Lastly, I'm planning on trying to get people interested in VPN to meet on
Friday morning at the summit to discuss all the VPN topics that have been
coming up.

Regards,

Paul Michali (pc_m)

On Wed, May 20, 2015 at 7:54 AM Mathieu Rohon mathieu.ro...@gmail.com
wrote:

 Hi paul,

 this is also something that we would like to introduce for BGP/MPLS VPNs
 [2].

 We choose to allow tenants to attach existing networks ( it might evolve
 to subnets) to bgpvpn-connection objects, by updating the bgpvpn-connection
 object, which is the equivalent of the ipsec-site-connection object.

 So I think that Option B is suitable here.

 Concerning backward compatibility, I think VPNaas is still considered as
 experimental, am I wrong? Do you have to provide backward compatbility in
 this case?

 Mathieu

 [2] https://review.openstack.org/#/c/177740/

 On Wed, May 13, 2015 at 8:59 PM, Paul Michali p...@michali.net wrote:

 Hi,

 There has been, over the years, some mention about having VPN IPSec
 connections supporting multiple CIDRs for local (left side) private
 networks, in addition to the current support for multiple peer CIDRs (right
 side).

 I'm raising the question again with these goals:

 1) Determine if the reference IPSec implementations support this
 capability
 2) Determine if there is a community desire to enhance VPN to support
 this capability (for all VPN types)
 3) See what would be the best way to handle this (changes to API and
 model)
 4) Identify any consequences of making this change.

 Note: My assumption here is something that could be used for any type of
 VPN connection - current IPSec, future BGP/MPLS VPN, DM VPN, etc.

 Here is some information that was gathered from people on the VPN team so
 far. Please correct any inaccuracies and comment on the items...

 (1) It looks like OpenSwan and Libreswan will support this capability.
 StrongSwan will support this with IKEv2. For IKEv1, a Cisco Unity plugin
 extensions is needed. I'm not sure what that implies [1].

 (2) Do we, as a community, want to enhance VPNaaS to provide this
 capability of N:M subnets for VPN implementations? Putting on my vendor
 hat, I can see cases where customers want to be able to only create one
 connection and reference multiple subnets on each end. Is there a desire to
 do this and bake it into the reference implementation (thus making it
 available for other implementations)?

 (3) Currently, the vpn service API includes the router and subnet ID. The
 IPSec connection command includes the peer CIDR(s). For reference, here are
 two of the APIs:

 usage: neutron vpn-service-create [-h] [-f
 {html,json,shell,table,value,yaml}]
   [-c COLUMN] [--max-width integer]
   [--prefix PREFIX]
   [--request-format {json,xml}]
   [--tenant-id TENANT_ID]
 [--admin-state-down]
   [--name NAME] [--description
 DESCRIPTION]
   ROUTER SUBNET

 usage: neutron ipsec-site-connection-create [-h]
 [-f
 {html,json,shell,table,value,yaml}]
 [-c COLUMN]
 [--max-width integer]
 [--prefix PREFIX]
 [--request-format {json,xml}]
 [--tenant-id TENANT_ID]
 [--admin-state-down] [--name
 NAME]
 [--description DESCRIPTION]
 [--mtu MTU]
 [--initiator
 {bi-directional,response-only}]
 [--dpd
 action=ACTION,interval=INTERVAL,timeout=TIMEOUT]
 --vpnservice-id VPNSERVICE
 --ikepolicy-id IKEPOLICY
 --ipsecpolicy-id IPSECPOLICY
 --peer-address PEER_ADDRESS
 --peer-id PEER_ID --peer-cidr
 PEER_CIDRS --psk PSK

 I could envision several ways to handle this (feel free to add more).
 Here are some thoughts on this...

 A) Allow multiple subnets for the vpn service API. The implication here
 is that all types of VPN

Re: [openstack-dev] [Neutron] - Neutron social Monday night

[Paul Michali]

Thanks for organizing!

On Sat, May 16, 2015 at 11:26 PM Kevin Benton blak...@gmail.com wrote:

 Hi everybody!

 I got us a place for Monday night at 8pm.

 Here is the link:
 http://www.yelp.com/biz/yaggers-downtown-restaurant-and-sports-bar-vancouver

 Don't forget to show up, they are opening the restaurant just for us!
 On May 12, 2015 2:16 PM, Kevin Benton blak...@gmail.com wrote:

 Hello all,

 I thought it would be fun to have a get-together for Neutron developers
 (and anyone else interested in Neutron development) on Monday night to get
 acquainted before we get into the design sessions on Tuesday.

 I am planning on 8 PM so people will have time to finish the booth crawl.
 The place will depend on how many people want to go. It will probably be a
 sports bar or something similar since that's worked out well the last
 couple of times.

 If you are interested, please fill out this quick RSVP by the end of the
 week so I can get a rough estimate of the number of people I would need to
 make a reservation for. Then I will announce the location over the weekend
 on this same thread.

 http://lockwaittimeoutexceeded.rsvpify.com/

 Cheers,
 Kevin Benton

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] Supporting multiple local subnets for VPN connections

[Paul Michali]

 ignoring that subnet arg, or we could
specify additional CIDRs or subnets in the connection API.

This seems to be backward compatible, but is awkward. If the user wants to
update a connection, and change the subnet that is mentioned in the vpn
service, we have the same issue as option A (granted, it is not handled
well today either).

(4) Obviously, if we change the existing API, there there is the question
of how do we handle backward compatibility? By doing this change, will we
be creating other issues?

I'm not sure why the subnet needed to be specified in the service API,
unless the assumption was that there would only be one subnet for all
connections on the service. Anyone know why? I don't think it is used,
until you make a connection.

Please express your thoughts!

Paul Michali (pc_m)

[1] https://wiki.strongswan.org/projects/strongswan/wiki/ConnSection
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] How should edge services APIs integrate into Neutron?

[Paul Michali]

 be used for providing
 intend VPN use case.


 https://blueprints.launchpad.net/neutron/+spec/intent-based-service-chaining



 Thanks

 Vikram



 *From:* Paul Michali [mailto:p...@michali.net]
 *Sent:* 06 May 2015 01:38
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* [openstack-dev] [neutron] How should edge services APIs
 integrate into Neutron?



 There's been talk in VPN land about new services, like BGP VPN and DM
 VPN. I suspect there are similar things in other Advanced Services. I
 talked to Salvatore today, and he suggested starting a ML thread on this...




 Paul, can you elaborate on any DM VPN proposal, I can't find any
 spec/blueprint about it?


   Can someone elaborate on how we should integrate these API extensions
 into Neutron, both today, and in the future, assuming the proposal that
 Salvatore has is adopted?



 I could see two cases. The first, and simplest, is when a feature has an
 entirely new API that doesn't leverage off of an existing API.



 The other case would be when the feature's API would dovetail into the
 existing service API. For example, one may use the existing vpn_service API
 to create the service, but then create BGP VPN or DM VPN connections for
 that service, instead of the IPSec connections we have today.



 If there are examples already of how to extend an existing API extension
 that would help in understanding how to do this.



 I see that there are RESOURCE_ATTRIBUTE_MAPs with the information on the
 API, and I see that the plugin has a supported_extension_aliases, but
 beyond that, I'm not really sure how it all hooks up, and how to extend an
 existing extension.



 I'm assuming that the python-neutronclient would also need to be updated.





 So... the intent here is to start some discussion on how we do this,
 such that we have some things figured out before the summit and can save
 some time.



 Thanks in advance,



 Paul Michali (pc_m)


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron] How should edge services APIs integrate into Neutron?

[Paul Michali]

There's been talk in VPN land about new services, like BGP VPN and DM VPN.
I suspect there are similar things in other Advanced Services. I talked to
Salvatore today, and he suggested starting a ML thread on this...

Can someone elaborate on how we should integrate these API extensions into
Neutron, both today, and in the future, assuming the proposal that
Salvatore has is adopted?

I could see two cases. The first, and simplest, is when a feature has an
entirely new API that doesn't leverage off of an existing API.

The other case would be when the feature's API would dovetail into the
existing service API. For example, one may use the existing vpn_service API
to create the service, but then create BGP VPN or DM VPN connections for
that service, instead of the IPSec connections we have today.

If there are examples already of how to extend an existing API extension
that would help in understanding how to do this.

I see that there are RESOURCE_ATTRIBUTE_MAPs with the information on the
API, and I see that the plugin has a supported_extension_aliases, but
beyond that, I'm not really sure how it all hooks up, and how to extend an
existing extension.

I'm assuming that the python-neutronclient would also need to be updated.


So... the intent here is to start some discussion on how we do this, such
that we have some things figured out before the summit and can save some
time.

Thanks in advance,

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][vpnaas] On-demand VPNaaS IRC meeting

[Paul Michali]

Since it has been a while, I decided to plan for a VPN meeting in our time
slot of Tuesday, 1600 UTC, for tomorrow the 5th of May.

Please join in!

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] On-demand VPNaaS IRC meeting

[Paul Michali]

Re: https://wiki.openstack.org/wiki/Meetings/VPNaaS


On Mon, May 4, 2015 at 12:03 PM Paul Michali p...@michali.net wrote:

 Since it has been a while, I decided to plan for a VPN meeting in our time
 slot of Tuesday, 1600 UTC, for tomorrow the 5th of May.

 Please join in!

 Paul Michali (pc_m)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] Need guidance - functional tests and rootwrap

[Paul Michali]

So, I think I see where it is failing, but I'm not quite sure how to
resolve.  It appears that the functional test for the namespace module and
strongswan module, will end up invoking neutron-vpn-netns-wrapper script,
which is setup to point to
neutron-vpnaas/services/vpn/common/netns_wrapper.py.

The main() for this, will run execute() in oslo, but with paths mounted.
The issue here, is that it will register three config options, and one is
the path and file for rootwrap.conf. It uses the default of
/etc/neutron/rootwrap.conf. However, when running the functional test, it
sets up the project's .tox/dsvm-functional-sswan/etc/neutron/ with all the
rootwrap info, instead of /etc/neutron/. It looks to be that the attribute
is rootwrap_config, and is in default.

Here are the questions (for me)...

Should I modify the tests to set cfg.CONF.rootwrap_config to the
environment? If so how do I specify the area? It depends on the functional
test target being run, and that could be dsvm-functional or
dsvm-functional-sswan (.tox/test name/etc/neutron).

Or, should I put all the rootwrap config files in /etc/neutron (like they
would be when devstack is stacked)? I could call the deploy_rootwrap.sh
script in Neutron, but use sudo, so that it places it in /etc/neutron. I
would have to create my own version of the neutron
configure_for_func_tests.sh _install_rootwrap_sudoers() method that sets
the destination to use /etc/neutron and not the project's .tox/test-name
area. I think it may also have to add the neutron-vpn-netns-wrapper script
to the sudo list?

Or, should the netns_wrapper.py be changed to look for the config entry in
the [vpnagent] space, so that it can be in vpnagent.ini and we apply the
path there? This changes that (maybe unused) configuration.  I do know that
if I change the default for the rootwrap_config config entry to point to my
project's tox area, the tests pass.

Still don't have my head fully wrapped around this...

Regards,

PCM


On Mon, Apr 27, 2015 at 9:34 AM Paul Michali p...@michali.net wrote:

 I just checked with another run and the log shows that the vpnaas.filters
 file is indeed in the .tox area. So, still stumped as to why the rootwrap
 daemon is looking for the config file in /etc/neutron/rootwrap.d/

 Any ideas?

 PCM


 On Mon, Apr 27, 2015 at 7:21 AM Paul Michali p...@michali.net wrote:

 The latter. If you look at Jenkins results for 168115, it shows that
 there is an invalid config file at /etc/neutron/rootwrap.conf. However, it
 should be accessing
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf.

 The deploy_rootwrap.sh call in tox.ini should place the files in the .tox
 area (it does locally), and the OS_ROOTWRAP_CMD (and
 OS_ROOTWRAP_DAEMON_CMD) are set to these areas as well. It just seems that
 the rootwrap daemon is not using the right file.

 In the latest run, I dumped out environment variables, which show this:

 2015-04-24 15:19:15.499 
 http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_15_499
  | 2015-04-24 15:19:15.466 | OS_ROOTWRAP_DAEMON_CMD=sudo 
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/bin/neutron-rootwrap-daemon
  
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf2015-04-24
  15:19:15.499 
 http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_15_499
  | 2015-04-24 15:19:15.467 | OS_SUDO_TESTING=12015-04-24 15:19:15.499 
 http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_15_499
  | 2015-04-24 15:19:15.469 | OS_ROOTWRAP_CMD=sudo 
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/bin/neutron-rootwrap
  
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf


 I have added a test case in the module that is failing that shows the
 value of cfg.CONF.AGENT.root_helper_daemon. It shows:

 sudo
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/bin/neutron-rootwrap-daemon
 /opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf

 Is that correct?

 When the test cases run, there is this error:

 2015-04-24 15:19:18.536 
 http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_536
  | 2015-04-24 15:19:18.486 | 2015-04-24 15:19:17.986 31628 INFO 
 oslo_rootwrap.client [-] Spawned new rootwrap daemon process with 
 pid=316672015-04-24 15:19:18.536 
 http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_536
  | 2015-04-24 15:19:18.487 | 2015-04-24 15:19:18.470 31628 ERROR 
 neutron.agent.linux.utils [-] 2015-04-24 15:19:18.536 
 http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd

Re: [openstack-dev] [neutron] Need guidance - functional tests and rootwrap

[Paul Michali]

The latter. If you look at Jenkins results for 168115, it shows that there
is an invalid config file at /etc/neutron/rootwrap.conf. However, it should
be accessing
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf.

The deploy_rootwrap.sh call in tox.ini should place the files in the .tox
area (it does locally), and the OS_ROOTWRAP_CMD (and
OS_ROOTWRAP_DAEMON_CMD) are set to these areas as well. It just seems that
the rootwrap daemon is not using the right file.

In the latest run, I dumped out environment variables, which show this:

2015-04-24 15:19:15.499
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_15_499
| 2015-04-24 15:19:15.466 | OS_ROOTWRAP_DAEMON_CMD=sudo
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/bin/neutron-rootwrap-daemon
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf2015-04-24
15:19:15.499 
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_15_499
| 2015-04-24 15:19:15.467 | OS_SUDO_TESTING=12015-04-24 15:19:15.499
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_15_499
| 2015-04-24 15:19:15.469 | OS_ROOTWRAP_CMD=sudo
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/bin/neutron-rootwrap
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf


I have added a test case in the module that is failing that shows the value
of cfg.CONF.AGENT.root_helper_daemon. It shows:

sudo
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/bin/neutron-rootwrap-daemon
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functional-sswan/etc/neutron/rootwrap.conf

Is that correct?

When the test cases run, there is this error:

2015-04-24 15:19:18.536
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_536
| 2015-04-24 15:19:18.486 | 2015-04-24 15:19:17.986 31628 INFO
oslo_rootwrap.client [-] Spawned new rootwrap daemon process with
pid=316672015-04-24 15:19:18.536
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_536
| 2015-04-24 15:19:18.487 | 2015-04-24 15:19:18.470 31628 ERROR
neutron.agent.linux.utils [-] 2015-04-24 15:19:18.536
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_536
| 2015-04-24 15:19:18.489 | Command: ['neutron-vpn-netns-wrapper',
'--mount_paths=/etc:/var/lib/neutron/vpnaas/func-8f1b728c-6eca-4042-9b6b-6ef66ab9352a/etc,/var/run:/var/lib/neutron/vpnaas/func-8f1b728c-6eca-4042-9b6b-6ef66ab9352a/var/run',
'--cmd=nofiltercommand']2015-04-24 15:19:18.537
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_537
| 2015-04-24 15:19:18.490 | Exit code: 222015-04-24 15:19:18.537
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_537
| 2015-04-24 15:19:18.492 | Stdin: 2015-04-24 15:19:18.537
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_537
| 2015-04-24 15:19:18.493 | Stdout: 2015-04-24 15:19:18.537
http://logs.openstack.org/15/168115/29/check/check-neutron-vpnaas-dsvm-functional-sswan/c857bcd/console.html.gz#_2015-04-24_15_19_18_537
| 2015-04-24 15:19:18.495 | Stderr: 2015-04-24 15:19:18.456 31727
ERROR neutron_vpnaas.services.vpn.common.netns_wrapper [-] Incorrect
configuration file: /etc/neutron/rootwrap.conf

Now, there is a vpnaas.filters file that I copy into the rootwrap.d
area, using this command in the tox.ini:

cp {toxinidir}/etc/neutron/rootwrap.d/vpnaas.filters
{envdir}/etc/neutron/rootwrap.d/

The file has the neutorn-vpn-netns-wrapper entry in it. Maybe the copy
is failing?

Regards,

PCM


On Sun, Apr 26, 2015 at 9:39 PM Angus Lees g...@inodes.org wrote:

 The tox.ini entry for dsvm-fullstack sets OS_ROOTWRAP_CMD=sudo
 {envbindir}/neutron-rootwrap {envdir}/etc/neutron/rootwrap.conf (and
 something similar for rootwrap-daemon).

 Is this the answer you were looking for, or are you saying OS_ROOTWRAP_CMD
 doesn't appear to be honoured in your case?

  - Gus

 On Sat, 25 Apr 2015 at 00:45 Ihar Hrachyshka ihrac...@redhat.com wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256

 On 04/24/2015 04:02 PM, Ihar Hrachyshka wrote:
  On 04/24/2015 03:48 PM, Paul Michali wrote:
  Hi, I'm floundering a bit, and could use some guidance on
  this...
 
  For the neutron-vpnaas repo, I am trying to modify the
  functional jobs (dsvm-functional and dsvm-functional-sswan) to
  act in a similar manner to neutron, where devstack

[openstack-dev] [neutron] Need guidance - functional tests and rootwrap

[Paul Michali]

Hi,
I'm floundering a bit, and could use some guidance on this...

For the neutron-vpnaas repo, I am trying to modify the functional jobs
(dsvm-functional and dsvm-functional-sswan) to act in a similar manner to
neutron, where devstack is configured, but no stacking is performed.

I'm trying to do the same thing and have the jobs doing the configuration
only. Side note: there are two jobs, because there are currently two
reference implementations of VPN drivers, each of which require a different
IPSec package installed.

As part of this setup, in tox.ini, the neutron deploy_rootwrap.sh script is
called which places the rootwrap filters and config file in the repo's
.tox/dsvm-functional/etc/neutron/ area (or
./tox/dsvm-functional-sswan/etc/neutron/).

Now, the issue I see is that tests trying to run ip commands, are failing
saying that the config file is invalid:

ERROR neutron_vpnaas.services.vpn.common.netns_wrapper [-] Incorrect
configuration file: /etc/neutron/rootwrap.conf

As you can see, this is trying to access the rootwrap.conf in /etc/neutron
and not the one in
/opt/stack/new/neutron-vpnaas/.tox/dsvm-functioanl-sswan/etc/neutron/.

For Neutron, how is the dsvm-functional job directing the rootwrap daemon
to use the files in the repo's .tox area?

Here is the review in quesstion:  https://review.openstack.org/#/c/168115/

Can anyone advise?

Thanks in advance!

Paul Michali (pc_m)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] FYI:VPNaaS APIs moving from experimental to current

[Paul Michali]

The VPNaaS APIs have been unchanged for over a year and were last marked as
experimental. Bug 1433561 is restoring the VPNaaS API reference
documentation (was lost in a site update). As part of the review [1], there
was some discussion about the experimental status, and so the status was
changed to CURRENT.

If there are any other actions that are needed to make this transition (or
that would prevent it), please comment in the review and reply to this
message. Otherwise, we can mark this API as current.

Regards,

Paul Michali

[1] https://review.openstack.org/#/c/167609/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [documentation]OpenStack API Reference for VPNaaS...

[Paul Michali]

Hi,

I created and am trying to work on
https://bugs.launchpad.net/openstack-api-site/+bug/1433561.

Is there someone who can advise me on how to create the WADL file for
VPNaaS? I see the LBaaS one and can manually try to convert the old VPNaaS
API documentation to a similar format, using a text editor, but before I go
to those lengths, I'm wondering if there is a better way (tools?).

In addition to the raw conversion of the content, there is namespace info
and the like that I don't know how to handle.

Any advice would be greatly appreciated, especially if it makes it less
painful!


PCM (Paul Michali)

IRC pc_m (irc.freenode.com)
Twitter... @pmichali
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][vpnaas] VPNaaS Subteam meetings

[Paul Michali]

Given the votes so far, the proposal is to move the meeting time to 1600
UTC on Tuesday. The channel is openstack-meeting-3 (as the only one
available).

In addition, the meeting will be on-demand, so if you want to have a
meeting, send email to this mailing list, at least 24 hours before the
meeting, and update the agenda on the wiki with the topic(s) you want to
discuss and the date of the meeting being requested.

https://wiki.openstack.org/wiki/Meetings/VPNaaS

Regards,

PCM (Paul Michali)

IRC pc_m (irc.freenode.com)
Twitter... @pmichali


On Mon, Mar 9, 2015 at 3:39 PM, Paul Michali p...@michali.net wrote:

 I guess I'll vote for (D), so that there is the possibility of early (1400
 UTC) and late (2100) on alternating weeks, given we don't have much to
 discuss lately and then changing to (C), if things pick up.

 Let's discuss at Tuesday's meeting (note DST change for US folks), at 1500
 UTC.



 PCM (Paul Michali)

 IRC pc_m (irc.freenode.com)
 Twitter... @pmichali


 On Fri, Mar 6, 2015 at 1:14 AM, Joshua Zhang joshua.zh...@canonical.com
 wrote:

 Hi all,

 I would also vote for (A) with 1500 UTC which is 23:00 in Beijing
 time -:)

 On Fri, Mar 6, 2015 at 1:22 PM, Mohammad Hanif mha...@brocade.com
 wrote:

   Hi all,

  I would also vote for (C) with 1600 UTC or later.  This  will
 hopefully increase more participation from the Pacific time zone.

  Thanks,
 —Hanif.

   From: Mathieu Rohon
 Reply-To: OpenStack Development Mailing List (not for usage questions)
 Date: Thursday, March 5, 2015 at 1:52 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [neutron][vpnaas] VPNaaS Subteam meetings

 Hi,

  I'm fine with C) and 1600 UTC would be more adapted for EU time Zone :)

  However, I Agree that neutron-vpnaas meetings was mainly focus on
 maintaining the current IPSec implementation, by managing the slip out,
 adding StrongSwan support and adding functional tests.
  Maybe we will get a broader audience once we will speak about adding
 new use cases such as edge-vpn.
  Edge-vpn use cases overlap with the Telco WG VPN use case [1]. May be
 those edge-vpn discussions should occur during the Telco WG meeting?

 [1]
 https://wiki.openstack.org/wiki/TelcoWorkingGroup/UseCases#VPN_Instantiation

 On Thu, Mar 5, 2015 at 3:02 AM, Sridhar Ramaswamy sric...@gmail.com
 wrote:

 Hi Paul.

  I'd vote for (C) and a slightly later time-slot on Tuesdays - 1630
 UTC (or later).

  The meetings so far was indeed quite useful. I guess the current busy
 Kilo cycle is also contributing to the low turnout. As we pick up things
 going forward this forum will be quite useful to discuss edge-vpn and,
 perhaps, other vpn variants.

  - Sridhar

  On Tue, Mar 3, 2015 at 3:38 AM, Paul Michali p...@michali.net wrote:

  Hi all! The email, that I sent on 2/24 didn't make it to the mailing
 list (no wonder I didn't get responses!). I think I had an issue with my
 email address used - sorry for the confusion!

  So, I'll hold the meeting today (1500 UTC meeting-4, if it is still
 available), and we can discuss this...


  We've been having very low turnout for meetings for the past several
 weeks, so I'd like to ask those in the community interested in VPNaaS, 
 what
 the preference would be regarding meetings...

  A) hold at the same day/time, but only on-demand.
 B) hold at a different day/time.
 C) hold at a different day/time, but only on-demand.
 D) hold as a on-demand topic in main Neutron meeting.

  Please vote your interest, and provide desired day/time, if you pick
 B or C. The fallback will be (D), if there's not much interest anymore for
 meeting, or we can't seem to come to a consensus (or super-majority :)

  Regards,

  PCM

  Twitter: @pmichali
 TEXT: 6032894458
 PCM (Paul Michali)

  IRC pc_m (irc.freenode.com)
 Twitter... @pmichali



 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Best Regards
 Zhang Hua(张华）
 Software Engineer | Canonical
 IRC:  zhhuabj

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe