To summarize what I’m trying to do with option (A)…

I want to test VPN in DevStack by setting up two private networks, two routers, 
and a shared public network. The VMs created in the private networks should be 
able to access the public network, but not the other private network (e.g. VM 
on private-A subnet can ping public interface of router2 on private-B subnet)


Do I need to create the second router and private network using a different 
Do I need to setup security group rules to allow the access desired?
What local.conf settings do I need for this setup (beyond what I have below)?

I’ve been trying so many different combinations (using both single and two 
devstack setups, trying provider net, using single/multiple tenants) and have 
been getting a variety of different results, from unexpected ping results, to 
VMs stuck in power state PAUSED, that I’m lost as to how to set this up. I 
think I’m hung up on the security group rules and how to setup the bridges.

What I’d like to do, is just focus on this option (A) - using a single devstack 
with multiple routers, and see if that works. If not, I can focus on option 
(B), using two devstacks/hosts.

Since I’m pretty much out of ideas on how to fix this for now, I’m going to try 
to see if I can get on a bare metal setup, which has worked in the past.

Any ideas? I’d like to verify VPNaaS reference implementation with the new repo 
changes. Been spending some time over the holiday vacation playing with this, 
with no joy. :(

PCM (Paul Michali)

MAIL …..….<>
IRC ……..… pc_m (<>)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83

On Dec 31, 2014, at 2:35 PM, Paul Michali (pcm) 
<<>> wrote:

Just more data…

I keep consistently seeing that on private subnet, the VM can only access 
router (as expected), but on privateB subnet, the VM can access the private I/F 
of router1 on private subnet. From the router’s namespace, I cannot ping the 
local VM (why not?). Oddly, I can ping router1’s private IP from router2 

I tried these commands to create security group rules (are they wrong?):

# There are two default groups created by DevStack
group=`neutron security-group-list | grep default | cut -f 2 -d' ' | head -1`
neutron security-group-rule-create --protocol ICMP $group
neutron security-group-rule-create --protocol tcp --port-range-min 22 
--port-range-max 22 $group
group=`neutron security-group-list | grep default | cut -f 2 -d' ' | tail -1`
neutron security-group-rule-create --protocol ICMP $group
neutron security-group-rule-create --protocol tcp --port-range-min 22 
--port-range-max 22 $group

The only change that happens, when I do these commands, is that the VM in 
privateB subnet can now ping the VM from private subnet, but not vice versa. 
From router1 namespace, it can then access local VMs. From router2 namespace it 
can access local VMs and VMs in private subnet (all access).

It seems like I have some issue with security groups, and I need to square that 
away, before I can test VPN out.

Am I creating the security group rules correctly?
My goal is that the private nets can access the public net, but not each other 
(until VPN connection is established).

Lastly, in this latest try, I set OVS_PHYSICAL_BRIDGE=br-ex. In earlier runs 
w/o that, there were QVO interfaces, but no QVB or QBR interfaces at all. It 
didn’t seem to change connectivity, however.


PCM (Paul Michali)

MAIL …..….<>
IRC ……..… pc_m (<>)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83

On Dec 31, 2014, at 10:33 AM, Paul Michali (pcm) 
<<>> wrote:

I’ve been playing a bit with trying to get VPNaaS working post-repo split, and 
haven’t been successful. I’m trying it a few ways with DevStack, and I’m not 
sure whether I have a config error, setup issue, or there is something due to 
the split.

In the past (and it’s been a few months since I verified VPN operation), I used 
two bare metal machines and an external switch connecting them. With a DevStack 
cloud running on each. That configuration is currently setup for a vendor VPN 
solution, so I wanted to try different methods to test the reference VPN 
implementation. I’ve got two ideas to do this:

A) Run DevStack and create two routers with a shared “public” network, and two 
private networks, setting up a VPN connection between the private nets.
B) Run two DevStack instances (on two VMs) and try to setup a provider network 
between them.

I’m starting with A (though I did try B quickly, but it didn’t work), and I 
spun up the stack, added a second router (all under the same tenant), created 
another private network, and booted a Cirros VM in each private net.

Before even trying VPN, I checked pings. From the first private net VM 
(, I could ping on the pubic net, including the public IP of the 
second private net’s public interface for its router. I cannot ping the VM from 
the host. That seems all expected to me.

What seems wrong is the other VM (this is on the post stack net I created). 
Like the other VM, I can ping public net IPs. However, I can also ping the 
private net address of the first network’s router (! Shouldn’t that 
have failed (at least that was what I was expecting)? I can’t ping the VM on 
that side though. Another curiosity is that the VM got the second IP on the 
subnet (, unlike the other private net, where DHCP and a compute probe 
got the 2nd and 3rd IPs. There is DHCP enabled on this private network.

When I tried VPN, both connections show as DOWN, and all I see are phase 1 
ident packets. I cannot ping from VM to VM. I don’t see any logging for the 
OpenSwan processes, so not to sure how to debug. Maybe I can try some ipsec 
show command?

I’m not too sure what is wrong with this setup.

For a comparison, I decided to do the same thing, using stable/juno. So, I 
fired up a VM and cloned DevStack with stable/juno and stacked. This time, 
things are even worse! When I try to boot a VM, and then check the status, the 
VM is in PAUSED power state. I can’t seem to unpause (nor do I know why it is 
in this state). Verified this with both Cirros 3.3, 3.2, and Ubuntu cloud 

| Property                             | Value                                  
| OS-DCF:diskConfig                    | MANUAL                                 
| OS-EXT-AZ:availability_zone          | nova                                   
| OS-EXT-SRV-ATTR:host                 | juno                                   
| OS-EXT-SRV-ATTR:hypervisor_hostname  | juno                                   
| OS-EXT-SRV-ATTR:instance_name        | instance-00000001                      
| OS-EXT-STS:power_state               | 3                                      
| OS-EXT-STS:task_state                | -                                      
| OS-EXT-STS:vm_state                  | active                                 
| OS-SRV-USG:launched_at               | 2014-12-31T15:15:33.000000             
| OS-SRV-USG:terminated_at             | -                                      
| accessIPv4                           |                                        
| accessIPv6                           |                                        
| config_drive                         |                                        
| created                              | 2014-12-31T15:15:24Z                   
| flavor                               | m1.tiny (1)                            
| hostId                               | 
5b0c48250ccc0ac3fca8a821e29e4b154ec0b101f9cc0a0b27071a3f       |
| id                                   | ec5c8d70-ae80-4cc3-a5bb-b68019170dd6   
| image                                | cirros-0.3.3-x86_64-uec 
(797e4dee-8c03-497f-8dac-a44b9351dfa3) |
| key_name                             | -                                      
| metadata                             | {}                                     
| name                                 | peter                                  
| os-extended-volumes:volumes_attached | []                                     
| private network                      |                               
| progress                             | 0                                      
| security_groups                      | default                                
| status                               | ACTIVE                                 
| tenant_id                            | 7afb5bc1d88d462c8d57178437d3c277       
| updated                              | 2014-12-31T15:15:34Z                   
| user_id                              | 4ff18bdbeb4d436ea4ff1bcd29e269a9       

| ID                                   | Name  | Status | Task State | Power 
State | Networks         |
| ec5c8d70-ae80-4cc3-a5bb-b68019170dd6 | peter | ACTIVE | -          | Paused   
   | private= |

Any ideas why the VM won’t start up correctly? I didn’t see anything on a 
google search.

For reference, here is my local.conf currently:


disable_service n-net
enable_service q-svc
enable_service q-agt
enable_service q-dhcp
enable_service q-l3
enable_service q-meta
enable_service neutron
enable_service q-vpn


# Q_USE_SECGROUP=True # was False

# VIRT_DRIVER=libvirt





Originally, I had floating pool lines and net names, but even with all these 
commented out, I have the same issue with the VM (didn’t think they were 

For this stable/juno, Devstack is using commit 817e9b6, and Neutron is using 

I’ll try to play with option B some more as well, though I need to figure out 
how to setup the provider network correctly. If I can get time, I’ll 
reconfigure the bare metal setup I have in the lab to try stable/juno and then 
kilo reference VPN as well.

If anyone has done this with a VM (either one or two), using juno or kilo, 
please pass along your local.conf, so I can compare.

PCM (Paul Michali)

MAIL …..….<>
IRC ……..… pc_m (<>)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83

OpenStack-dev mailing list<>

OpenStack-dev mailing list<>

OpenStack-dev mailing list

Reply via email to