Re: [Openstack] Networking issues in Essex
I've only deployed openstack for the first time a couple weeks ago, but FWIW... I had similar symptoms on my Essex test deployment (on Ubuntu 12.04) turned out my problem was taht while the br100 bridge was up and configured the underlying eth1 physical interface was down so the bits went nowhere. 'ifconfig eth1 up' fixed all, followed ofcoures by fixing in /etc/network/interfaces as well so this happens on it's own in future. -Jon On Thu, Jul 12, 2012 at 02:56:57PM +1000, Michael Chapman wrote: :Hi all, I'm hoping I could get some assistance figuring out my networking :problems with a small Essex test cluster. I have a small Diablo cluster :running without any problems but have hit a wall in deploying Essex. : :I can launch VMs without issue and access them from the compute host, but :from there I can't access anything except the host, DNS services, and other :VMs. : :I have separate machines running keystone, glance, postgresql, rabbit-mq :and nova-api. They're all on the .os domain with 172.22.1.X IPs : :I have one machine running nova-compute, nova-network and nova-api, with a :public address 192.43.239.175 and also an IP on the 172.22.1.X subnet in :the .os domain. It has the following nova/conf: : :--dhcpbridge_flagfile=/etc/nova/nova.conf :--dhcpbridge=/usr/bin/nova-dhcpbridge :--logdir=/var/log/nova :--state_path=/var/lib/nova :--lock_path=/var/lock/nova :--force_dhcp_release :--iscsi_helper=tgtadm :--libvirt_use_virtio_for_bridges :--connection_type=libvirt :--root_helper=sudo nova-rootwrap :--verbose :--ec2_private_dns_show_ip : :--network_manager=nova.network.manager.FlatDHCPManager :--rabbit_host=os-amqp.os :--sql_connection=postgresql://[user]:[password]@os-sql.os/nova :--image_service=nova.image.glance.GlanceImageService :--glance_api_servers=os-glance.os:9292 :--auth_strategy=keystone :--scheduler_driver=nova.scheduler.simple.SimpleScheduler :--keystone_ec2_url=http://os-key.os:5000/v2.0/ec2tokens : :--api_paste_config=/etc/nova/api-paste.ini : :--my_ip=192.43.239.175 :--flat_interface=eth0 :--public_interface=eth1 :--multi_host=True :--routing_source_ip=192.43.239.175 :--network_host=192.43.239.175 : :--dmz_cidr=$my_ip : :--ec2_host=192.43.239.175 :--ec2_dmz_host=192.43.239.175 : :I believe I'm seeing a natting issue of some sort - my VMs cannot ping :external IPs, though DNS seems to work. :ubuntu@monday:~$ ping www.google.com :PING www.l.google.com (74.125.237.148) 56(84) bytes of data. :AWKWARD SILENCE : :When I do a tcpdump on the compute host things seem fairly normal, even :though nothing is getting back to the VM : :root@ncios1:~# tcpdump icmp -i br100 :tcpdump: verbose output suppressed, use -v or -vv for full protocol decode :listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes :14:35:28.046416 IP 10.0.0.8 syd01s13-in-f20.1e100.net: ICMP echo request, :id 5002, seq 9, length 64 :14:35:28.051477 IP syd01s13-in-f20.1e100.net 10.0.0.8: ICMP echo reply, :id 5002, seq 9, length 64 :14:35:29.054505 IP 10.0.0.8 syd01s13-in-f20.1e100.net: ICMP echo request, :id 5002, seq 10, length 64 :14:35:29.059556 IP syd01s13-in-f20.1e100.net 10.0.0.8: ICMP echo reply, :id 5002, seq 10, length 64 : :I've pored over the iptables nat rules and can't see anything amiss apart :from the masquerades that are automatically added: (I've cut out some empty :chains for brevity) : :root@ncios1:~# iptables -L -t nat -v :Chain PREROUTING (policy ACCEPT 22 packets, 2153 bytes) : pkts bytes target prot opt in out source :destination : 22 2153 nova-network-PREROUTING all -- anyany anywhere : anywhere : 22 2153 nova-compute-PREROUTING all -- anyany anywhere : anywhere : 22 2153 nova-api-PREROUTING all -- anyany anywhere : anywhere : :Chain INPUT (policy ACCEPT 12 packets, 1573 bytes) : pkts bytes target prot opt in out source :destination : :Chain OUTPUT (policy ACCEPT 31 packets, 2021 bytes) : pkts bytes target prot opt in out source :destination : 31 2021 nova-network-OUTPUT all -- anyany anywhere : anywhere : 31 2021 nova-compute-OUTPUT all -- anyany anywhere : anywhere : 31 2021 nova-api-OUTPUT all -- anyany anywhere :anywhere : :Chain POSTROUTING (policy ACCEPT 30 packets, 1961 bytes) : pkts bytes target prot opt in out source :destination : 31 2021 nova-network-POSTROUTING all -- anyany anywhere :anywhere : 30 1961 nova-compute-POSTROUTING all -- anyany anywhere :anywhere : 30 1961 nova-api-POSTROUTING all -- anyany anywhere :anywhere : 30 1961 nova-postrouting-bottom all -- anyany anywhere : anywhere :0 0 MASQUERADE tcp -- anyany 192.168.122.0/24! :192.168.122.0/24 masq ports: 1024-65535 :0 0 MASQUERADE udp -- anyany 192.168.122.0/24! :192.168.122.0/24 masq ports: 1024-65535 :0 0 MASQUERADE all --
Re: [Openstack] Networking issues in Essex
Thanks for the tip, unfortunately the interfaces are already up. - Michael On Thu, Jul 12, 2012 at 10:15 PM, Jonathan Proulx j...@csail.mit.edu wrote: I've only deployed openstack for the first time a couple weeks ago, but FWIW... I had similar symptoms on my Essex test deployment (on Ubuntu 12.04) turned out my problem was taht while the br100 bridge was up and configured the underlying eth1 physical interface was down so the bits went nowhere. 'ifconfig eth1 up' fixed all, followed ofcoures by fixing in /etc/network/interfaces as well so this happens on it's own in future. -Jon On Thu, Jul 12, 2012 at 02:56:57PM +1000, Michael Chapman wrote: :Hi all, I'm hoping I could get some assistance figuring out my networking :problems with a small Essex test cluster. I have a small Diablo cluster :running without any problems but have hit a wall in deploying Essex. : :I can launch VMs without issue and access them from the compute host, but :from there I can't access anything except the host, DNS services, and other :VMs. : :I have separate machines running keystone, glance, postgresql, rabbit-mq :and nova-api. They're all on the .os domain with 172.22.1.X IPs : :I have one machine running nova-compute, nova-network and nova-api, with a :public address 192.43.239.175 and also an IP on the 172.22.1.X subnet in :the .os domain. It has the following nova/conf: : :--dhcpbridge_flagfile=/etc/nova/nova.conf :--dhcpbridge=/usr/bin/nova-dhcpbridge :--logdir=/var/log/nova :--state_path=/var/lib/nova :--lock_path=/var/lock/nova :--force_dhcp_release :--iscsi_helper=tgtadm :--libvirt_use_virtio_for_bridges :--connection_type=libvirt :--root_helper=sudo nova-rootwrap :--verbose :--ec2_private_dns_show_ip : :--network_manager=nova.network.manager.FlatDHCPManager :--rabbit_host=os-amqp.os :--sql_connection=postgresql://[user]:[password]@os-sql.os/nova :--image_service=nova.image.glance.GlanceImageService :--glance_api_servers=os-glance.os:9292 :--auth_strategy=keystone :--scheduler_driver=nova.scheduler.simple.SimpleScheduler :--keystone_ec2_url=http://os-key.os:5000/v2.0/ec2tokens : :--api_paste_config=/etc/nova/api-paste.ini : :--my_ip=192.43.239.175 :--flat_interface=eth0 :--public_interface=eth1 :--multi_host=True :--routing_source_ip=192.43.239.175 :--network_host=192.43.239.175 : :--dmz_cidr=$my_ip : :--ec2_host=192.43.239.175 :--ec2_dmz_host=192.43.239.175 : :I believe I'm seeing a natting issue of some sort - my VMs cannot ping :external IPs, though DNS seems to work. :ubuntu@monday:~$ ping www.google.com :PING www.l.google.com (74.125.237.148) 56(84) bytes of data. :AWKWARD SILENCE : :When I do a tcpdump on the compute host things seem fairly normal, even :though nothing is getting back to the VM : :root@ncios1:~# tcpdump icmp -i br100 :tcpdump: verbose output suppressed, use -v or -vv for full protocol decode :listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes :14:35:28.046416 IP 10.0.0.8 syd01s13-in-f20.1e100.net: ICMP echo request, :id 5002, seq 9, length 64 :14:35:28.051477 IP syd01s13-in-f20.1e100.net 10.0.0.8: ICMP echo reply, :id 5002, seq 9, length 64 :14:35:29.054505 IP 10.0.0.8 syd01s13-in-f20.1e100.net: ICMP echo request, :id 5002, seq 10, length 64 :14:35:29.059556 IP syd01s13-in-f20.1e100.net 10.0.0.8: ICMP echo reply, :id 5002, seq 10, length 64 : :I've pored over the iptables nat rules and can't see anything amiss apart :from the masquerades that are automatically added: (I've cut out some empty :chains for brevity) : :root@ncios1:~# iptables -L -t nat -v :Chain PREROUTING (policy ACCEPT 22 packets, 2153 bytes) : pkts bytes target prot opt in out source :destination : 22 2153 nova-network-PREROUTING all -- anyany anywhere : anywhere : 22 2153 nova-compute-PREROUTING all -- anyany anywhere : anywhere : 22 2153 nova-api-PREROUTING all -- anyany anywhere : anywhere : :Chain INPUT (policy ACCEPT 12 packets, 1573 bytes) : pkts bytes target prot opt in out source :destination : :Chain OUTPUT (policy ACCEPT 31 packets, 2021 bytes) : pkts bytes target prot opt in out source :destination : 31 2021 nova-network-OUTPUT all -- anyany anywhere : anywhere : 31 2021 nova-compute-OUTPUT all -- anyany anywhere : anywhere : 31 2021 nova-api-OUTPUT all -- anyany anywhere :anywhere : :Chain POSTROUTING (policy ACCEPT 30 packets, 1961 bytes) : pkts bytes target prot opt in out source :destination : 31 2021 nova-network-POSTROUTING all -- anyany anywhere :anywhere : 30 1961 nova-compute-POSTROUTING all -- anyany anywhere :anywhere : 30 1961 nova-api-POSTROUTING all -- anyany anywhere :anywhere : 30 1961 nova-postrouting-bottom all -- anyany anywhere
[Openstack] Networking issues in Essex
Hi all, I'm hoping I could get some assistance figuring out my networking problems with a small Essex test cluster. I have a small Diablo cluster running without any problems but have hit a wall in deploying Essex. I can launch VMs without issue and access them from the compute host, but from there I can't access anything except the host, DNS services, and other VMs. I have separate machines running keystone, glance, postgresql, rabbit-mq and nova-api. They're all on the .os domain with 172.22.1.X IPs I have one machine running nova-compute, nova-network and nova-api, with a public address 192.43.239.175 and also an IP on the 172.22.1.X subnet in the .os domain. It has the following nova/conf: --dhcpbridge_flagfile=/etc/nova/nova.conf --dhcpbridge=/usr/bin/nova-dhcpbridge --logdir=/var/log/nova --state_path=/var/lib/nova --lock_path=/var/lock/nova --force_dhcp_release --iscsi_helper=tgtadm --libvirt_use_virtio_for_bridges --connection_type=libvirt --root_helper=sudo nova-rootwrap --verbose --ec2_private_dns_show_ip --network_manager=nova.network.manager.FlatDHCPManager --rabbit_host=os-amqp.os --sql_connection=postgresql://[user]:[password]@os-sql.os/nova --image_service=nova.image.glance.GlanceImageService --glance_api_servers=os-glance.os:9292 --auth_strategy=keystone --scheduler_driver=nova.scheduler.simple.SimpleScheduler --keystone_ec2_url=http://os-key.os:5000/v2.0/ec2tokens --api_paste_config=/etc/nova/api-paste.ini --my_ip=192.43.239.175 --flat_interface=eth0 --public_interface=eth1 --multi_host=True --routing_source_ip=192.43.239.175 --network_host=192.43.239.175 --dmz_cidr=$my_ip --ec2_host=192.43.239.175 --ec2_dmz_host=192.43.239.175 I believe I'm seeing a natting issue of some sort - my VMs cannot ping external IPs, though DNS seems to work. ubuntu@monday:~$ ping www.google.com PING www.l.google.com (74.125.237.148) 56(84) bytes of data. AWKWARD SILENCE When I do a tcpdump on the compute host things seem fairly normal, even though nothing is getting back to the VM root@ncios1:~# tcpdump icmp -i br100 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on br100, link-type EN10MB (Ethernet), capture size 65535 bytes 14:35:28.046416 IP 10.0.0.8 syd01s13-in-f20.1e100.net: ICMP echo request, id 5002, seq 9, length 64 14:35:28.051477 IP syd01s13-in-f20.1e100.net 10.0.0.8: ICMP echo reply, id 5002, seq 9, length 64 14:35:29.054505 IP 10.0.0.8 syd01s13-in-f20.1e100.net: ICMP echo request, id 5002, seq 10, length 64 14:35:29.059556 IP syd01s13-in-f20.1e100.net 10.0.0.8: ICMP echo reply, id 5002, seq 10, length 64 I've pored over the iptables nat rules and can't see anything amiss apart from the masquerades that are automatically added: (I've cut out some empty chains for brevity) root@ncios1:~# iptables -L -t nat -v Chain PREROUTING (policy ACCEPT 22 packets, 2153 bytes) pkts bytes target prot opt in out source destination 22 2153 nova-network-PREROUTING all -- anyany anywhere anywhere 22 2153 nova-compute-PREROUTING all -- anyany anywhere anywhere 22 2153 nova-api-PREROUTING all -- anyany anywhere anywhere Chain INPUT (policy ACCEPT 12 packets, 1573 bytes) pkts bytes target prot opt in out source destination Chain OUTPUT (policy ACCEPT 31 packets, 2021 bytes) pkts bytes target prot opt in out source destination 31 2021 nova-network-OUTPUT all -- anyany anywhere anywhere 31 2021 nova-compute-OUTPUT all -- anyany anywhere anywhere 31 2021 nova-api-OUTPUT all -- anyany anywhere anywhere Chain POSTROUTING (policy ACCEPT 30 packets, 1961 bytes) pkts bytes target prot opt in out source destination 31 2021 nova-network-POSTROUTING all -- anyany anywhere anywhere 30 1961 nova-compute-POSTROUTING all -- anyany anywhere anywhere 30 1961 nova-api-POSTROUTING all -- anyany anywhere anywhere 30 1961 nova-postrouting-bottom all -- anyany anywhere anywhere 0 0 MASQUERADE tcp -- anyany 192.168.122.0/24! 192.168.122.0/24 masq ports: 1024-65535 0 0 MASQUERADE udp -- anyany 192.168.122.0/24! 192.168.122.0/24 masq ports: 1024-65535 0 0 MASQUERADE all -- anyany 192.168.122.0/24! 192.168.122.0/24 Chain nova-api-snat (1 references) pkts bytes target prot opt in out source destination 30 1961 nova-api-float-snat all -- anyany anywhere anywhere Chain nova-compute-snat (1 references) pkts bytes target prot opt in out source destination 30 1961 nova-compute-float-snat all -- anyany anywhere anywhere Chain nova-network-POSTROUTING (1 references) pkts bytes target prot opt in out source destination 0 0 ACCEPT all -- anyany 10.0.0.0/8 nri5.nci.org.au 0
