Greetings,
I am trying to use Ubuntu Server 12.04 in a VMware vm as a test-bed for hosting
several lxc containers while I work some configuration kinks out of them.
Eventually I plan to host them directly on physical hardware.
Anyway, I am having trouble convincing the lxc guests to talk to the network
outside the box hosting the vm hosting the container.
I am wondering if anyone has had any experience making such a configuration
work.
I'll try to fill in the relevant details.
The host box has the following network devices:
eth0 Link encap:Ethernet HWaddr 00:90:f5:b5:eb:e6 UP BROADCAST
RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0
overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B)
TX bytes:0 (0.0 B) Interrupt:67
lo Link encap:Local Loopback inet addr:127.0.0.1
Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK
RUNNING MTU:16436 Metric:1 RX packets:460 errors:0 dropped:0
overruns:0 frame:0 TX packets:460 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:0 RX bytes:35888 (35.8 KB)
TX bytes:35888 (35.8 KB)
vmnet1 Link encap:Ethernet HWaddr 00:50:56:c0:00:01 inet
addr:172.16.67.1 Bcast:172.16.67.255 Mask:255.255.255.0 inet6 addr:
fe80::250:56ff:fec0:1/64 Scope:Link UP BROADCAST RUNNING MULTICAST
MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:79 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
vmnet8 Link encap:Ethernet HWaddr 00:50:56:c0:00:08 inet
addr:192.168.59.1 Bcast:192.168.59.255 Mask:255.255.255.0 inet6
addr: fe80::250:56ff:fec0:8/64 Scope:Link UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1 RX packets:1 errors:0 dropped:0
overruns:0 frame:0 TX packets:79 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:1000 RX bytes:0 (0.0 B)
TX bytes:0 (0.0 B)
wlan0 Link encap:Ethernet HWaddr e0:91:53:35:39:97 inet
addr:192.168.0.100 Bcast:192.168.0.255 Mask:255.255.255.0 inet6
addr: fe80::e291:53ff:fe35:3997/64 Scope:Link UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1 RX packets:11345 errors:0 dropped:0
overruns:0 frame:0 TX packets:10362 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:1000 RX bytes:6413336 (6.4
MB) TX bytes:1866889 (1.8 MB) Interrupt:18
Memory:ffffc90003320000-ffffc90003320100
wlan0 eventually leads to a wireless router and the internet.
The routing table on the host is:
Kernel IP routing tableDestination Gateway Genmask Flags
Metric Ref Use Iface192.168.0.0 0.0.0.0 255.255.255.0 U 2
0 0 wlan0172.16.67.0 0.0.0.0 255.255.255.0 U 0
0 0 vmnet1192.168.59.0 0.0.0.0 255.255.255.0 U 0
0 0 vmnet8169.254.0.0 0.0.0.0 255.255.0.0 U 1000 0
0 wlan00.0.0.0 192.168.0.1 0.0.0.0 UG 0 0
0 wlan0
Next, the vm's network devices:
br0 Link encap:Ethernet HWaddr 2a:40:9c:29:c8:ac inet
addr:192.168.2.254 Bcast:192.168.2.255 Mask:255.255.255.0 inet6
addr: fe80::2c4c:89ff:fe79:d51f/64 Scope:Link UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1 RX packets:755 errors:0 dropped:0
overruns:0 frame:0 TX packets:809 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:0 RX bytes:120905 (120.9
KB) TX bytes:83379 (83.3 KB)
eth0 Link encap:Ethernet HWaddr 00:0c:29:17:22:0a inet
addr:192.168.59.128 Bcast:192.168.59.255 Mask:255.255.255.0 inet6
addr: fe80::20c:29ff:fe17:220a/64 Scope:Link UP BROADCAST RUNNING
MULTICAST MTU:1500 Metric:1 RX packets:129267 errors:0 dropped:0
overruns:0 frame:0 TX packets:75502 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:1000 RX bytes:189241404
(189.2 MB) TX bytes:4219414 (4.2 MB)
lo Link encap:Local Loopback inet addr:127.0.0.1
Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK
RUNNING MTU:16436 Metric:1 RX packets:69 errors:0 dropped:0
overruns:0 frame:0 TX packets:69 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:0 RX bytes:7048 (7.0 KB)
TX bytes:7048 (7.0 KB)
veth5c5qSm Link encap:Ethernet HWaddr 2a:40:9c:29:c8:ac inet6 addr:
fe80::2840:9cff:fe29:c8ac/64 Scope:Link UP BROADCAST RUNNING PROMISC
MULTICAST MTU:1500 Metric:1 RX packets:458 errors:0 dropped:0
overruns:0 frame:0 TX packets:592 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:1000 RX bytes:63277 (63.2
KB) TX bytes:59013 (59.0 KB)
br0 is the bridge device intended to join all the lxc containers and eth0
connects to the host's vmnet8
The routing table on the vm is:
Kernel IP routing tableDestination Gateway Genmask Flags
Metric Ref Use Iface0.0.0.0 192.168.59.2 0.0.0.0 UG
100 0 0 eth0192.168.2.0 0.0.0.0 255.255.255.0 U 0
0 0 br0192.168.59.0 0.0.0.0 255.255.255.0 U 0 0
0 eth0
This routing table confuses me because the default gateway is the first entry
which seems strange. That being said, I can reach the internet from the vm
without any problem.I've also got iptables configured to do the masquerading
for the br0 device:
# Generated by iptables-save v1.4.12 on Thu Sep 6 21:23:39 2012*nat:PREROUTING
ACCEPT [290:47394]:INPUT ACCEPT [13:2468]:OUTPUT ACCEPT
[1719:115881]:POSTROUTING ACCEPT [28:2088]:ForwardedPorts - [0:0]-A PREROUTING
-j ForwardedPorts-A POSTROUTING -m mark --mark 0x9 -j MASQUERADE-A POSTROUTING
-o eth0 -j MASQUERADECOMMIT# Completed on Thu Sep 6 21:23:39 2012# Generated
by iptables-save v1.4.12 on Thu Sep 6 21:23:39 2012*mangle:PREROUTING ACCEPT
[3:354]:INPUT ACCEPT [3:354]:FORWARD ACCEPT [0:0]:OUTPUT ACCEPT
[0:0]:POSTROUTING ACCEPT [0:0]:Masquerade - [0:0]-A PREROUTING -j Masquerade-A
Masquerade -i br0 -j MARK --set-xmark 0x9/0xffffffffCOMMIT# Completed on Thu
Sep 6 21:23:39 2012# Generated by iptables-save v1.4.12 on Thu Sep 6 21:23:39
2012*filter:INPUT ACCEPT [0:0]:FORWARD ACCEPT [0:0]:OUTPUT ACCEPT
[77183:3310031]:Blocked - [0:0]:Firewall - [0:0]:RoutedDevices -
[0:0]:TrustedDevices - [0:0]:TrustedPorts - [0:0]-A INPUT -j Blocked-A INPUT -j
Firewall-A FORWARD -j Blocked-A FORWARD -j RoutedDevices-A FORWARD -j
Firewall-A Firewall -j TrustedDevices-A Firewall -p icmp -m icmp --icmp-type
any -j ACCEPT-A Firewall -p esp -j ACCEPT-A Firewall -p ah -j ACCEPT-A Firewall
-d 224.0.0.251/32 -p udp -m udp --dport 5353 -j ACCEPT-A Firewall -p udp -m udp
--dport 631 -j ACCEPT-A Firewall -p tcp -m tcp --dport 631 -j ACCEPT-A Firewall
-m state --state RELATED,ESTABLISHED -j ACCEPT-A Firewall -j TrustedPorts-A
Firewall -j REJECT --reject-with icmp-host-prohibited-A RoutedDevices -i br0 -j
ACCEPT-A TrustedDevices -i lo -j ACCEPT-A TrustedDevices -i br0 -j ACCEPT-A
TrustedPorts -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPTCOMMIT#
Completed on Thu Sep 6 21:23:39 2012
Finally, one of the lxc containers:
eth0 Link encap:Ethernet HWaddr 00:16:3e:38:88:bb inet
addr:192.168.2.1 Bcast:192.168.2.255 Mask:255.255.255.0 inet6 addr:
fe80::216:3eff:fe38:88bb/64 Scope:Link UP BROADCAST RUNNING MULTICAST
MTU:1500 Metric:1 RX packets:900 errors:0 dropped:0 overruns:0
frame:0 TX packets:697 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000 RX bytes:90043 (90.0 KB) TX
bytes:93265 (93.2 KB)
lo Link encap:Local Loopback inet addr:127.0.0.1
Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK
RUNNING MTU:16436 Metric:1 RX packets:39 errors:0 dropped:0
overruns:0 frame:0 TX packets:39 errors:0 dropped:0 overruns:0
carrier:0 collisions:0 txqueuelen:0 RX bytes:4292 (4.2 KB)
TX bytes:4292 (4.2 KB)
Kernel IP routing tableDestination Gateway Genmask Flags
Metric Ref Use Iface0.0.0.0 192.168.2.1 0.0.0.0 UG 0
0 0 eth0192.168.2.0 0.0.0.0 255.255.255.0 U 0
0 0 eth0
Again the strange routing table with the default gateway at the top of the
list. I should mention that it takes a strangely long time (several seconds) to
ssh from the vm into one of the containers...From the lxc container, I can ping
myself (192.168.2.1), my default gateway (192.168.2.254), and my default
gateway's outgoing interface (192.168.59.128)However, I cannot ping my default
gateway's default gateway (192.168.59.2):
PING 192.168.59.2 (192.168.59.2) 56(84) bytes of data.From 192.168.2.1
icmp_seq=1 Destination Host UnreachableFrom 192.168.2.1 icmp_seq=2 Destination
Host UnreachableFrom 192.168.2.1 icmp_seq=3 Destination Host UnreachableFrom
192.168.2.1 icmp_seq=4 Destination Host UnreachableFrom 192.168.2.1 icmp_seq=5
Destination Host Unreachable
--- 192.168.59.2 ping statistics ---5 packets transmitted, 0 received, +5
errors, 100% packet loss, time 4022ms
Anyway, I would sincerely appreciate any help...
Cheers,
Peter-Frank Spierenburg.
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users
