I am trying to get LXD containers to work inside an EC2 instance. I want
each container to have its own VPC IP address (e.g. 10.0.0.x) so that it
appears like another VM.
Here's what I've managed to find so far:
1. EC2 networking only allows traffic to/from IP addresses which have
been explicitly assigned as secondary addresses to the instance.
This is relatively straightforward:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/MultipleIP.html
although there are limits:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI
For example, a t2.medium instance allows up to three NICs each with 6 IP
addresses
Note also that if you create a second NIC then the primary NIC loses its
auto public IP address, so you have to use an Elastic IP if you want
external connectivity.
2. I was able to create a bridge on a second NIC (leaving my primary NIC
with its original config, so as not to lock myself out), and create LXD
instances statically configured with the secondary addresses of the EC2
instance.
Unfortunately, it seems that Amazon has static mappings of IP addresses
to the instance's MAC address. I've been able to demonstrate this with
tcpdump. Indeed, if you send out ARP queries from the primary
interface, EC2 proxy-ARP responds with the MAC address of the instance.
This means that the LXD container's MAC addresses are not learned, and
traffic can't get to them :-(
So I'm wondering if anyone has come across this, and how they've dealt
with it? Approaches I can think of:
- put the containers on a separate internal subnet (say lxdbr0), and add
one-to-one NAT mappings using iptables. [Normally this would also
require proxy ARP on the outside as well, but it seems EC2 is doing that
for me already anyway]
- messing around with ebtables so that the MAC addresses of incoming
(and possibly outgoing) packets are rewritten at layer 2
- any other suggestions?
Thanks,
Brian Candler.
_______________________________________________
lxc-users mailing list
lxc-users@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
