(Long reply follows…)
Personally, I think you need to look at the big picture for such deployments.
From what I read below, you are asking, “how do I extend my layer-2 subnets
between data centers such that container1 in Europe can talk with container6 in
Asia, etc”. If this is true, I think you need to look at deploying data center
hardware (servers with multiple NICs, IPMI/DRAC/iLO interfaces) with proper
L2/L3 routing (L2TP/IPSEC, etc). And, you must look at how your failover
services will work in this design. It’s easy to get a couple of servers
working with a simple design, but those simple designs tend to go to production
very fast without proper testing and design.
All that said, here is one way I would tackle this type of request:
* Get servers with at least 3 NICs (preferably 5)
* One iLO/DRAC/IPMI interface for out-of-band management
* One for Container server management (ie: LXD1 IP 220.127.116.11) - use a second
NIC for redundancy in a bonded configuration
* One for Container hosting network (ie container1, container2, etc) - use a
second NIC for redundancy and VLANs to separate traffic
* Get firewalls in each location with L2TP/IPSEC support (pfSense works great)
* Extend your L2 networks between your sites with L2TP
* Secure the connection with IPSSEC
* On your LXD servers, create 2 bonded NICs
* One for container management (eth0, eth1)
* One for hosting network (eth2, eth3)
* Use VLANs on hosting network to separate traffic
* Configure your containers with the appropriate VLAN tag (ie: 501)
Once the above is done, your containers can talk w/each other in different
locations. You can use firewall rules to allow/deny IP connections from your
container VMs. You can extend both your container management and hosting
networks across the L2 tunnel allowing you to move VMs at will.
* For server bonded connections, I use linux mode type 6; works well, provides
great throughput, requires no special configuration on directly-connected
* On the LXD side, create multiple profiles with VLAN configurations.
Personally, I have 2 profiles: one for VLAN 501 and one for VLAN 502. Local
firewall provides security between container networks.
* Be mindful of the services you share across the tunnels. Things like iSCSI,
NFS, etc will kill your network performance because of the chatty type of
Some good references:
Caution: L2 networks have a lot of broadcast traffic. If your site-to-site
connections are slow, your entire extended L2 network will suffer. Must find a
way to suppress L2 broadcast/multicast between sites.
Hope this helps. Happy to share my LXD configurations with anyone...
On Sep 18, 2016, at 5:16 AM, Tomasz Chmielewski <man...@wpkg.org> wrote:
It's easy to create a "LAN" for LXD containers on a single LXD server - just
attach them to the same bridge, use the same subnet (i.e. 10.10.10.0/24) -
done. Containers can communicate with each other using their private IP address.
However, with more then one LXD server *not* in the same LAN (i.e. two LXD
servers in different datacentres), the things get tricky.
Is anyone using such setups, with multiple LXD servers and containers being
able to communicate with each other?
LXD1: IP 18.104.22.168, Europe LXD2: IP 22.214.171.124, Asia
container1, 10.10.10.10 container4, 10.10.10.20
container2, 10.10.10.11 container5, 10.10.10.21
container3, 10.10.10.12 container6, 10.10.10.22
LXD3: IP 126.96.36.199, US
While I can imagine setting up many OpenVPN tunnels between all LXD servers
(LXD1-LXD2, LXD1-LXD3, LXD2-LXD3) and constantly adjusting the routes as
containers are stopped/started/migrated, it's a bit of a management nightmare.
And even more so if the number of LXD servers grows.
lxc-users mailing list
lxc-users mailing list