On 2/3/19 5:37 AM, Rich Freeman wrote:
Nothing wrong with that approach. I use systemd-nspawn to run a bunch
of containers, hosted in Gentoo, and many of which run Gentoo. However,
these all run systemd and I don't believe you can run nspawn without a
systemd host (the guest/container can be anything). These are containers
running full distros with systemd in my case, not just single-process
containers, in my case. However, nspawn does support single-process
containers, and that includes with veth, but nspawn WON'T initialize
networking in those containers (ie DHCP/etc), leaving this up to the guest
(it does provide a config file for systemd-networkd inside the guest if
it is in use to autoconfigure DHCP).
ACK
That makes me think that systemd-nspawn is less of a fit for what I'm
wanting to do.
I'm not exactly certain what you're trying to accomplish, but namespaces
are just a kernel system call when it comes down to it (two of them I
think offhand). Two util-linux programs provide direct access to them
for shell scripts: unshare and nsenter. If you're just trying to run a
process in a separate namespace so that it can use veth/etc then you could
probably initialize that in a script run from unshare. If you don't need
more isolation you could run it right from the host filesystem without
a separate mount or process namespace. Or you could create a new mount
namespace but only modify specific parts of it like /var/lib or whatever.
That's quite close to what I'm doing. I'm actually using unshare to
create a mount / network / UTS namespace (set) and then running some
commands in them.
The namespaces are functioning as routers. I have an OvS switch
connected to the main / default (unnamed) namespace and nine (internal)
OvS ports, each one in a different namespace. Thus forming a backbone
between the ten network namespaces.
Each of the nine network namespaces then has a veth pair that connects
back to the main network namespace as an L2 interface that VirtualBox
(et al) can glom onto as necessary.
This way I can easily have nine completely different networks that VMs
can use. My main home network has a route to these networks via my
workstation. (I'm actually using routing protocols to distribute this.)
So the main use of the network namespaces is as a basic IP router.
There doesn't /need/ to be any processes running in them. I do run BIRD
in the network namespaces for simplicity reasons. But that's more
ancillary.
I don't strictly need the mount namespaces for what I'm currently doing.
That's left over from when I was running Quagga and /needed/ to alter
some mounts to run multiple instances of Quagga on the same machine.
I do like the UTS namespace so that each ""router has a different host
name when I enter it.
Maybe this helps explain /what/ I'm doing. As for /why/ I'm doing it,
well because reasons. Maybe not even good reasons. But I'm still doing
it. ¯\_(ツ)_/¯ I'm happy to discuss this in a private thread if anyone
is really curious.
People generally equate containers with docker but as you seem to get
you can do a lot with namespaces without basically running completely
independent distros.
Yep. I feel like independent distros, plus heavier weight management
daemons on top are a LOT more than I want.
As stated, I don't really /need/ to run processes in the containers. I
do because it's easy. The only thing I /need/ is the separate IP stack
/ configuration.
Now, I will point out that there are good reasons for keeping things
separate - they may or may not apply to your application. If you just
want to run a single daemon on 14 different IPs and have each of those
daemons see the same filesystem minus /var/lib and /etc that is something
you could certainly do with namespaces and the only resource cost would
be the storage of the extra /var/lib and /etc directories (they could
even use the same shared libraries in RAM, and indeed the same process
image itself I think).
Yep.
The only gotcha is that I'm not sure how much of it is already done, so
you may have to roll your own. If you find generic solutions for running
services in partially-isolated namespaces with network initialization
taken care of for you I'd be very interested in hearing about it.
I think there are a LOT of solutions for creating and managing
containers. (I'm using the term "container" loosely here.) The thing
is that many of them are each their own heavy weight entity. I have yet
to find any that integrate well with OS init scripts.
I feel like what I want to do can /almost/ be done with netifrc. Or
that netifrc could be extended to do what (I think is) /little/
additional work to do it.
I don't know that network namespaces are strictly required. I've been
using them for years. That being said, the current incarnation of
Virtual Routing and Forwarding (VRF) provided by l3mdev seems to be very
promising. I expect that I could make VRF (l3mdev) do what I wanted to
do too. At least the part that I /need/. I'm not sure how to launch
processes associated with the VRF (l3mdev). I'm confident it's
possible, but I've not done it.
But, even VRF (l3mdev) is not supported by netifrc. I feel like the
Policy Based Routing (PBR) is even a kludge and largely consists of
(parts of) the ip / tc commands being put into the /etc/conf.d/net file.
I feel like bridging / bonding / VLANs have better support than PBR
does. All of which are way better supported than VRF (l3mdev) which is
better supported than network namespaces.
Though, I'm not really surprised. All of the init scripts that I've
seen seem to be designed around the premise of a singular system and
have no knowledge that there might be other (virtual) systems. What
little I know about Docker is that even it's configuration is singular
system in nature and still only applies to the instance that it's
working on. I've not seen any OS init scripts that are aware of the
fact that they might be working on other systems. I think the closest
I've seen is FreeBSD jails. But even that is separate init scripts,
which are again somewhat focused on the jail.
I need to do some thinking about /what/ /specifically/ I want to do
before I start thinking about /how/ to go about doing it.
That being said, I think it would be really nice to have various
interfaces tagged with what NetNS they belong to and use the same
net.$interface type init scripts for them.
