Re: [gentoo-user] VRFs / Jails / Containers
On 2/5/19 10:55 AM, Rich Freeman wrote: Yeah, I think you're over-reading into my posts. I'm mostly reacting to your ideas and not trying to be prescriptive. So we have a feedback loop. I'm trying to understand why you're saying what you're saying. I'm still looking for possibilities and pros / cons of each. For example, you talked about running openrc inside the container, and then I talked about how you might do that, and then you asked why would you want to run openrc inside the container. Well, since I don't know exactly what you're doing I don't know - if it doesn't make sense then don't do it... :) Seeing as how my container really is the same as the host, at least when it comes to file system and files, OpenRC is what is there. Hence my interest in re-using what's already there. That is /if/ doing such does not have too many cons. Essentially. It isn't like it won't work if you do something else - it is just a design principle. ACK Suppose you want to manually launch a container without using the init.d script, just as you might launch a non-forking server process to do some console debugging if you were having an issue. And so on. I see what you're saying by starting a typical daemon manually vs init scripts. Given that what I'm talking about doing doesn't leave a process running (I'm ignoring BIRD for the moment) there's nothing to kill to stop it. What I'm doing is really a series of commands that stand the service up and a command that stops it. I guess that I could have an independent script for this and then have the init script do nothing more than call the script with a start parameter. But I feel like my independent script is functionally identical to an init script. It is just a shell script, so it isn't like it won't work if you do it all in the script. *nod* Interesting. I didn't realize that linux even supported creating network namespaces without an associated process. Maybe you don't need one after all. I've been using network namespaces without a process for quite a while. They are extremely handy. I guess since network interfaces can do their netfilter/etc logic without any processes actually listening on them it makes sense that these namespaces might have their own existence. ;-) They don't need to. Which is why I was back to putting the (re)start / stop commands in the init script. IMO they should, but that is like saying that your 5000 line C program should actually have 5000 lines and whitespace, and not look like the javascript source to gmail. It is somewhat subjective, as gcc doesn't care if the whole thing is one big mass of punctuation... :) Yes, there is subjectivity to it. But there are also Gentoo methodologies and guidelines. Can you actually start openrc in a container using a parameter-driven runlevel that isn't a number? I don't know. I'll have to try and find out. I believe you can pass a numeric runlevel to init and it will start on that runlevel (though I'm not sure you can start openrc that way in non-trivial configs as that might skip the boot runlevel, assuming openrc doesn't override this sysvinit behavior). You can't use the kernel command line since containers don't have a separate kernel. I think I understand what you're saying. I also doubt that I will need a boot runlevel in a container the way that I'm doing them. I can see needing a net. and maybe a BIRD init script in the target runlevel. But other than that, there's virtually nothing that does not already exist from the host's file system / environment. You can of course change the default runlevel for openrc using config files in /etc, but those are shared with the host in your proposed design. ACK I /can/ use a mount namespace and have a different /etc. But I'd rather not do that unless there is a need to do so. Maybe you could hack something together here, but honestly I'm not sure what you're getting by not having a separate /etc or at least a bind mount for the openrc config. I currently have no need for a separate /etc. So why have something that I don't currently need? Again, assuming you need openrc in the container at all. Seeing as how the container has the same files as the host, that means that OpenRC is already in the container. So if I can make use of it without causing problems, then why not?
Re: [gentoo-user] VRFs / Jails / Containers
On Mon, Feb 4, 2019 at 7:10 PM Grant Taylor wrote: > > On 02/04/2019 02:58 PM, Rich Freeman wrote: > > So, I think we're miscommunicating a bit here... > > It happens. > Yeah, I think you're over-reading into my posts. I'm mostly reacting to your ideas and not trying to be prescriptive. For example, you talked about running openrc inside the container, and then I talked about how you might do that, and then you asked why would you want to run openrc inside the container. Well, since I don't know exactly what you're doing I don't know - if it doesn't make sense then don't do it... :) > > I'm saying that an init.d script shouldn't try to do anything other > > than initialize a service, which should be implemented outside the > > init.d script. > > It sounds like you are saying that an init script shouldn't do anything > other than (re)start/stop a service and that there should be a separate > script (binary / command) that is the service. Essentially. It isn't like it won't work if you do something else - it is just a design principle. Suppose you want to manually launch a container without using the init.d script, just as you might launch a non-forking server process to do some console debugging if you were having an issue. And so on. It is just a shell script, so it isn't like it won't work if you do it all in the script. > > ip netns add myContainer > ip link add myContainer type veth peer name myHost netns myContainer > ip link set myContainer up > ip addr add 192.0.2.1/24 dev myContainer > ip netns exec myContainer ip link set myHost up > ip netns exec myContainer ip addr add 192.0.2.2/24 dev myHost > Interesting. I didn't realize that linux even supported creating network namespaces without an associated process. Maybe you don't need one after all. I guess since network interfaces can do their netfilter/etc logic without any processes actually listening on them it makes sense that these namespaces might have their own existence. > So, I'm not sure why those commands need to or should live inside > something other than the init script. They don't need to. IMO they should, but that is like saying that your 5000 line C program should actually have 5000 lines and whitespace, and not look like the javascript source to gmail. It is somewhat subjective, as gcc doesn't care if the whole thing is one big mass of punctuation... :) > Why can't I have a single /etc/runlevels/myContainer that is never used > outside of the container and only used inside the container? Remember > that the host and container share the same file system. Can you actually start openrc in a container using a parameter-driven runlevel that isn't a number? I believe you can pass a numeric runlevel to init and it will start on that runlevel (though I'm not sure you can start openrc that way in non-trivial configs as that might skip the boot runlevel, assuming openrc doesn't override this sysvinit behavior). You can't use the kernel command line since containers don't have a separate kernel. You can of course change the default runlevel for openrc using config files in /etc, but those are shared with the host in your proposed design. Maybe you could hack something together here, but honestly I'm not sure what you're getting by not having a separate /etc or at least a bind mount for the openrc config. Again, assuming you need openrc in the container at all. -- Rich
Re: [gentoo-user] VRFs / Jails / Containers
On 2/4/19 3:50 PM, Grant Taylor wrote: > On 02/03/2019 11:23 AM, Michael Orlitzky wrote: >> Ultimately netifrc is just a shell script that parses another shell >> script to construct a third shell script. I don't think doing it with >> only two shell scripts is that much less elegant =) > > The elegance, or lack there of, is not in the number of shell scripts. > Rather the fact that tc (QoS) parameters are stuffed into a command line > verses having things split out and parsed is what I dislike. Take VLANs > for example, there is a netifrc parameter for specifying the VLAN IDs > that belong on an interface. Netifrc will then construct the commands. > People don't need to know how to construct the commands themselves to > utilize VLANs. tc (QoS) is not anywhere nearly as nice. > If you're willing to go all-out to do this right, you can implement that parsing yourself. You can create a package called netifrc-qos that consists of (only?) two files: an OpenRC service script, and an associated conf.d file. Inside the conf.d file you can use whatever variables you like, and then have the init script parse them and run the resulting commands. Afterwards you'd create an ebuild for it, install it from an overlay, and then edit the conf.d file accordingly. It's a lot of work, but that's the most Gentoo way to do it IMO. You might also want to check with the OpenRC/netifrc projects to see if they have anything like this in the works already. Either way they may be willing to collaborate on a design.
Re: [gentoo-user] VRFs / Jails / Containers
On 02/04/2019 02:58 PM, Rich Freeman wrote:
So, I think we're miscommunicating a bit here...
It happens.
I'm saying that an init.d script shouldn't try to do anything other
than initialize a service, which should be implemented outside the
init.d script.
It sounds like you are saying that an init script shouldn't do anything
other than (re)start/stop a service and that there should be a separate
script (binary / command) that is the service.
(I'm going to assume that's accurate unless / until you say otherwise.)
Does this apply if the ""service is something as simple as enabling or
disabling IP forwarding? Should the init script call a separate script
to write the proper value to the requisite proc entry?
So, if you have a shell script that launches a container, then you should
call it from the init.d script. You shouldn't merge them into a single
init.d script that has 30 lines of container setup logic or whatever.
I think the issue that I'm having, and part of what you're calling out
is that the script for a ""container (network namespace) is more than it
should be.
Consider the following commands to start the ""container:
ip netns add myContainer
ip link add myContainer type veth peer name myHost netns myContainer
ip link set myContainer up
ip addr add 192.0.2.1/24 dev myContainer
ip netns exec myContainer ip link set myHost up
ip netns exec myContainer ip addr add 192.0.2.2/24 dev myHost
Consider the following command to stop the ""container:
ip netns del myContainer
I feel like those two sections could easily fit within an OpenRC init
script:
start() {
ip netns add myContainer
ip link add myContainer type veth peer name myHost netns myContainer
ip link set myContainer up
ip addr add 192.0.2.1/24 dev myContainer
ip netns exec myContainer ip link set myHost up
ip netns exec myContainer ip addr add 192.0.2.2/24 dev myHost
}
stop() {
ip netns del myContainer
}
So, I'm not sure why those commands need to or should live inside
something other than the init script.
Please help me understand what I'm missing or not understanding.
Of course. That shell script that launches a container could very well
just launch sysvinit which runs openrc which runs another set of init.d
scripts INSIDE the container to initialize it.
Now I'm starting to think that you are under the impression that the
""container(s) that I'm working with are more complicated and have many
things running in them, more akin to a full OS. That's not the case for
me or my use case.
About the only other added complication might be launching BIRD and / or
an additional network interface.
Yup - though I would think the scripts inside the container would be
fairly different, as they are doing different things. The scripts inside
the container aren't starting containers, for a start...
The contents of the ""container(s) that I'm using are identical to the
host. They actually /are/ the host. I'm not using mount namespaces.
So the ""container ~> network namespace sees the exact same files as the
host.
The only reason that I (sometimes) use the UTS namespace is so that
uname (et al) return a different name when run inside the NetNS.
OpenRC/Netifrc are run by sysvinit in Gentoo, as I mention later on.
These two are not mutually exclusive.
Okay.
Not sure how much of it would be re-use. The scripts inside/outside
the container would likely have different roles.
I would think that I could (re)start / stop BIRD inside the NetNS the
exact same way I do on the host. I would expect that I could use the
same "rc-service bird …" command inside and outside.
Honestly, I wouldn't go sticking container init.d scripts inside the host
init.d. I mean, I guess you could, but again, separation of concerns
and all that. You're going to have to use a separate /etc/runlevels,
so why not just a whole separate /etc?
Why do I need to use a separate /etc/runlevels?
Why can't I have a single /etc/runlevels/myContainer that is never used
outside of the container and only used inside the container? Remember
that the host and container share the same file system.
--
Grant. . . .
unix || die
Re: [gentoo-user] VRFs / Jails / Containers
On 2/4/19 5:10 PM, Grant Taylor wrote: Consider the following commands to start the ""container: ip netns add myContainer ip link add myContainer type veth peer name myHost netns myContainer ip link set myContainer up ip addr add 192.0.2.1/24 dev myContainer ip netns exec myContainer ip link set myHost up ip netns exec myContainer ip addr add 192.0.2.2/24 dev myHost Consider the following command to stop the ""container: ip netns del myContainer This is the other method that I'm starting containers. unshare --mount=/run/mountns/$container --net=/run/netns/$container --uts=/run/utsns/$container /bin/true nsenter --mount=/run/mountns/$container --net=/run/netns/$container --uts=/run/utsns/$container /bin/hostname $container I can use nsenter to execute similar ip link & addr commands to bring the links up. -- Grant. . . . unix || die
Re: [gentoo-user] VRFs / Jails / Containers
So, I think we're miscommunicating a bit here... On Mon, Feb 4, 2019 at 4:10 PM Grant Taylor wrote: > > On 02/04/2019 11:55 AM, Rich Freeman wrote: > > IMO I would separate your container logic from your service manager logic. > > I'm not exactly sure what you mean by "container logic" vs "service > manager logic" and how they differ. I'm assuming that the former > creates / destroys the container and that the latter manages > (re)starting/stopping services where ever they are at. I'm saying that an init.d script shouldn't try to do anything other than initialize a service, which should be implemented outside the init.d script. So, if you have a shell script that launches a container, then you should call it from the init.d script. You shouldn't merge them into a single init.d script that has 30 lines of container setup logic or whatever. > > I'd like to see a way that I can have standard service init scripts and > use them where ever I want them, either inside a container or outside on > the host. Of course. That shell script that launches a container could very well just launch sysvinit which runs openrc which runs another set of init.d scripts INSIDE the container to initialize it. > I'm wanting to avoid having an init script that creates the container > and starts services therein. I'd rather start the container and then > start the services therein using the same type of init scripts, just > called within different contexts. Yup - though I would think the scripts inside the container would be fairly different, as they are doing different things. The scripts inside the container aren't starting containers, for a start... > > As a result, I'd suggest considering using sysvinit inside your > > containers to do the work. > > That is a possibility. But I feel like that's tantamount to saying > "Gentoo doesn't have an answer for what you're wanting to do, so just > use Sys V init scripts." I don't like it. > > I like the idea of re-using standard OpenRC / NetifRC scripts inside the > containers too. Especially if the services don't conflict anywhere. To > me, this re-uses the existing Gentoo methodology in different contexts. OpenRC/Netifrc are run by sysvinit in Gentoo, as I mention later on. These two are not mutually exclusive. > The more that I think about it, largely in response to emails in this > thread, I believe that I want the same overall thing to create the > network between the default / main / unnamed NetNS and the container, as > well as likely re-using the OpenRC / NetifRC scripts to configure things > inside the container. Not sure how much of it would be re-use. The scripts inside/outside the container would likely have different roles. > I think, and would be curious to have someone confirm or refute, that I > could add configuration information to /etc/conf.d/net for the xyz123 > interface inside the container and use an /etc/init.d/net.xyz123 init > script sym-linked to /etc/init.d/net.lo script. > > My host would not have net.xyz123 in any runlevel. Certainly not boot > or default. Honestly, I wouldn't go sticking container init.d scripts inside the host init.d. I mean, I guess you could, but again, separation of concerns and all that. You're going to have to use a separate /etc/runlevels, so why not just a whole separate /etc? -- Rich
Re: [gentoo-user] VRFs / Jails / Containers
On 02/04/2019 11:55 AM, Rich Freeman wrote: IMO I would separate your container logic from your service manager logic. I'm not exactly sure what you mean by "container logic" vs "service manager logic" and how they differ. I'm assuming that the former creates / destroys the container and that the latter manages (re)starting/stopping services where ever they are at. If you have a script that launches a container, then all you need is a generic init.d script that runs it. I guess that's one way to do it. But that doesn't seem very Gentoo to me. I'd like to see a way that I can have standard service init scripts and use them where ever I want them, either inside a container or outside on the host. As long as I don't want to run the same service in multiple places, I don't see a problem with doing that. Multiple instances starts to get more tricky, but is still possible, and should be location agnostic. I launch nspawn containers from systemd units all the time. The only logic in the units is running the command line to start nspawn. IMO if you start mixing the two it will just make it harder to maintain. Sure, an init.d script CAN do anything, but that doesn't mean that you should do it this way. I'm wanting to avoid having an init script that creates the container and starts services therein. I'd rather start the container and then start the services therein using the same type of init scripts, just called within different contexts. Without creating a separate reply I wanted to react to your other email detailing your config. It strikes me that you might not even need containers to set up all those interfaces and the routing between them. However, the container probably still makes sense so that random processes trying to listen on 0.0.0.0 on the host don't end up attaching to all those virtual interfaces. Yes, I could have all the interfaces on the host. But I'm doing a number of different things and don't want to spoil the host. The nice containers that I mentioned are long standing containers. I routinely stand up 10 ~ 100 more for various tests. I'm also using network namespaces as an isolation so that I can easily do various things with networking without the added complexity of isolating things from each other via command line or policy based routing. Each network namespace can easily have it's view of 0.0.0.0 (as a good example) and it's own routing table. I don't need to bother with PBR / ip rules / iptables complexities. Each NetNS just knows about it's local interfaces. Really all you need is some initialization inside each container and then the kernel is doing all the work. You don't really need any userspace process running in the container except for the fact that kernel namespaces are attached to processes. I mostly agree. I am running BIRD inside the container, but that's more of a would be nice to have and I can work around not having it. There are also the occasional commands that I want to run to do troubleshooting (ping, traceroute, etc) as well as dynamically modifying the containers which is usually done via "nsenter …" or "ip netns exec $NetNSname …" commands. As a result, I'd suggest considering using sysvinit inside your containers to do the work. That is a possibility. But I feel like that's tantamount to saying "Gentoo doesn't have an answer for what you're wanting to do, so just use Sys V init scripts." I don't like it. I like the idea of re-using standard OpenRC / NetifRC scripts inside the containers too. Especially if the services don't conflict anywhere. To me, this re-uses the existing Gentoo methodology in different contexts. You might run openrc/netifrc to do the network setup inside each container, or just have sysvinit run a shell script that initializes and then terminates, leaving init running childless indefinitely (I assume it supports this). If you want a process to noop indefinitely at minimal cost that is basically the definition of what sysvinit does... The more that I think about it, largely in response to emails in this thread, I believe that I want the same overall thing to create the network between the default / main / unnamed NetNS and the container, as well as likely re-using the OpenRC / NetifRC scripts to configure things inside the container. I think, and would be curious to have someone confirm or refute, that I could add configuration information to /etc/conf.d/net for the xyz123 interface inside the container and use an /etc/init.d/net.xyz123 init script sym-linked to /etc/init.d/net.lo script. My host would not have net.xyz123 in any runlevel. Certainly not boot or default. I think that would mean that I could run rc-service net.xyz123 start inside the container and re-use existing Gentoo methodology. Now I wonder if I could use custom runlevels for each container and rely on standard init system. }:-) But that's a different
Re: [gentoo-user] VRFs / Jails / Containers
On 02/03/2019 11:23 AM, Michael Orlitzky wrote: Ultimately netifrc is just a shell script that parses another shell script to construct a third shell script. I don't think doing it with only two shell scripts is that much less elegant =) The elegance, or lack there of, is not in the number of shell scripts. Rather the fact that tc (QoS) parameters are stuffed into a command line verses having things split out and parsed is what I dislike. Take VLANs for example, there is a netifrc parameter for specifying the VLAN IDs that belong on an interface. Netifrc will then construct the commands. People don't need to know how to construct the commands themselves to utilize VLANs. tc (QoS) is not anywhere nearly as nice. Bridging and bonding is similarly more graceful than tc (QoS). You could go all the way and write your own OpenRC service as /etc/init.d/whatever. That's sort of where I'm gravitating at the moment. Something I can (re)start/stop via standard init commands. You can make it depend on the network being up, and then just write everything that you want it to do into the start function with the corresponding "undo" steps in the stop function. Maybe it will need to depend on the lowest level of networking. Maybe. Seeing as how it would provide networking between the host and the namespaces (containers), I think it would functionally be parallel to the networking services. I think namespaces could be up even if the main network was not. If the series of commands is long and complicated and if you sometimes want to do/undo this subset of the configuration independently, then that's how I'd do it. The number of commands is really dependent on what I'm doing at a higher level. I can see having relatively similar commands for different namespaces broken out into separate files such that it's easy to (re)start/stop individual namespaces. I might see if there's a way to re-use the same file much like net. is a sym-link to net.lo. -- Grant. . . . unix || die
Re: [gentoo-user] VRFs / Jails / Containers
On Mon, Feb 4, 2019 at 1:44 PM Grant Taylor wrote: > > I'm starting to wonder if I'm going to be better off writing new scripts > that will match existing init scripts and their methodology to > (re)start/stop namespaces / containers / jails. Perhaps firejail will > give me what I want or provide insight. > IMO I would separate your container logic from your service manager logic. If you have a script that launches a container, then all you need is a generic init.d script that runs it. I launch nspawn containers from systemd units all the time. The only logic in the units is running the command line to start nspawn. IMO if you start mixing the two it will just make it harder to maintain. Sure, an init.d script CAN do anything, but that doesn't mean that you should do it this way. Without creating a separate reply I wanted to react to your other email detailing your config. It strikes me that you might not even need containers to set up all those interfaces and the routing between them. However, the container probably still makes sense so that random processes trying to listen on 0.0.0.0 on the host don't end up attaching to all those virtual interfaces. Really all you need is some initialization inside each container and then the kernel is doing all the work. You don't really need any userspace process running in the container except for the fact that kernel namespaces are attached to processes. As a result, I'd suggest considering using sysvinit inside your containers to do the work. You might run openrc/netifrc to do the network setup inside each container, or just have sysvinit run a shell script that initializes and then terminates, leaving init running childless indefinitely (I assume it supports this). If you want a process to noop indefinitely at minimal cost that is basically the definition of what sysvinit does... -- Rich
Re: [gentoo-user] VRFs / Jails / Containers
On 02/04/2019 09:23 AM, Laurence Perkins wrote: Have you tried firejail? It gives you convenient ways to set up the container parameters consistently and is in the repo. No, I have not. Thank you for the pointer. Its invocation is also simple enough to not clutter up your startup scripts. I don't think I mind adding things to start up scripts. I'm more looking for the most Gentoo way to do what I'm wanting to do without relying on something on top of Gentoo. So if that involves adding things to start up scripts, I'm cool with it. I just don't want to add an entire subsystem, like Docker (et al), if I don't actually have to. I'm starting to wonder if I'm going to be better off writing new scripts that will match existing init scripts and their methodology to (re)start/stop namespaces / containers / jails. Perhaps firejail will give me what I want or provide insight. -- Grant. . . . unix || die
Re: [gentoo-user] VRFs / Jails / Containers
On Sat, 2019-02-02 at 19:32 -0700, Grant Taylor wrote: > Does Gentoo have any support for VRFs or (chroot) Jails or > Containers > without going down the Docker (et al) path? > > I'm wanting to do some things with a Gentoo router that is trivial to > do > with network namespaces via manual commands ~> scripts. But that's > far > from standard Gentoo init script based system. And I'd like > something > more Gentoo standards based. > > Does Gentoo have or support anything like this natively? Or am I > getting into territory where I'm rolling my own? > Have you tried firejail? It gives you convenient ways to set up the container parameters consistently and is in the repo. Its invocation is also simple enough to not clutter up your startup scripts. LMP
Re: [gentoo-user] VRFs / Jails / Containers
On 2/3/19 12:39 PM, Grant Taylor wrote: On 2/3/19 6:26 AM, Michael Orlitzky wrote: You can add commands to your existing network configuration that will be run when an interface comes up. For example, in /etc/conf.d/net, ifup_wlan0="iwconfig \$int key s:secretkey enc open essid foobar" Ya I find that to be an absolute kludge. Does it work? Yes. Is it clean? Probably not. Is it graceful? Absolutely not. Think about how it's possible to configure bridging / bonding / VLANs via various parameters and having netifrc construct the commands that are run in the background. Ultimately netifrc is just a shell script that parses another shell script to construct a third shell script. I don't think doing it with only two shell scripts is that much less elegant =) You could go all the way and write your own OpenRC service as /etc/init.d/whatever. You can make it depend on the network being up, and then just write everything that you want it to do into the start function with the corresponding "undo" steps in the stop function. If the series of commands is long and complicated and if you sometimes want to do/undo this subset of the configuration independently, then that's how I'd do it.
Re: [gentoo-user] VRFs / Jails / Containers
On 2/3/19 6:26 AM, Michael Orlitzky wrote: You can add commands to your existing network configuration that will be run when an interface comes up. For example, in /etc/conf.d/net, ifup_wlan0="iwconfig \$int key s:secretkey enc open essid foobar" Ya I find that to be an absolute kludge. Does it work? Yes. Is it clean? Probably not. Is it graceful? Absolutely not. Think about how it's possible to configure bridging / bonding / VLANs via various parameters and having netifrc construct the commands that are run in the background. I'd love to see something that assumes the commands run in the main / default / unnamed network namespace / VRF unless otherwise specified. I'd love to be able to add a parameter to a configuration file that tells sshd to run in a specific VRF like Alarig was wanting to do. Heck, I'd like to see init scripts gracefully deal with the fact that there should be multiple instances of a daemon running, even if they are simply on different ports, much less different VRFs or namespaces.
Re: [gentoo-user] VRFs / Jails / Containers
On 2/3/19 1:50 AM, Alarig Le Lay wrote: For the VRF part, Gentoo supports it; it’s in the upstream kernel sources. Yep. I've been doing Network Namespaces, and VRF to a lesser degree, for quite a while now. It's just all been manual or ad-hock scripts. I only tried it once, but failed because my sshd should have been lunch in my VRF and I didn’t quickly find a way to do it. Yep. That's the type of integration that I've found lacking. I'm only currently asking about how to configure the various network components, not even how to run processes inside of the various systems. But otherwise, it worked. It absolutely manually works. I'm looking for the thing(s) to allow the Gentoo OS init scripts to take over some of the management. That's what I'm finding lacking. I asked my question because I was hoping that someone would know about something I didn't. ;-)
Re: [gentoo-user] VRFs / Jails / Containers
On 2/2/19 11:09 PM, Bill Kenworthy wrote: I am unclear on what you are trying to do. See my reply to Rich's message for a description. I find the gentoo scripts good for the simple case but a complex case almost always needs extra help. Yep. I was hoping that there was something that I was unaware of or could extend to do what I want to do. If its networking, could something like shorewall help? No, I don't think that Shorewall or a similar firewall config management system will help. I also find those systems annoying. Sure, they have their benefits. But why do I need them when I should be able to do the same thing on a stock Gentoo (or other) Linux system? After all they are using the same kernel. (Maybe a different version or config there of.) I will occasionally look at those solutions and treat them like themed Lego sets. I build them, look at them, analyze them, and pull out the distinct Lego bricks that I want to use in my own system. }:-)
Re: [gentoo-user] VRFs / Jails / Containers
On 2/3/19 5:37 AM, Rich Freeman wrote: Nothing wrong with that approach. I use systemd-nspawn to run a bunch of containers, hosted in Gentoo, and many of which run Gentoo. However, these all run systemd and I don't believe you can run nspawn without a systemd host (the guest/container can be anything). These are containers running full distros with systemd in my case, not just single-process containers, in my case. However, nspawn does support single-process containers, and that includes with veth, but nspawn WON'T initialize networking in those containers (ie DHCP/etc), leaving this up to the guest (it does provide a config file for systemd-networkd inside the guest if it is in use to autoconfigure DHCP). ACK That makes me think that systemd-nspawn is less of a fit for what I'm wanting to do. I'm not exactly certain what you're trying to accomplish, but namespaces are just a kernel system call when it comes down to it (two of them I think offhand). Two util-linux programs provide direct access to them for shell scripts: unshare and nsenter. If you're just trying to run a process in a separate namespace so that it can use veth/etc then you could probably initialize that in a script run from unshare. If you don't need more isolation you could run it right from the host filesystem without a separate mount or process namespace. Or you could create a new mount namespace but only modify specific parts of it like /var/lib or whatever. That's quite close to what I'm doing. I'm actually using unshare to create a mount / network / UTS namespace (set) and then running some commands in them. The namespaces are functioning as routers. I have an OvS switch connected to the main / default (unnamed) namespace and nine (internal) OvS ports, each one in a different namespace. Thus forming a backbone between the ten network namespaces. Each of the nine network namespaces then has a veth pair that connects back to the main network namespace as an L2 interface that VirtualBox (et al) can glom onto as necessary. This way I can easily have nine completely different networks that VMs can use. My main home network has a route to these networks via my workstation. (I'm actually using routing protocols to distribute this.) So the main use of the network namespaces is as a basic IP router. There doesn't /need/ to be any processes running in them. I do run BIRD in the network namespaces for simplicity reasons. But that's more ancillary. I don't strictly need the mount namespaces for what I'm currently doing. That's left over from when I was running Quagga and /needed/ to alter some mounts to run multiple instances of Quagga on the same machine. I do like the UTS namespace so that each ""router has a different host name when I enter it. Maybe this helps explain /what/ I'm doing. As for /why/ I'm doing it, well because reasons. Maybe not even good reasons. But I'm still doing it. ¯\_(ツ)_/¯ I'm happy to discuss this in a private thread if anyone is really curious. People generally equate containers with docker but as you seem to get you can do a lot with namespaces without basically running completely independent distros. Yep. I feel like independent distros, plus heavier weight management daemons on top are a LOT more than I want. As stated, I don't really /need/ to run processes in the containers. I do because it's easy. The only thing I /need/ is the separate IP stack / configuration. Now, I will point out that there are good reasons for keeping things separate - they may or may not apply to your application. If you just want to run a single daemon on 14 different IPs and have each of those daemons see the same filesystem minus /var/lib and /etc that is something you could certainly do with namespaces and the only resource cost would be the storage of the extra /var/lib and /etc directories (they could even use the same shared libraries in RAM, and indeed the same process image itself I think). Yep. The only gotcha is that I'm not sure how much of it is already done, so you may have to roll your own. If you find generic solutions for running services in partially-isolated namespaces with network initialization taken care of for you I'd be very interested in hearing about it. I think there are a LOT of solutions for creating and managing containers. (I'm using the term "container" loosely here.) The thing is that many of them are each their own heavy weight entity. I have yet to find any that integrate well with OS init scripts. I feel like what I want to do can /almost/ be done with netifrc. Or that netifrc could be extended to do what (I think is) /little/ additional work to do it. I don't know that network namespaces are strictly required. I've been using them for years. That being said, the current incarnation of Virtual Routing and Forwarding (VRF) provided by l3mdev seems to be very promising. I
Re: [gentoo-user] VRFs / Jails / Containers
On 2/2/19 10:56 PM, Grant Taylor wrote: On 2/2/19 7:36 PM, Bill Kenworthy wrote: LXC containers ?? Maybe. I just feel like that's more heavy weight than I want. I'm functionally running a series of ip commands to configure networking in a special way. You can add commands to your existing network configuration that will be run when an interface comes up. For example, in /etc/conf.d/net, ifup_wlan0="iwconfig \$int key s:secretkey enc open essid foobar" (taken from the example file that ships with OpenRC).
Re: [gentoo-user] VRFs / Jails / Containers
On Sat, Feb 2, 2019 at 11:52 PM Grant Taylor wrote: > > On 2/2/19 9:39 PM, Michael Jones wrote: > > systemd-nspawn is also an option, but I don't think that'll work with > > OpenRC. > > Ya I moved (back to) Gentoo to get away from systemd. I'm not > going to voluntarily opt to use it, or any of it's children. That's > /my/ opinion. I know others opinions differ. > Nothing wrong with that approach. I use systemd-nspawn to run a bunch of containers, hosted in Gentoo, and many of which run Gentoo. However, these all run systemd and I don't believe you can run nspawn without a systemd host (the guest/container can be anything). These are containers running full distros with systemd in my case, not just single-process containers, in my case. However, nspawn does support single-process containers, and that includes with veth, but nspawn WON'T initialize networking in those containers (ie DHCP/etc), leaving this up to the guest (it does provide a config file for systemd-networkd inside the guest if it is in use to autoconfigure DHCP). I'm not exactly certain what you're trying to accomplish, but namespaces are just a kernel system call when it comes down to it (two of them I think offhand). Two util-linux programs provide direct access to them for shell scripts: unshare and nsenter. If you're just trying to run a process in a separate namespace so that it can use veth/etc then you could probably initialize that in a script run from unshare. If you don't need more isolation you could run it right from the host filesystem without a separate mount or process namespace. Or you could create a new mount namespace but only modify specific parts of it like /var/lib or whatever. People generally equate containers with docker but as you seem to get you can do a lot with namespaces without basically running completely independent distros. Now, I will point out that there are good reasons for keeping things separate - they may or may not apply to your application. If you just want to run a single daemon on 14 different IPs and have each of those daemons see the same filesystem minus /var/lib and /etc that is something you could certainly do with namespaces and the only resource cost would be the storage of the extra /var/lib and /etc directories (they could even use the same shared libraries in RAM, and indeed the same process image itself I think). The only gotcha is that I'm not sure how much of it is already done, so you may have to roll your own. If you find generic solutions for running services in partially-isolated namespaces with network initialization taken care of for you I'd be very interested in hearing about it. -- Rich
Re: [gentoo-user] VRFs / Jails / Containers
For the VRF part, Gentoo supports it; it’s in the upstream kernel sources. I only tried it once, but failed because my sshd should have been lunch in my VRF and I didn’t quickly find a way to do it. But otherwise, it worked. -- Alarig
Re: [gentoo-user] VRFs / Jails / Containers
On 3/2/19 12:52 pm, Grant Taylor wrote: > On 2/2/19 9:39 PM, Michael Jones wrote: >> systemd-nspawn is also an option, but I don't think that'll work with >> OpenRC. > > Ya I moved (back to) Gentoo to get away from systemd. I'm not > going to voluntarily opt to use it, or any of it's children. That's > /my/ opinion. I know others opinions differ. > > Thank you for the information all the same. > I am unclear on what you are trying to do. I find the gentoo scripts good for the simple case but a complex case almost always needs extra help. If its networking, could something like shorewall help? BillK
Re: [gentoo-user] VRFs / Jails / Containers
On 2/2/19 9:39 PM, Michael Jones wrote: systemd-nspawn is also an option, but I don't think that'll work with OpenRC. Ya I moved (back to) Gentoo to get away from systemd. I'm not going to voluntarily opt to use it, or any of it's children. That's /my/ opinion. I know others opinions differ. Thank you for the information all the same.
Re: [gentoo-user] VRFs / Jails / Containers
systemd-nspawn is also an option, but I don't think that'll work with OpenRC. On Sat, Feb 2, 2019 at 9:56 PM Grant Taylor < gtay...@gentoo.tnetconsulting.net> wrote: > On 2/2/19 7:36 PM, Bill Kenworthy wrote: > > LXC containers ?? > > Maybe. > > I just feel like that's more heavy weight than I want. > > I'm functionally running a series of ip commands to configure networking > in a special way. > > Maybe I should look into what it takes to extend netifrc to support what > I want. I sort of think that VRF could model off of bonding and / or > bridge and / or VLAN devices. At least in the master / slave aspect. > > I'm sure that veth will be a new concept, but it may be able to model > after a tunnel interface. > > It would be really nice to have network namespace support. But I don't > see anything that could be modeled off of. > >
Re: [gentoo-user] VRFs / Jails / Containers
On 2/2/19 7:36 PM, Bill Kenworthy wrote: LXC containers ?? Maybe. I just feel like that's more heavy weight than I want. I'm functionally running a series of ip commands to configure networking in a special way. Maybe I should look into what it takes to extend netifrc to support what I want. I sort of think that VRF could model off of bonding and / or bridge and / or VLAN devices. At least in the master / slave aspect. I'm sure that veth will be a new concept, but it may be able to model after a tunnel interface. It would be really nice to have network namespace support. But I don't see anything that could be modeled off of.
Re: [gentoo-user] VRFs / Jails / Containers
On 3/2/19 10:32 am, Grant Taylor wrote: > Does Gentoo have any support for VRFs or (chroot) Jails or Containers > without going down the Docker (et al) path? > > I'm wanting to do some things with a Gentoo router that is trivial to > do with network namespaces via manual commands ~> scripts. But that's > far from standard Gentoo init script based system. And I'd like > something more Gentoo standards based. > > Does Gentoo have or support anything like this natively? Or am I > getting into territory where I'm rolling my own LXC containers ?? BillK
