Re: anopa: init system/service manager built around s6
On 4/10/2015 6:41 PM, Aristomenis Pikeas wrote: Laurent (s6-rc), Olivier (anopa), Toki (supervision framework), and Gorka (s6-overlay), I'm having a lot of trouble figuring out the differences between your projects. The s6 suite of utils can be considered building blocks for a full init system, but what does each of your projects do on top of s6? A breakdown of differences: + s6-rc: Laurent's version of an init system based on s6, meant to bring a machine up/down. + anopa: Olivier's version of an init system base on s6, meant to bring a machine up/down. + supervision framework: Toki's version of a complete framework-agnostic init system, partially geared towards OpenRC. Last I checked he had progressed into supporting a complete init (with OpenRC support). + s6-overlay: this is meant for Docker containers, and is most likely the one you want. Yes, there is duplication of effort between Laurent and Olivier, that's ok though - I personally argue that choice is a good thing. :) There are actually many, many projects out there, if you know what to look for, that might provide clues or insights. Ignite on github I believe has some rudimentary init stuff in it, although it's runit based. For a bit of context, my goal is the simplest init system that could possibly work, to be run inside of a docker container. I need to start services and gracefully handle SIGTERM/SIGKILL, with everything logged to standard out. That's about it. But this is proving to be difficult with s6. I've been chipping away at things, but it's slow going between understanding all of the tricky bash-isms and learning about all of the relevant s6 components. If by "tricky bash-isms" you mean the shell redirections and exec and all of that, well...once you can visualize it, it's not that bad really. I don't believe any of the projects use bash directly. Toki's project (as well as my own) assume /bin/sh, which at this time usually means an ash variant. Laurent and Olivier have *nothing* done in bash (beyond the build process). If anything, I think all of the projects are trying hard to avoid "bash specific" implementations; believe me when I say, I've looked at *a lot* of shell scripting in the last 5 months, and I can say that a lot of projects with shell scripts are actually fairly clean. (Yes, I sound a little surprised when I say that)
Re: anopa: init system/service manager built around s6
On 11/04/2015 03:41, Aristomenis Pikeas wrote: I'm having a lot of trouble figuring out the differences between your projects. The s6 suite of utils can be considered building blocks for a full init system, but what does each of your projects do on top of s6? s6-rc and anopa are very similar, except that anopa is released and s6-rc isn't - and won't be for a couple months. (If Olivier agrees, I'll discuss s6-rc design with him.) The point of both s6-rc and anopa is to implement a complete dependency-based init system that handle both one-time initialization scripts and long-running services, the latter being supervised by s6. They're mostly aimed at full-fledged machines ; you probably do not need them in a Docker container, unless you really have a lot of services. AIUI, Toki's supervision framework aims to ease integration of supervision systems with existing init schemes like SysVinit/OpenRC. You probably do not need that either in a Docker container. Gorka's s6-overlay integrates s6 into a Docker image for you, saving you the hassle of writing init scripts by hand for your container. This package will probably help you. For a bit of context, my goal is the simplest init system that could possibly work, to be run inside of a docker container. I need to start services and gracefully handle SIGTERM/SIGKILL, with everything logged to standard out. That's about it. s6-overlay is what you need. You probably don't need anything else. But this is proving to be difficult with s6. From what you are saying in the beginning of your mail, I understand that the difficulty comes from the abundance of peripheral projects with unclear use cases. Did I get your meaning correctly, or are you running into other difficulties ? I've been chipping away at things, but it's slow going between understanding all of the tricky bash-isms and learning about all of the relevant s6 components. Bashisms ? There are no bashisms in s6. Or in s6-overlay. There is no shell at all involved in either of these packages! I suspect you are getting confused by a few projects that are outside your scope. Maybe we need a meta-documentation page, that explains which project does what. This surprises me, however, because I think we're still a quite small community... When in doubt, go back to the basics. s6 itself is not more complicated than runit; there's just more in it, and all the extras are non-essential. http://skarnet.org/software/s6/overview.html should help you make sense of the s6 components. If something is unclear on that page, please tell me what you would like to see. And to use s6 in a Docker container, forget about complete init scripts and service management: you don't need all that. s6-overlay is the only community package you need; it should provide you with the minimal amount of necessary plumbing to run an s6-based container without headaches. Good luck, and please ask more questions if you're running into difficulties. -- Laurent
Re: anopa: init system/service manager built around s6
Laurent (s6-rc), Olivier (anopa), Toki (supervision framework), and Gorka (s6-overlay), I'm having a lot of trouble figuring out the differences between your projects. The s6 suite of utils can be considered building blocks for a full init system, but what does each of your projects do on top of s6? For a bit of context, my goal is the simplest init system that could possibly work, to be run inside of a docker container. I need to start services and gracefully handle SIGTERM/SIGKILL, with everything logged to standard out. That's about it. But this is proving to be difficult with s6. I've been chipping away at things, but it's slow going between understanding all of the tricky bash-isms and learning about all of the relevant s6 components. If any of the four of you, or anyone else on this list, could provide some guidance, I'd greatly appreciate it. I'm convinced that s6 is a better solution than runit, but at this point I may switch back, as things mostly Just Work. Looking forward to hearing your thoughts, Aris > On Apr 10, 2015, at 16:50, Laurent Bercot wrote: > > > Wow! Congratulations for such an important piece of work. > > This is actually pretty embarrassing for me, and I guess I only > have myself to blame: I'm too secretive, and if people are going > to use s6 as a foundation to build more tools - which is definitely > a good thing - then I should embrace it and be more open about my > projects. > > The thing is, I'm currently working on the exact same project: > an init system and service manager. I call it s6-rc, and I intend > to make it a part of some future release of s6. I got the design > down, as well as parts of the code, but a lot remains to be written. > > And you really overtook me on the spot here. So, well done. > > Still, I believe the design work I did on s6-rc is valid, and > AFAICT addresses some things that anopa does not (and vice-versa, of > course.) > > So, shall we join forces ? Shall I present the s6-rc design, and see > if there are things we can put in common, and where to go from here ? > > -- > Laurent >
Re: anopa: init system/service manager built around s6
Wow! Congratulations for such an important piece of work. This is actually pretty embarrassing for me, and I guess I only have myself to blame: I'm too secretive, and if people are going to use s6 as a foundation to build more tools - which is definitely a good thing - then I should embrace it and be more open about my projects. The thing is, I'm currently working on the exact same project: an init system and service manager. I call it s6-rc, and I intend to make it a part of some future release of s6. I got the design down, as well as parts of the code, but a lot remains to be written. And you really overtook me on the spot here. So, well done. Still, I believe the design work I did on s6-rc is valid, and AFAICT addresses some things that anopa does not (and vice-versa, of course.) So, shall we join forces ? Shall I present the s6-rc design, and see if there are things we can put in common, and where to go from here ? -- Laurent
anopa: init system/service manager built around s6
Hey there, So for some time now, I've been meaning to make changes to my system and move to something based around a supervision suite, specifically s6. To do so, I needed something to take care of the whole boot process and such, and eventually decided to make my own init system/service manager, which would take care of that and help with dependencies a bit. Since what I have now is starting to look like something, I thought I'd share, so here we go: introducing anopa. anopa is a bunch of tools/scripts, built around s6, aimed to provide an init system and service manager for Linux systems. Assuming you're familiar with Laurent's description of the three stages of init[1], anopa provides execline scripts for the different stages; That is: - stage1 to be the init that will get things ready, i.e. create the runtime service repository (and s6-svscan scandir), then exec into s6-svscan (as PID1) - stage2 is the script that blocks until s6-svscan is ready (or the catch-all logger running, really) and triggers starting all services - stage3 handles the reboot/poweroff procedure - And there's also stage0 & stage4 which are meant to be used as init/shutdown resp. inside an initramfs s6 works with services, defined via their servicedir. Since it is build around it, anopa uses the same principle; However, a service from anopa's point-of-view can either be one-shot, i.e. a process to start on boot (during stage 1) and that's it (possibly with a counter-part one to be run on shutdown, during stage 3), or a long-run, i.e. a process to be started during phase 2, and to stay up until stage 3 is initiated. Therefore, you can have two kinds of services, and servicedir: one-shot, and long-run. A servicedir is simply a directory, in which the definition of the service is held. Long-run services requires a file run to be present, which will be the long-running process launched & supervised. Therefore, the rule is that if a servicedir contains a file run it is of a long-run service, else of a one-shot one. One-shot services instead have a script start to be run when started, and a script stop to be run when stopped. A one-shot service can either have only a start script, only a stop script, both, or none. In the later case, it can be used simply to order things. Both one-shot & long-run servicedirs can also include folders needs, wants, after, and/or before to define dependencies/ordering. In short, each of those contains regular empty files, names of the services. Anything in after means the current service will only be started after those services; Anything in before means it'll be started before they are. Those are only ordering directives, i.e. if the mentioned services aren't getting started, directives are ignored. wants allows to specify services to automatically started (w/out any ordering), while needs is a special case: the service will be automatically started, it will also be added into after, and should it fail to start, the service won't be started but be in error (dependency failed) instead. There's a bit more description in the man pages, but that's the basic idea. To get more onto how things are meant to work with anopa: during stage1 the runtime service repository is created. This isn't just a simply copy of files, but features some extra little things, more on that in a moment. Then s6-svscan is started, but all services had a file down so nothing (but the catch-all logger) is actually started. That logger unblocks stage2, which will have anopa start everything, one-shot & long-run, using specified ordering. stage3 does the stopping, so to e.g. reboot there's a script that simply calls `s6-svscanctl -rb` leaving stage3 (which s6-svscan execs into) to take care of stopping everything in (reverse) order. Back on create the runtime repo, it's not just a copy of said repo to a tmpfs (/run) but actually features a few things. The way this works in anopa would be to use aa-enable(1) to create the service repository, as well as the scandir. This is done by using source directories, and optionally merging in configuration directories. The idea is that you will not create your service repository manually, but instead using aa-enable(1) from pre-established definitions. The service repository will contain all (enabled) servicedirs, both one-shots and long-runs. For long-run servicedirs, symlinks will be put into directory .scandir which will be used by s6-svscan as its scandir. A typical organization would work like the following : - /usr/lib/services Source directory containing available servicedirs. This is where servicedirs from packages would be installed. - /etc/anopa/services Source directory containing available servicedirs. This is where the administrator can define its own servicedirs, either because they're not provided by packages, or to be used instead. - /etc/anopa/enabled List directory containing either empty regular files, whose name is the name of a service to enable on boot
