I need your advice on this web page
Hi all, I'm writing you because you're by far the most init aware people I know. I've created a web page about features and benefits of a few init systems. The audience for this web page isn't people like all of you, but instead for the average DIY Linux user, so it deliberately contains some oversimplifications. It's here: http://www.troubleshooters.com/linux/init/features_and_benefits.htm I'd appreciate if some of you could look this thing over and identify any factual errors. I'm not talking about oversimplifications, I'm talking about dead bang wrong info. As usual, I appreciate all that you are all doing! Thanks, SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance
Re: Could s6-scscan ignore non-servicedir folders?
On Wed, 21 Jan 2015 18:24:58 +0100 Olivier Brunel j...@jjacky.com wrote: Hi Laurent, So you mentioned breaking compatibility recently, and I figure that might be a good time for me to mention something. I'd like to set up my system around s6, and have been working on this lately. I'll have to setup some scripts for different init stages, using s6-svscan as stage 2, as you've described elsewhere. But I also want to have a system to start (and stop) services in order. I see this whole idea of order/dependency is something that is being talked about, but currently not supported. Can't you do something like this: http://smarden.org/runit/faq.html#depends Furthermore, I want this system of mine to include other kinds of services, that is one-time process/scripts that needs to be run once (on boot), and that's it. And to make things simpler, I want to have it all work together, mixing longrun services (s6 supervised) and oneshot services when it comes to dependency/order definition. I do too. If you have a run-once thing that quickly returns, couldn't you just not exec the thing in the run script, and then have the last statement in your run script write a down file to the service? I'm assuming that s6 does down files the same way as daemontools. Then, all that remains is to have stage 1 of your boot delete all the down files that were put there to achieve run-once. If it really is a daemon, but you don't want it respawned, couldn't you have the finish script (they have those in runit, I don't know about s6) write the down file? You know what you could do? You could make a $servicedir/oneshot shellscript that does something like this: touch ./down cat rm $scriptname/down $whatever/enable_oneshots.sh Then all you'd need to do is run $whatever/enable_oneshots.sh in stage 1, and right after that truncate and make executable $whatever/enable_oneshots. SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance
Re: I need your advice on this web page
On Fri, 16 Jan 2015 14:12:07 +0100 Laurent Bercot ska-supervis...@skarnet.org wrote: On 16/01/2015 01:05, Steve Litt wrote: http://www.troubleshooters.com/linux/init/features_and_benefits.htm (I'm lacking sleep and I'm going to talk about systemd. Not a good combination. So, apologies in advance for the rant, for the inevitable coarse language, and for the very opinionated post.) Hi Steve, Hi Laurent, First, thanks for answering me. This email was *precisely* the kind of feedback I was looking for, and will result in changes to my document. The main comment that I have to make after reading your document is that despite your attempt at impartiality, and avowed liking of daemontools- inspired schemes, it is still systemd-centric and biased in its favor. Not purposefully, of course, but simply because the systemd propaganda machine works, and has already taught you to think in systemd terms - which, let it be said openly, are often pure marketing bullshit. :-) I guess I *did* appear impartial. This is the first time in history I've been accused of being pro-systemd. Please tell Don Armstrong --- maybe he'll let me back on Debian-User (just kidding, I'm done with that mailing list). Let's dig into some of those. I. Socket activation. This has to be my new favorite marketing buzzword. Socket activation, people. (My sockets are activated. I put my feet into them, and now they move. It's awesome.) Last summer, I wrote a post about it - and you were in that discussion, Steve: http://forums.gentoo.org/viewtopic-t-994548-postdays-0-postorder-asc-start-25.html#7581522 The short version of it is that socket activation, as systemd defines it, is a hack that mixes several different already-existing concepts in a shaker, and what you get in the end is *worse* than if you had nothing at all - but since everything is mixed and confused, nobody notices, and systemd can pretend it's doing that wonderful thing that no other system does, and people believe it. When you think socket activation, the questions to ask are the following. (I wrote answers from the s6 point of view, which mostly applies to other supervision suites too.) Q1. Does the init system work as a super-server pre-opening and binding sockets so that daemons do not have to do it themselves ? A1. It is NOT the freaking init system's job to pre-open sockets. Doing so requires the init system to be aware of every single socket-listening daemon on the machine, which translates into a central registry. And that is how you turn Unix into Microsoft Windows. If you need to pre-open and bind sockets all at once, use inetd. This is exactly what inetd does, and at least it doesn't require running as process 1, or communicating over D-Bus. Better, use decent superservers, such as s6-tcpserver or tcpsvd, one per service. For Unix domain sockets, which is what systemd focuses on (and rightly so), there's s6-ipcserver. Starting those superservers in parallel, as any supervision suite can, will end up being just as fast as trying to open every possible socket early on in process 1. There is no reason at all why a superserver should be tied to an init system. Q2. Does the init system allow you to start processes as soon as the sockets are open, before the servers are ready ? This is the much touted benefit of socket activation on http://0pointer.de/blog/projects/socket-activation.html A2. HELL NO. WHY ON EARTH WOULD I WANT THAT ? - Doing so has a serious reliability cost: if a service ends up having issues, but dependent services have been started and are assuming that it's working, hilarity will ensue. I mean, you could also play Dance Dance Revolution on a mat made of old WWII landmines. What could possibly go wrong ? - It's especially twisted with logging. Sure, start your daemons before the logger is running, no problem. That way, if anything goes wrong, you'll have no way of knowing what happened. Have fun debugging. - The speed benefits are minimal at best. As I wrote in my post, daemons can perform their first writes in parallel, but as soon as they have to read, they block anyway, waiting for their dependencies. The only case where daemons write and never read, and could benefit from such a scheme, is... when they write to their logger. And, as we just saw, it is a really good idea to write logs before the logger is guaranteed operational. This is just not worth it. Simply starting all the services as soon as possible in parallel will have the same benefits - the kernel will schedule everything to the best of its abilities, and there will be no risk of spectacular crashes. Q3. Does the init system allow you to hold a copy of a bound socket for the daemon to retrieve if it has to restart ? A3. This is what I call fd-holding, and is the *only* thing of value in socket activation. No, supervision suites do not perform
Execline: was s6 init-stage1
On Tue, 06 Jan 2015 04:28:59 +0100 Laurent Bercot ska-supervis...@skarnet.org wrote: Far be it from me to discourage you from your noble quest ! But you could write it in sh and just use the 'redirfd' command from execline, which does the FIFO magic. I read the documentation of execline, complete with the diagrams, and didn't understand a word of it. I think a lot more documentation and a lot of examples would help immensely. SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance
Re: runit source control?
On Mon, 09 Feb 2015 19:22:26 +0100 Laurent Bercot ska-supervis...@skarnet.org wrote: On 09/02/2015 18:50, Buck Evan wrote: Is there truly no public access to the source control for runit? If so, it's decidedly odd. Not that odd. Small projects don't really need an SCM. Most pre-git SCMs are cumbersome or impractical. runit predates git. Gerrit chose to focus on the code, not on the packaging - that's all there is to it. I totally agree. As long as a project is primarily a one man band, there's no need for all that stuff. It's probable that other peoples contributions can be sent in as patches against the last tarballed version. SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance
Re: thoughts on rudimentary dependency handling
On Wed, 7 Jan 2015 14:25:28 -0800 Avery Payne avery.p.pa...@gmail.com wrote: On Wed, Jan 7, 2015 at 7:23 AM, Steve Litt sl...@troubleshooters.com wrote: I'm pretty sure this conforms to James' preference (and mine probably) that it be done in the config and not in the init program. To satisfy Laurent's preference, everything but the exec cron -f could be commented out, and if the user wants to use this, he/she can uncomment all the rest. Or your run script writing program could have an option to write the dependencies, or not. I've pretty much settled on a system-wide switch in sv/.env (which in the scripts will show up as ../.env). The switch will, by default, follow Laruent's behavior of naive launching, ie. no dependencies are up, missing dependencies cause failures, I'm having trouble understanding exactly what you're saying. You mean the executable being daemonized fails, by itself, because a service it needs isn't there, right? You *don't* mean that the init itself fails, right? This is all pretty cool, Avery! Currently, my best guess is that eventually my daily driver computer will be initted by runit (with Epoch as a backup, should I bork my runit). It sounds like what you're doing will ease runit config by an order of magnitude. and the admin must check logging for notifications. Enabling the feature would be as simple as echo 1 /etc/sv/.env/NEEDS_ENABLED ...and every new service launch would receive it. You could also force-reload with a restart command. Without the flag, the entire chunk of dependency code is bypassed and the launch continues as normal. I'm not sure what you're saying. Are you saying that the dependency code is in the runscript, but within an IF statement that checks for ../env/NEEDS_ENABLED? The goal is the same but the emphasis has changed. This will be considered a fall-back feature for those systems that do not have such a tool available, or have constraints that force the continued use of a shell launcher. It is the option of last resort, and while I think I can make it work fairly consistently, it will come with some warnings in the wiki. For Laurent, he wouldn't even need to lift a finger - it fully complies with his desires out of the box. ;-) As new tools emerge in the future, I will be able to write a shunt into the script that detects the tool and uses it instead of the built-in scripted support. This will allow Laurent's work to be integrated without messing anything up, so the behavior will be the same, but implemented differently. Now THAT'S Unix at work! Finally, with regard to the up vs actually running issue, I'm not even going to try and address it due to the race conditions involved. The best I will manage is to first issue the up, then do a service check to confirm that it didn't die upon launch, which for a majority (but not all) cases should suffice. Yes, there are still race conditions, but that is fine - I'm falling back to the original model of service fails continually until it succeeds, which means a silently-failed child dependency that was missed by the check command will still cause the parent script to fail, because the daemon itself will fail. It is a crude form of graceful failure. So the supervisor starts the parent again...and again...until the truant dependency is up and running, at which point it will bring the parent up. Like I said, this will be a fall-back feature, and it will have minor annoyances or issues. Yes. If I'm understanding you correctly, you're only going so far in determinint really up, because otherwise writing a one size fits all services thing starts getting way too complicated. I was looking at runit docs yesterday before my Init System presentation, and learned that I'm supposed to put my own Really Up code in a script called ./check. Right now the biggest problem is handling all of the service tool calls. They all have the same grammar, (tool) (command) (service name), so I can script that easily. Getting the tools to show up as the correct command and command option is something else, and I'm working on a way to wedge it into the use-* scripts so that the tools are set up out of the box all at the same time. This will create $SVCTOOL, and a set of $CMDDOWN, $CMDUP, $CMDCHECK, etc. that will be used in the scripts. **Once that is done I can fully test the rest of the dependency concept and get it fleshed out.** If anyone wants to see it, email me directly and I'll pass it along, but there's not much to look at. If I read the preceding correctly, you're making service tool calls for runit, s6, perp and nosh grammatically identical. Are you doing that so that your run scripts can invoke the init-agnostic commands, so you just have one version of your scripts? However you end up doing the preceding, I think it's essential to thoroughly document it, complete with examples. I think that the additional layer
Re: anopa: init system/service manager built around s6
On Fri, 10 Apr 2015 18:41:45 -0700 Aristomenis Pikeas pik...@gmail.com wrote: [snip] For a bit of context, my goal is the simplest init system that could possibly work, to be run inside of a docker container. warningEverything I say here is sort of academic. Also, a lot of the things I say about daemontools are probably true of s6 too./warning Hi Aristomenis, I always thought the simplest possible init system is Rich Felker's init; the one whose listing is at the bottom of this blog entry: http://ewontfix.com/14/ Theoretically, Rich Felker's init is all you need in order to init, pushing things like process management to be run by /etc/rc or whatever you want to call that shellscript. I was able to init with Rich Felker's script going to OpenRC for process management (such as it is in OpenRC), but there were problems with reboots requiring journal restores, and other problems. It's surprising I didn't attempt to use Rich Felker's init to run daemontools or daemontools-encore, with /etc/rc doing things like starting udev (or vdev or whatever) and devfs and network initialization, and finally running daemontools (via svscanboot) to manage processes such as virtual terminals and daemons. As far as sending all log messages to stdout, I don't know if daemontools can do that, but I know daemontools can send all log messages to the same file, assuming the daemontools log run scripts are configured to do that. I need to start services and gracefully handle SIGTERM/SIGKILL, with everything logged to standard out. That's about it. But this is proving to be difficult with s6. Daemontools correctly handles SIGTERM/SIGKILL, if by correctly you mean restarting it. With daemontools (and I believe s6 too), the way to stop a daemon is with the svc -d command. I've been chipping away at things, but it's slow going between understanding all of the tricky bash-isms and learning about all of the relevant s6 components. I'm under the impression that s6 offers use of the execline language as a way to avoid bash-isms. I found execline's documentation insufficient for me to learn it, but my impression is its purpose is to provide you with a language that's less tricky and more determinant and more versatile than bash. If any of the four of you, or anyone else on this list, could provide some guidance, I'd greatly appreciate it. I'm convinced that s6 is a better solution than runit, but at this point I may switch back, as things mostly Just Work. I think s6 has a documentation problem. I compiled it, but was never able to get it to run, mostly because I was unable to find, in the docs, what to do after I got it compiled. And there were way too few examples of execline code, and I couldn't figure out the language logic from the descriptive diagrams. As far as s6 vs runit, my impression is that s6, runit, nosh and perp are all daemontools-influenced inits that, relative to the entire constellation of inits out there, are all excellent and very similar to each other. Runit happens to be the one I was able to actually get running, but if I were as systems-programming smart as the average person on this list, I'm sure I could have gotten all four running. And once again, all of them have documentation problems. Once runit, s6, perp and nosh have documentation useful for the average person, the world will be a wonderful place. By the way, if you're looking for a simple init, consider Epoch: http://universe2.us/epoch.html . In my opinion it's the best documented of all init systems, and in my opinion it's the quickest and easiest to get running on any old random Linux machine (can't run on BSD, but neither can docker). Unfortunately, it's not daemontools derived and therefore doesn't manage processes in the foreground with all the advantages foreground management confers. And it spawns processes consecutively, by number, rather than concurrently, so you might not want to use it to init something with fifty daemons. But it's still pretty good, and it's a good learning tool, IMHO. SteveT Steve Litt Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: staggering runsv startup
On Fri, 05 Jun 2015 00:10:05 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: What you really want is a real service manager that works on top of a process supervision system and that would managed a complete, ordered initialization sequence for you. Steve is saying that process supervisors are lacking real service management capabilities, and he's right. Process supervision does not offer service management; service management is more complex and one layer above. I'm not familiar with the definitions of service managers and process supervisors, but the simple ability to declare the order of initial, bootup startup, would go a long way. I'm not saying such ordering would free me from needing to check for required other services being ready in this service's run script. I'm just saying that ordering would eliminate the tendency of such checks to result in one service coming up per cycle around /service, and would produce reasonable boot times, because in most cases teh required services *would* be up and functioning by the time the service that needs them is started. There are some tools to accomplish service management on top of process supervision. One that I like is anopa: http://jjacky.com/anopa/ but it's designed to work with s6, not runit. I'm also working on a service management system for s6, that should hit beta soon. This is very good news. Ordering is sorely needed in the daemontools world, and I think your new S6 service management system would be the first to do that without kludges like I enumerated in other emails. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: staggering runsv startup
On Thu, 04 Jun 2015 13:41:12 -0700 Jameson Graef Rollins jroll...@finestructure.net wrote: Hi, all. I am using runit to supervise a large set of nearly identical processes. Each process accesses certain IO-bound shared resources (e.g. NFS mount) at startup. At system initialization, when runsvdir is launched, it launches all these processes (via runsv) essentially simultaneously. This causes a big resource contention at initialization that occasionally causes problems. What I would like is to somehow stagger the startup of the processes, to avoid the resource contention. I could do this by putting a random sleep into the ./run scripts, but this would also cause random startup delays on subsequent process restarts via sv restart or the like (which we occasionally need to do). What I would prefer instead is to add random delays to the startup of the *runsv* processes, since this would only apply at system initialization. Unfortunately I can't see any way to do that right now (other than somehow wrapping the runsv binary itself). Does anyone know any way to accomplish what I'm looking for? I don't believe runsvdir supports any options that would apply here. Is it possible to somehow point runsvdir to a alternate runsv executable to which I could add the random delays? Any suggestions would be much appreciated. Thanks. jamie. In the 1 script, put down files in every directory under /service except one called order. Then, the order process is called, and it erases the down file from one at a time, sleeping 1 second after each. When all of them have been undowned, have order put a down file in its own directory and stop itself. The one thing I dislike about daemontools type process managers is you can't order the processes. SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: staggering runsv startup
On Thu, 04 Jun 2015 14:44:38 -0700 Jameson Graef Rollins jroll...@finestructure.net wrote: On Thu, Jun 04 2015, Lucy Pseudonym ne.tet...@gmail.com wrote: You can create `down` files in the service dirs as described in [1] and enable the services from a script at boot time. Hi, Lucy. That's an interesting suggestion. It would require building out more/other infrastructure, though, which is something I was trying to avoid. I'm also already using the `down` files to control which services are started at boot. If I did what you suggest it would require managing the startup list in some other way. Thanks for the suggestion, though. It is an idea I hadn't considered. jamie. put reallydown files in services you truly intend to be down, and have the script that deletes them not delete them if there's a reallydown file. Obviously this is kludge city, but I think what you really need is ordering of the managed services, and as far as I know, no daemontools inspired program does that in a straightforward manner. SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: dependant services
On Mon, 08 Jun 2015 21:08:38 +0100 Jonathan de Boyne Pollard j.deboynepollard-newsgro...@ntlworld.com wrote: The systemd dictum is that to truly take advantage of parallel startup, one eliminates orderings as far as possible. Which is where socket activation comes in. Part of socket activation is systemd opening server sockets early, and passing them to the server processes that get run. Because clients depend from the availability of the sockets, rather than from the availability of the final services, clients and servers can actually start in parallel, and the client is *not* declared as dependent from the *service* being up. Eeeeuuu! Am I the only one here who is grossed out by the preceding paragraph? A socket activated init has no idea who programmed the server, or what idioms that programmer used. Telling clients here's your socket, we're sure that it works sounds a little like check's in the mail or we come in peace. Oh, wait, will there be a systemd compliance sticker given only to servers who do it the systemd way? Nice! Jonathan, am I hallucinating, or does your paragraph basically say that system activation depends on an assumption? Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Shutdown/reboot script order?
Hi all, I know the last part of system shutdown or reboot, after I've killed my daemontools-supervised processes, has the following features: * killall5 -15; sleep 2; killall5 -9; sleep 2 * umount -a * mount -o remount,ro /dev/sda1 * /sbin/halt or /sbin/reboot Could somebody confirm which order these things come in? Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: First thoughts on s6
On Wed, 17 Jun 2015 16:32:42 -0700 Avery Payne avery.p.pa...@gmail.com wrote: With yesterday's mention of the idea of a clearinghouse, the concept that daemon options need to be versioned has been in the back of my head. Here is an idea: versioned definitions. A crude example: /etc/sv/sshd/envdir/DAEMON - /etc/sv/sshd/envdir/current/DAEMON /etc/sv/sshd/envdir/DAEMONOPTS - /etc/sv/sshd/envdir/current/DAEMONOPTS /etc/sv/sshd/envdir/current- /etc/sv/sshd/versions/v3 /etc/sv/sshd/envdir/versions/v3/DAEMON /etc/sv/sshd/envdir/versions/v3/DAEMONOPTS /etc/sv/sshd/envdir/versions/v2/DAEMON /etc/sv/sshd/envdir/versions/v2/DAEMONOPTS Settings are now versioned and encapsulated to their definition; old definitions can be kept indefinitely, allowing you to support older legacy installs; you can still pull just one definition of /etc/sv/sshd (and all of the files and directories inside of it) out of a tarball, put it in place, and everything works; You or somebody had better document that clearly and simply. Maybe with a diagram or two. There are two possible thoughts an unfamiliar user could have when seeing all those symlinks: 1) Too crazy, later days! 2) What a cool way of representing everything, *I* should have thought of it! I think the difference between those two reactions is about a half a page of text and maybe a diagram or two. If you incorporate this system, when the time comes, send me back this email to remind me, and I'll help with the document. By the way, Avery, that's a slick solution to a real problem. Nice one! SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: Shutdown/reboot script order?
On Tue, 09 Jun 2015 18:08:08 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 09/06/2015 16:24, Steve Litt wrote: * killall5 -15; sleep 2; killall5 -9; sleep 2 * umount -a * mount -o remount,ro /dev/sda1 * /sbin/halt or /sbin/reboot Could somebody confirm which order these things come in? You got the right order. Note that killall5 is heavy, kludgy and System V-specific. Does the preceding sentence mean that killall5 is available only when sysvinit is installed? That could be a problem. If your shell has a kill builtin, you can simply replace killall5 -15 with kill -15 -1, and killall5 -9 with kill -9 -1. It will work on Linux. If your shell does not have a kill builtin, you're out of luck, unless your script is running as process 1. I think I could theoretically change Suckless-Init to run /bin/rc.shutdown with an exec so it *is* pid1. I think. This is one of the reasons why I advocate running the last part of the shutdown procedure as process 1 - it makes the final nuke easy and portable. How would one run a shutdown script as a PID higher than 1, then magically change back to PID1? Would you run 2 different shutdown scripts: one for the standard stuff, and one for the killalls? Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: What interrupt does Ctrl+Alt+Delete send PID1?
On Thu, 11 Jun 2015 10:04:13 -0700 (PDT) Robert Hencke robert.hen...@gmail.com wrote: On Thu, Jun 11, 2015 at 12:59 PM, Steve Litt sl...@troubleshooters.com wrote: Hi all, What interrupt does Ctrl+Alt+Delete send PID1? Google just tells how to tweak /etc/inittab to react to Ctrl+Alt+Del, but in my case /etc/inittab isn't being used: I need to react to the specific interrupt. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key I believe this is SIGINT. Thanks Robert, I have a feeling that SIGINT isn't getting all the way to PID1. When I do the following: kill -s SIGINT 1 It runs rc.shutdown with the argument reboot and reboots the system, just like it's supposed to. But when I press Ctrl+Alt+Del at a virtual terminal, it doesn't. That makes me suspect that either Ctrl+Alt+Del is sending a different interrupt to PID 1, or the SIGINT it's sending isn't getting to PID 1. I'll try to research if further. Thanks again, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
s6 ordering and run-once?
Hi all, I'm studying the s6 website. The only reference I found to controlling startup order was a sentence about creating symlinks one by one. I found a sentence about run-once types of things (I think the example was bringing up the network), but then it didn't tell me *how* to run a run-once. Does anyone know if there are ways to order beyond one symlink at a time or s6-svwait? Does anyone know how to do a run-once service without putting an infinite sleep loop at the end? Also, are there any flag files, other than down, of which I should be aware? Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: comparison
That's an understatement! And, as the old saying goes, the best time to plant a tree is 20 years ago. The second best time is today. Thanks Avery! SteveT On Tue, 16 Jun 2015 13:09:32 -0700 James Powell james4...@hotmail.com wrote: And supervision-scripts has been that generic profile that can be used in 99% of situations. Sadly, Avery, I wish we could have had your work about 8 years ago. The UNIX world might have been a vastly different place. Sent from my Windows Phone From: Avery Paynemailto:avery.p.pa...@gmail.com Sent: 6/16/2015 11:26 AM To: supervision@list.skarnet.orgmailto:supervision@list.skarnet.org Subject: Re: comparison On 6/16/2015 5:22 AM, James Powell wrote: Very true, but something always seems to say something along the lines of if we had done #2 years ago, we might have avoided a huge mess that now exists. Agreed. The same applies to init systems. If there are ready to use feet wetting, taste testing scripts ready to go, the job of importing things just gets easier on the distribution. Also agreed. Actually, there's some discussion on the mailing list from a few months back about this. From: Steve Littmailto:sl...@troubleshooters.com Sent: 6/16/2015 4:45 AM To: supervision@list.skarnet.orgmailto:supervision@list.skarnet.org Subject: Re: comparison On Tue, 16 Jun 2015 04:05:29 -0700 James Powell james4...@hotmail.com wrote: I agree Laurent. Though, even though complete init+supervision systems like Runit exist, it's been nearly impossible to get a foothold with any alternatives to sysvinit and systemd effectively. I think one of the major setbacks has been the lack of ready-to-use script sets, like those included with OpenRC, various rehashes of sysvinit and bsdinit scripts, and systemd units just aren't there ready to go. The true problem is that each daemon needs its own special environment variables, command flags, and other gobbledygook that is specific to getting it up and running, and a master catalog of all settings doesn't exist. Compounding that is the normal and inevitable need for each supervision author to do their own thing, in their own way, so tools get renamed, flags get mapped, return codes aren't consistent. That's just the framework, we haven't talked about run scripts yet. Who wants to write hundreds of scripts? Each hand-cobbled script is an error-prone task, and that implies the potential for hundreds of errors, bugs, strange behaviors, etc. This is the _entire_ reason for supervision-scripts. It was meant to be a generic one size fits most solution to providing prefabricated run scripts, easing or removing the burden for package maintainers, system builders, etc. All of the renaming and flags and options and environment settings and other things are abstracted away as variables that are correctly set for whatever environment you have. With all of that out of the way, it becomes much easier to actually write scripts to launch things under multiple environments. A single master script handles it all, reduces debugging, and can be easily swapped out to support chainload launchers from s6 and nosh. The opposite end of this is Laurent's proposal to compile the scripts so they are built into existence. If I'm understanding / imagining this correctly, this would take all of the settings and with a makefile bake each script into existence with all of the steps and settings needed. It would in effect provide the same thing I am doing but it would make it static to the environment. There's nothing wrong with the approach, and the end result is the same. The only difference between Laurent's approach and mine, is that Laurent's would need to re-bake your scripts if your framework changes; in my project, you simply run a single script and all of the needed settings change on the fly. I'm not sure of the pros/cons to either approach, as I would hazard a guess that any system switching between frameworks may also require a reboot if a new init is desired. Here's the rub: in both cases, the settings for each service/daemon/whatever are key to getting things running. Again, we come back to the idea of a master catalog of settings. If it existed, then half of the problem would be resolved. There are lots of examples out there, but, they're not all in one place. So I try to toil over supervision-scripts when I get time, and make that catalog. Even if people don't like what I'm doing with the master run script itself, that doesn't matter. *What matters is that I've managed to capture the settings for the various daemons, along with some annotations*. Because I took the time to support envdir, and the settings for each daemon are stored in this format, those settings can be extracted and used elsewhere. I'm slowly creating that master catalog in a plaintext format that
First thoughts on s6
Hi all, I have a proof of concept s6 running, *supervision only*, on a Plop Linux VM. It is started by sysvinit (inittab), and supervises tty5 just fine, including logging (which is how I found out my run command originally had bad syntax). Here are some first impressions... Plop Linux has one of those /dev/random generators that just stops after a few characters. This caused the ./configure on skalibs to lock up while checking for /dev/random (thank you very much for the clear messages). In fact, /dev/random on this VM locks up dd too. I strongarmed the problem by removing /dev/random and making a hard link called /dev/random to /dev/urandom. What the heck, it's an experimental VM I can throw away, so any side effects this caused probably won't cause me a whole lot of trouble. I'd suggest a ./configure option called --urandom-instead, because this isn't the first time I've seen a /dev/random act like this, and probably most people don't want to pull the strongarm I did (I wouldn't either if it were a real machine). My tty5 takes 1 second from Ctrl+D til the time the next login prompt is printed. Is this a feature, or a bug? The following two commands worked *very* differently: * cat /service/tty5/log/main/current | tai64nlocal * cat /service/tty5/log/main/current | tai64n The former command worked as expected and printed out /mm/dd plus time timestamps. The latter command appeared to print out two different tai64n timestamps, both fairly close to each other but different. I think the following additions would make the s6 website a lot easier to use, and a lot easier for somebody to begin using s6: * FAQ * A page with simple files, including run and log/run for a tty. Here are the run files I used for tty5: For the service: #!/bin/sh exec 21 echo Starting tty5 2 exec /sbin/getty tty5 For the tty5 log: #!/bin/sh exec s6-log t ./main They seemed to work well. Anyway, so far so good. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: What interrupt does Ctrl+Alt+Delete send PID1?
On Fri, 12 Jun 2015 11:31:44 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 12/06/2015 02:54, Steve Litt wrote: * Whole thing running in a VirtualBox VM Ah, I'm not sure how all this plays with virtualization. VirtualBox looks like a full virtualizer with a separate kernel for the guest, so it probably makes no difference, but if your guest kernel is also your host kernel and it's container-based virtualization, then pid 1 rules may change completely. I'm not familiar enough with virtualized systems or containers to know exactly what happens in that case. By the way, my Plop Linux boots up such that /proc/sys/kernel/ctrl-alt-del has value 1. Changing it to 0 doesn't enable the three finger salute to run /bin/rc.shutdown reboot. Well, your host system will definitely catch the 3 finger salute; it won't be sent as is to your guest system. Is there a way for the VM to send an emulated ctrl-alt-del press to the guest ? Thanks Laurent, It looks like once I fully document my VM setup, I'll need to do the thing on bare metal. Fortunately I have an experimental machine in the next room. I'll let you know the results --- probably in about a week from now. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Suckess Init + daemontools-encore + LittKit
Hi all, This email isn't meant to imply a competition with any other init systems or supervisors. Its main value is as a learning tool, so that if you meet people who don't really understand init but want to, you can show it to them. Here is where I document initting a Plop Linux system with a combination of Suckless Init plus daemontools-encore plus LittKit: http://troubleshooters.com/linux/diy/suckless_init_on_plop.htm I can tell you that, after performing the activities on that web page, I understand a heck of a lot more of the conversations I hear on this list. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: comparison
On Tue, 16 Jun 2015 09:29:15 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: In the meantime, if you don't want to get your hands dirty, you can still use s6-svscan/s6-supervise as a process supervision system without trying to make it run as an init system, just as you can use runsvdir/runsv as a process supervision system without using the runit binary. That is real modularity, that is the main reason why I believe runit and s6 are better designed than other-init-software-out-there, and it would be *trivial* to port your suckless init on Plop setup from daemontools(-encore) to runit or s6, even if you don't use every feature those packages provide. You could be right about that. I just finished doing the same setup, except instead of Suckless-daemontools-encore, I did Felker-s6-svscan. Once I cleaned up all my transposition errors, and modified LittKit to work with S6's command names and s6-svstat's output, it worked like a charm, and it appears to work just like Felker-daemontools-encore. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: Describing daemontools in 300 words?
On Sat, 13 Jun 2015 04:35:09 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 13/06/2015 03:19, Steve Litt wrote: When I tried S6 about 8 months ago (and I knew a lot less then), I was unable to get it installed and running. I already asked, and will ask again: please try it again. Since the last time you tried, there has been in particular: - the Christmas 2014 update, which has simplified installation a lot; the goal was precisely to make the software more accessible. - http://skarnet.org/software/s6/overview.html which I asked you to give me your opinion about but you never replied. I'm in a much better position to give you an opinion now: * I'm much more knowledgeable about inits than in 10/2014 * I now have a computer with hardware VM accelleration, makes all the difference in the world. * I've discovered Plop Linux, a bare bones, almosts transparent distro that's very easy to work on. I'll try very hard, within the next 2 weeks, to lay down an S6 init on a VirtualBox VM loaded with Plop Linux. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Describing daemontools in 300 words?
Hi all, If you had to describe daemontools, in the context of a general purpose process manager, to a technical person, in 300 words, what things would you say and what things would you leave out? Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: What interrupt does Ctrl+Alt+Delete send PID1?
On Fri, 12 Jun 2015 20:06:40 -0300 Guillermo gdiazhartu...@gmail.com wrote: 2015-06-12 9:44 GMT-03:00 Steve Litt: On Fri, 12 Jun 2015 11:31:44 +0200 Laurent Bercot wrote: Well, your host system will definitely catch the 3 finger salute; it won't be sent as is to your guest system. Is there a way for the VM to send an emulated ctrl-alt-del press to the guest ? Thanks Laurent, It looks like once I fully document my VM setup, I'll need to do the thing on bare metal. Did you use the right key combination? You send an emulated CTRL + Alt + Del to the guest OS on VirtualBox with HOST Key + Del. HOST Key is the right CTRL key by default, but you can change that via VirtualBox configuration. In other words, if you don't change the HOST key mapping, the emulated CTRL + Alt + Del is right CTRL + Del. A real CTRL + Alt + Del will indeed go to the host OS. Cheers, G. Thank you Guillermo, My daughter makes costumes for plays, and I've commissioned her to make me a dunce cap. Yes, of course Host+Del reboots the virtual machine. If I do this: echo 0 /proc/sys/kernel/ctrl-alt-del then Host+Del reboots by calling my /bin/rc.shutdown. If I leave the ctrl-alt-del file at its booted value of 1, it just skips /bin/rc.shutdown and brings it right down. To summarize: * I blamed it on a wrong interrupt, but that was false. * Then I blamed it on Suckless Init, but that was false. * Then I blamed it on Plop Linux, but that was false. * Finally, thanks to your guidance, I know the problem is defective user, and fixed the problem. I'll think of you often as I wear my dunce cap, and thanks so much for putting me out of my misery. This could have gone on for days. LOL, you can't make this up, folks! Thanks Guillermo, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: Describing daemontools in 300 words?
On Sat, 13 Jun 2015 02:27:12 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 13/06/2015 02:21, Steve Litt wrote: If you had to describe daemontools, in the context of a general purpose process manager, to a technical person, in 300 words, what things would you say and what things would you leave out? At this point I have to ask: are you using daemontools as a generic name for supervision suites, or as the original suite from DJB? If the latter, why are you advertising it over s6 or runit, which provide the same functionality but are actually maintained ? Good question. For my current, almost finished project, I need a process manager, not a whole init system. Daemontools works every time and I know it well. Same with daemontools-encore, which I'm getting to like better than the original. I haven't used runit for awhile, but as I remember it's more complex, possibly because I used it as a whole init. When I tried S6 about 8 months ago (and I knew a lot less then), I was unable to get it installed and running. As far as nosh, once I compiled it I didn't know what to do with it, and the nosh language remains an enigma to me, although I understand I can program its runscripts in sh also. (I'm not asking out of spite, I'm asking because if there's something you find in daemontools that you don't find in s6, I want to add it. :-) I think you'd have to subtract it. Daemontools and daemontools encore are simple and have a very small number of features. And I completely understand what they do. For instance, when you and Gerrit get to discussing whether /etc/runit/1 should have the same PID as /etc/runit/2, I try to follow both your logics (I can't remember right now which supports which option), but I can't keep up. With, let's say, daemontools-encore, I know I'm dealing with the equivalent of /etc/runit/2, and my Suckless Init is /etc/runit/1. I just understand it better, and in my case they're separate PIDs (and no, right now Suckless Init doesn't REspawn daemontools-encore). You know what it is? Daemontools-encore is a better learning tool because it's simpler. When I get ready to actually lay down a real init on my daily driver desktop, it won't be Suckless Init with daemontools encore and LittKit, it will be runit or S6 or possibly Epoch (which isn't a daemontools inspired init). I've gone far to make sure s6 was an improvement over daemontools in every possible way, and if I have missed something, I want to know.) :-) I think you've missed that it's hard to make a superset and keep it as easy for learning as the subset. It's just the nature of the beast. So let me rephrase with something that *is* being maintained: How would you describe the process manager (not init) daemontools-encore, to a technical person, in 300 words. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: Readiness notification for systemd
On Sat, 13 Jun 2015 20:48:15 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: Hello, A part of systemd's Embrace And Extend strategy is to entice daemon authors to link against the systemd library (and thus make every possible daemon systemd-dependent) by pretending it is the only way for them to notify readiness. As a step to obstruct that, I have written sdnotify-wrapper. It's a wrapper for daemons using the simple write a newline readiness notification mechanism advertised by s6, which converts that notification to the sd_notify format. So, it makes it possible for daemons to *not* use the sd_notify() interface and still be properly managed under systemd. http://skarnet.org/software/sdnotify-wrapper.c And now the questions begin :-) When I write my own daemons (for use with daemontools, daemontools-encore, runit, and soon to be S6), I use stdout to write to the log. Wouldn't the writenewline/close interfere with that? How do I take advantage of sdnotify-wrapper if I'm using S6? How do I take advantage of sdnotify-wrapper if I'm using daemontools-encore? I really like the fact that you've taken the fight to systemd. By all means, let *them* add the extra word in their unit files. SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: Readiness notification for systemd
On Sat, 13 Jun 2015 22:35:13 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 13/06/2015 21:46, Steve Litt wrote: When I write my own daemons (for use with daemontools, daemontools-encore, runit, and soon to be S6), I use stdout to write to the log. Wouldn't the writenewline/close interfere with that? It's more idiomatic to actually use stderr to write to the log. stderr is traditionally the place where I write diagnostic messages for an external observer, and stdout is traditionally the place where I write stuff as a part of my workflow. Logs fall in the former category, not in the latter; and daemons usually have no use for stdout. You could arguably call it a misdesign in daemontools (repeated in runit and s6) that the logging pipe is tied to the daemon's stdout, not its stderr, by default; fortunately, it's very easy to fix in the run script. Almost all of my run scripts start with fdmove -c 2 1, which redirects stdout to stderr (the shell equivalent would be exec 21). That's the behaviour I would also recommend for your daemons. That said, sdnotify-wrapper, as well as s6-notifywhenup, comes with a -d option that allows you to choose what descriptor the notification will be read on. So if you're dead set on logging to stdout, you can write your readiness newline to descriptor 3, for instance, and document this; the admin wanting to run your daemon will simply give the -d 3 option to the wrapper. How do I take advantage of sdnotify-wrapper if I'm using S6? You don't use sdnotify-wrapper, because you're already using a kickass supervision suite and have no need for crutches. :) How do I take advantage of sdnotify-wrapper if I'm using daemontools-encore? You don't. You only use sdnotify-wrapper if you're an administrator using systemd to manage a daemon that refuses to use sd_notify() and uses the write a newline to a file descriptor notification mechanism instead. The point of the wrapper is to prevent having to include systemd code in the daemon's code to make it systemd-compatible. If you've never been tempted to use sd_notify() in the code you write, and if you are not using systemd as a supervision framework, then you can quietly ignore everything about it. :) That's easy enough. One problem though: It seems like everything I write to stderr ends up in my readproctitle display in ps. SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: comparison
On Mon, 15 Jun 2015 17:37:45 -0700 Buck Evan b...@yelp.com wrote: Is there any resource that compares the capabilities of daemontools, daemontools-encore, runit, s6, and friends? One thing I can tell you is that daemontools and daemontools-encore were never intended to be init systems, whereas I'm pretty sure that runit, s6 and nosh intended to be part or all of an init system. In fact, I used runit as a complete init system, and it worked well. Regardless of the intent of daemontools and daemontools-encore, I just this week used daemontools to do almost all the system initialization for Plop Linux, and all the process management. PID1 was just an 89 line C program that forked /bin/rc.init, then wait() spun killing zombies and handling relevant signals. /bin/rc.init set the computer up enough that daemontools-encore would work, and then exec'ed daemontools-encore. HTH, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: comparison
On Mon, 15 Jun 2015 17:37:45 -0700 Buck Evan b...@yelp.com wrote: Is there any resource that compares the capabilities of daemontools, daemontools-encore, runit, s6, and friends? One more thing: It's been my subjective impression that daemontools-encore is a lot easier to install, set up, and keep running than original daemontools. SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: comparison
On Tue, 16 Jun 2015 09:29:15 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 16/06/2015 04:54, Steve Litt wrote: One thing I can tell you is that daemontools and daemontools-encore were never intended to be init systems, whereas I'm pretty sure that runit, s6 and nosh intended to be part or all of an init system. You keep saying that, but at least in the case of runit and s6, it is inaccurate. runit and s6 are process supervision systems, just like daemontools, and can be used with any init system. There are documentation pages that prove it: http://smarden.org/runit/useinit.html http://skarnet.org/software/s6/s6-svscan-not-1.html Yes but: http://smarden.org/runit/replaceinit.html#sysv http://skarnet.org/software/s6/s6-svscan-1.html There's no way I've found to do init=/any/file/of/daemontools(-encore) That's one thing. When you say s6 is more complex than daemontools, it's only more complex because there are more binaries, and more features, That's what I meant. s6 is a superset of daemontools, making it a little harder *for me, personally* to find my way around. Also, IIRC I couldn't compile s6 the last time, but the last time I knew nothing about the whole idea of init, so let me suspend saying it's more complicated until I've logged some time actually using s6. [snip] Now, the thing is, when you have a process supervision system, it makes your stock init redundant. init has two jobs: A. reap zombies Do you include listen for and handle relevant signals in reap zombies? B. maintain at least one other process alive, so that if everything on the machine is killed (save init itself), it is still possible to recover and avoid a hard and dirty reboot. Note that suckless init, or Rich Felker's suggested init in the otherwise excellent http://ewontfix.com/14/ article, do not perform B, and so, are *incorrect*. The error is minute, and probably inconsequential in any real life situation, but it is still there; and if you want the smallest possible pid 1 that will keep your machine usable under the most dire circumstances, you should not use suckless init, you should use runit. Yes. Someday I might change Suckless Init so that it *respawns* (supervises, whatever you want to call it) /bin/rc.init. I would also need a way that such a modified Suckless Init can pass along the information that this isn't the first time this boot, in case there are tasks I don't want to redo in /bin/rc.init. [snip] And neither I nor Gerrit did it first; the first one was Paul Jarc, who pioneered the setup using... daemontools. http://code.dogmap.org/svscan-1/ I'd forgotten about that article. Last time I read it, I didn't understand it at all. This time, when I read it, it reminded me that my simple /bin/rc.shutdown stage 3 means I'm short circuiting the logs during final shutdown. Yes, daemontools was never intended to be an init system, but even back then there was interest in running it as such, and Paul's experiments are what started it all. The problem with init is that it's not only stage 2, and stage 1 and 3 are heavily system-dependent; so making svscan work as init is possible, but requires good hacking skills. Also, even after you get into stage 2, those services that do system initialization require hacking skills. Gerrit recognized that, so he basically forfeited the idea - instead of using runsvdir as process 1, he created the minimal amount of pid 1 code necessary to handle stages 1 and 3, as well as any stage 2 (and runsvdir is the obvious choice for the stage 2 manager). runit is a basic process supervision system (runsv/runsvdir) with a simple init wrapper (runit). I did not want to give up on the idea; I designed s6-svscan so that it would be easier to run as pid 1 than daemontools' svscan, and managed to get something working reproducibly and in an almost automatable way, but as you experienced, the setup still requires some hacking skills, and I need more time to work around the non-portability issues and come up with a full turnkey init. In the meantime, if you don't want to get your hands dirty, you can still use s6-svscan/s6-supervise as a process supervision system without trying to make it run as an init system, just as you can use runsvdir/runsv as a process supervision system without using the runit binary. That is real modularity, that is the main reason why I believe runit and s6 are better designed than other-init-software-out-there, and it would be *trivial* to port your suckless init on Plop setup from daemontools(-encore) to runit or s6, even if you don't use every feature those packages provide. I'll be doing an s6 install in the next few weeks. Thanks, SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: What interrupt does Ctrl+Alt+Delete send PID1?
On Fri, 12 Jun 2015 02:35:28 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: There was a time where the kernel protected pid 1 against signals sent by user processes, and only sent it signals it generated itself such as SIGINT for Ctrl-Alt-Del. This is the reason why sysvinit uses /dev/initctl as a communication point between init and the shutdown program. Nowww you tell me! :-) Neither my Plop with Suckless Init, Plop with sysvinit, nor Plop live CD have /dev/initctl. Perhaps not so coincidentally, none of the three responds to Ctrl+Alt+Del. This wasn't a Suckless Init property, it appears to be a Plop property. SteveT Steve Litt June 2015 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: anopa 0.2.0
On Wed, 12 Aug 2015 13:50:19 +0200 Olivier Brunel j...@jjacky.com wrote: Hi Steve, On 08/11/15 20:40, Steve Litt wrote: On Tue, 11 Aug 2015 19:58:59 +0200 Hi j, I just got done reading http://jjacky.com/anopa/ . I love the inclusion of the needs directory, as well as directory after. I love the way you distinquish one-shots from long-run by whether your script is start or run. Reading http://jjacky.com/anopa/ , it seems like all the one-shots are run by PID1, whereas all the long-runs are run by s6. Does this mean that all one-shots must be run before the first long-run? If so, I think that's a too-strict restriction. If not, Anopa + s6 sounds tremendously encouraging. So, does Anopa give one a way of running a long-term before a one-shot? Of course. oneshots are run as children of aa-start, which on usual fashion during boot would happen from stage2 (aa-stage2 that is). Basically this is how it goes: - you use aa-stage1 as init, so it will be your PID1 on boot. It will fork exec aa-stage2, which will block using the trick Laurent described in s6 doc. - back in PID1, we exec into s6-svscan, which will then start the catch-all logger, thus unblocking aa-stage2 - aa-stage2 will then exec aa-start, where oneshots longruns can be started in any required order; so oneshot services can require for longrun services to be started, or ready, before they're started. Cheers, -j Thanks j, I couldn't *precisely* visualize your answer, but one thing came through loud and clear: === where oneshots longruns can be started in any required order; so oneshot services can require for longrun services to be started, or ready, before they're started. === This is *exactly* what I was hoping to hear. It sounds to me like theoretically anopa plus s6 will be a much better alternative than Suckless Init plus daemontools-encore[1] plus LittKit (http://troubleshooters.com/linux/diy/suckless_init_on_plop.htm), and because it can intermix one-shots and long-runs[2], better than runit. I'll be trying it out very soon. If it's what I think it is, I've been waiting a long time for something like anopa plus s6. [1] With a few changes to LittKit, you can use s6 instead of daemontools-encore [2]: I know it's likely I'll never need to run a one-shot *after* a long-run, but I want to have that capability. Runit doesn't give me that capability. Thanks, SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: s6: compatibility with make3.82
On Tue, 21 Jul 2015 23:39:31 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: And wow, my requirement is that the GNU make package be less than two years old, as opposed to eight years old. I had no problem compiling s6, and had no dog in the fight until reading the preceding general statement. Here's my response to the less than two years old requirement, penned long before I knew you, so please don't take it personally: http://www.troubleshooters.com/linux/politics_of_dependencies.htm IMHO two years is way too stringent a requirement. SteveT Steve Litt July 2015 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Re: OpenRC now supports daemon supervision using s6
On Sat, 25 Jul 2015 22:07:33 +0200 Laurent Bercot ska-supervis...@skarnet.org wrote: On 25/07/2015 21:43, Guillermo wrote: I was recently looking at OpenRC's source tarball to see how it implemented something, and was surprised by this little discovery: Nice finding! Thanks for the experiments. I didn't do a lot of reading, but it sounds to me like this solves one of OpenRC's two disadvantages: The fact that it can't respawn. For all I know it might amelliorate the other problem: insane init script complexity, by putting more of the details in the s6 run script. Me, I'd just go with s6 out of the box, but I guess OpenRC is more corporationally correct or something like that, and as such, I'm glad it can use s6 to do the supervision (and sysvinit to do the PID1, you can't make this up, folks). SteveT Steve Litt July 2015 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
How to alt-init Lubuntu?
Hi all, I tried to alt-init Lubuntu 15.04, and as a starting point, added init=/bin/bash to see where I would pop up and what I had to do. It came to a subset shell that looked something like busybox, and it appeared to me I was still in the initramfs. I tried using switch_root to mount the root partition, but for some reason (a week ago, I dont remember) that didn't work either. I tried removing the initrd= line from grub, and putting noinitrd on the kernel line, but that just kernel panicked. It was like before the initramfs this thing didn't have the drivers to read an ordinary ext4 partition. Have any of you had experience with situations like this, and what steps did you take to figure out how to alt-init it. Thanks, SteveT Steve Litt July 2015 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Re: Safely replacing /etc/s6-rc/compiled
On Thu, 12 Nov 2015 18:59:30 +0100 Jan Bramkamp <cr...@rlwinm.de> wrote: > Is there a way to update /etc/s6-rc/compiled without risking a broken > system if the power fails midway through the update? Couldn't you just back up the file(s), and if something goes wrong, boot back in with System Rescue CD and copy the backups back to original? SteveT Steve Litt November 2015 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques
Re: [ale] systemd talk from July has slide deck online now
On Mon, 7 Sep 2015 14:59:08 -0400 Jim Kinney <jim.kin...@gmail.com> wrote: > On Sep 7, 2015 1:43 PM, "Steve Litt" <sl...@troubleshooters.com> > wrote: > > > On Mon, 07 Sep 2015 11:21:24 -0400 > > Jim Kinney <jim.kin...@gmail.com> wrote: > > > > > http://ale.org/?p=632 > > > > > > The slides are now added to the bottom of the announcement page. > > > > Hi Jim, > > > > I think for your next presentation, you should either get rid of all > > comparisons to sysvinit and Upstart, or include comparisons to > > runit, OpenRC, Epoch, Suckless Init plus s6 plus LittKit, Suckless > > Init plus daemontools-encore plus LittKit, and the soon to be > > published s6-rc. > > > > If your presentation is simply "how to use systemd", you need no > > comparisons. If your presentation includes "why systemd is better", > > it would be misleading not to include the many excellent init > > systems available. > I am unfamiliar with those. Do they all start Linux? > > My goal was more to show what systemd does different from prior Linux > startups for the major distros. Hi Jim, Fair enough. Still, your comparison to only upstart and sysvinit could be misconstrued to mean it's the best init system. In fact, I'm pretty sure the systemd industry and fans will quote your presentation as proof of systemd being the best init. Here are some of the many good inits: * runit: Serves as both PID1 and typical daemontools-style service manager. * s6: Very advanced daemontools-style service manager. Requires some other PID1. Sysvinit will fill that bill (without any of the init scripts: Just one line in /etc/inittab). Personally, I used Suckless-Init to implement PID1, and LittKit to provide deterministic startup order of services. * s6-rc: This is coming out this month: I haven't used it. From what I understand, this has raised the bar by combining a top quality PID1 with the s6 service manager. * nosh: Another PID1 plus daemontools-style service manager. Its runscripts require a special language, I was unable to compile it eight months ago. Judging from the many things I've heard its author saying, this should be an excellent init if you can get it running. * Epoch: Trivially easy init system with declarative config file instead of init scripts, run scripts, or unit files. If I need to alt-init a computer in two hours, I'll be using Epoch. * OpenRC: A sort of advanced version of what sysvinit should have been. I've used it a couple times. I understand it has the ability to do all the same stuff as systemd achieves with its socket-activation. Is not capable of automatically rerunning crashed services. * RichFelker + LittKit + daemontools-encore: If you want the simplest possible init: One that absolutely anybody can understand and troubleshoot, this is it. Includes a 16 lines-of-C PID1. You can replace RichFelker with Suckless-Init if you want your PID1 to listen for SIGCHLD, SIGINT and SIGUSR1 and do the right thing, substitute the 83 lines-of-C Suckless Init. You'll need to write your own shutdown script, which isn't particularly difficult. If you want to know what RichFelker PID1 is, see the bottom of this page: http://ewontfix.com/14/ * Busybox Init: I know little about it. USE CASES: The main use for OpenRC, IMHO, is when your distro already comes with it. Gentoo, Funtoo, Manjaro OpenRC. Runit and Epoch are high quality, small footprint inits easily installed on any Linux not tied down with a bunch of systemd-isms. Both are easy to install, easy to configure, both can rerun crashed processes (Epoch gives you more choice in the matter). Unless a 1 second bootup is vital to you, I can't forsee a use case not conducive to Runit and Epoch. Servers, desktops, it's all good. Runit also works with BSD. S6 is a top notch service manager that can be combined with a separate PID1 (sysvinit, Suckless-Init, systemd) to init almost any use case. Its claim to fame is reliability: You just can't kill it. With the soon to come s6-rc, it will be a complete init system of the highest quality. (RichFelker|Suckless-Init) + LittKit + any daemontools-style service manager is best when your top priority is academic simplicity. Conforms to http://ewontfix.com/14/. It can be used to alt-init pretty much any Linux OS not glued down with systemd-isms, and it will do a good job as such. In reality, these kinds of inits serve more as demonstrations: For a production machine you'd probably use runit or s6-rc instead. Busybox Init is used when the priority is tiny and simple. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: [ale] systemd talk from July has slide deck online now
On Wed, 9 Sep 2015 18:03:52 +0200 Laurent Bercot <ska-supervis...@skarnet.org> wrote: > >> I am unfamiliar with those. Do they all start Linux? > >> My goal was more to show what systemd does different from prior > >> Linux startups for the major distros. > > I was just made aware of that article: > http://blog.darknedgy.net/technology/2015/09/05/0/ > > It's a pretty comprehensive list of the init systems for Linux. Several of those inits required services to be configured in XML. Tell me one more time just so I understand: Why would *anybody* configure their services with XML, thereby requiring the init system to have a built in XML parser? XML parser in PID1, what could possibly go wrong? SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Some suggestions about s6 and s6-rc
On Sat, 19 Sep 2015 11:11:37 -0700 Avery Payne <avery.p.pa...@gmail.com> wrote: > With regard to having scripted placement of down files, if it was in a > template or compiled as such, then the entire process of writing it > into the definition becomes trivial or moot. While there should > always be a manual option to override a script, or the option to > write one directly, I think the days of writing all of the > definitions needed by hand have long since past. > > But there is an issue that would keep this idea from easily > occurring. You would need to find a way to signal the daemon that it > the system is going down, vs. merely the daemon going down. I solved all this stuff, with LittKit, by defining files "reallydown" and "nodown", signifying "yes, this service truly is supposed to be down", and "this is the first service to run", respectively. I've tested it with s6 and with daemontools-encore (slightly different versions of the shellscripts), it works perfectly. Basically, on startup, before bringing up the process supervisor, you write "down" files to every service not containing a "nodown". Then you erase down files one at a time. LittKit in no way requires any modification to the supervision system. If the supervisor does what original daemontools does with the "down" file, LittKit can bring up services (and oneshots) one at a time, intermixing services and oneshots. Here's the LittKit README: http://troubleshooters.com/projects/littkit/README I'm not saying nine little shellscripts is the best solution to the situation, but it's not all that tough a situation. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Some suggestions about s6 and s6-rc
On Sun, 20 Sep 2015 11:06:34 +0800 "Casper Ti. Vector" <caspervec...@gmail.com> wrote: > On Sun, Sep 20, 2015 at 12:33:28AM +0200, Laurent Bercot wrote: > > I agree that the name collision is confusing, and it is an > > annoyance. > > Since s6-rc is still unreleased, perhaps we can still take the chance > to rename `up'/`down' in oneshots to `run'/`finish', in order to let > them look a little more unified? > Yes. The use of a file called "down" to tell the system not to run the process, and also the use of a script called "down" to perform an action at the appropriate time, will be holy hell to document, even if theoretically they cannot both happen in the same script directory. Also, don't be sure that they can't happen in the same script directory. LittKit uses daemontools-style "down" files for both oneshots and longruns, and there are probably other workarounds out there that do the same thing. They'll all break upon the introduction of a script called "down". Breaking workarounds isn't a valid reason not to do something, but the documentation problem is. The confusion it causes will disuade a non-trivial number of people from working with s6-rc, all because of a (I think unfortunate) filename. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Some suggestions about s6 and s6-rc
On Sun, 20 Sep 2015 15:15:52 +0800 "Casper Ti. Vector" <caspervec...@gmail.com> wrote: > (Accidentally sent to Colin as private mail, reposting verbatim here; > sorry for the disturbance...) > > Well, this naming issue is all about overloading... To circumvent the > overloading problem, we can also use some other name pairs like > `start'/`stop' or `begin'/`end' (Gentoo and LaTeX user here ;). But > that introduces another problem: more names to remember. Which in turns calls for much more descriptive names. If I understand correctly, the current ./up and ./down are used for one-shots only. If these things are run at boot time and "stopped" at shutdown time, they could be called onboot and onhalt. Otherwise onrun and onstop. Or something more descriptive. > So this is somewhat a dilemma. Different context and developers might > lead to different decisions; I personally would rather like to support > overloading the `run'/`finish' name pair, for the similar reason for > Unix's choice of the `x' permission bit to represent both "executable" > for files and "enter-able" for directories: they are loosely > correlated concepts in orthogonal "subspaces". You know they're orthogonal in the expected use case. I know they're orthogonal in the expected use case. But the guy just learning this stuff doesn't know that intuitively, and unless the docs are written **just right**, it will cause confusion. Needless confusion. Confusion pushing the prospective user to another init system. Maybe systemd. I wasn't around when the decisions were made how to use the 'x' permission, but I do remember that in the early 80's (so certainly when the 'x' permission was specified), RAM was so dear. I had to write a report object, in C, because I just couldn't afford an 80x66 array of char. I have a feeling that RAM scarcity played a part in the duality of the 'x' permission, and I remember when I was new to Linux that 'x' permission's duality confused me for a little while. Overloading filenames for orthogonal uses strikes me as clever, but unnecessary and a documentation nightmare. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Built-in ordering
On Sun, 20 Sep 2015 10:49:13 -0700 Avery Payne <avery.p.pa...@gmail.com> wrote: > Regarding the use of ordering during "stage 1", couldn't you just > have a single definition for stage 1, run through it to set up > whatever is needed, then transition to a new system state that > doesn't include that defintion with (insert system management here)? > What I am trying to ask is if the down files are really necessary. If you're willing to have a stage1 and stage2, no, it isn't. I personally consider the existance of a separate stage1 and stage2 to be suboptimal, and think the ability to mix ordering of runonce and runlongs is a good thing that might be essential in some use cases. So on suckless-init plus daemontools-encore (or s6) plus LittKit, if I had any stage1 at all, it was just remounting / read-write, running LittKit lk_prepare script, and then running whatever daemontools-encore calls their svscanboot program. All the rest happens in what would normally be called stage 2. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Some suggestions about s6 and s6-rc
On Sun, 20 Sep 2015 11:26:00 +0200 Laurent Bercot <ska-supervis...@skarnet.org> wrote: > On 20/09/2015 07:30, Steve Litt wrote: > > Yes. The use of a file called "down" to tell the system not to run > > the process, and also the use of a script called "down" to perform > > an action at the appropriate time, will be holy hell to document, > > even if theoretically they cannot both happen in the same script > > directory. > > I don't think so. The confusion happened here because we're talking > in a vacuum; when you use the tool, it's clear enough. You'll see. That's my point exactly. Unfamiliar with the tool, you read the docs, keep reading about "down" in two different contexts, get confused, and say "later days." And then there's this: What is to be gained by using the same filename in two different contexts? Other than some global search and replace, what would be the cost of changing "./down" to something more descriptive? Earlier in this thread, you yourself said: > I agree that the name collision is confusing, and it is an annoyance. If *you* think it's confusing, imagine the poor guy wanting a systemd alternative not called "sysvinit", who has heard that s6-rc + s6 (do I have that right?) is the best there is? Or how bout the poor daemontools familiar admin who's heard that s6-rc + s6 is the best and it's daemontools-inspired, so he goes in with a long-established understanding of "down" files and encounters this? Giving this file a better name is a couple days of thought and an hour of global search and replace. I think it's well worth it. > > Also, don't be sure that they can't happen in the same script > > directory. LittKit uses daemontools-style "down" files for both > > oneshots and longruns, and there are probably other workarounds out > > there that do the same thing. They'll all break upon the > > introduction of a script called "down". > > They won't with s6-rc. A definition directory is not a service > directory. Your scripts will work, don't worry. OK, then I'm not correctly envisioning what s6-rc really is. SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Built-in ordering
Hi all, It just occurred to me that built-in ordering could be as simple as making the very first loop, on, for want of a better word, svscanboot, search for a directory with a file called "mefirst". Upon encountering that directory, it would execute that directory's run script, which would perform all the down file kludges or other ordering mechanisms that the individual user wants. Of course, if one is going to do an initial dummy loop, one could also look for a file called "order" in each, whose contents indicate a number to be sorted in order to get the startup ordering of all longruns and oneshots. But IMHO that would require *a lot* more change to the supervisor than simply looking for the first "mefirst" encountered (and it would be a user error to have more than one directory with a "mefirst"). Of course, all this could be avoided the LittKit way and just run a script depositing "down" files before running the supervisor. But the LittKit way specifically defies Laurent's proposal that PID1 be able to restart the supervisor (I think). SteveT Steve Litt August 2015 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Supervising a pipeline?
Hi all, I'm making a thumb drive automounter using inotifywait piped into my Python program, which detects the proper CREATES and DELETES and automounts and autoumounts accordingly. Here's what I thought the run script would look like (runit dialect): #!/bin/sh exec /usr/bin/inotifywait /dev/disk | /usr/local/bin/automounter.py However, it was pointed out to me that if automounter.py crashed, inotifywait would keep spinning and shooting its voluminous stdout messages into the ether. And presumably the service wouldn't crash and restart. Is this doable in a way consistent with supervision suites? I completely understand that 95% of you think what I've suggested is a no-style kludge that shouldn't be done, and that I should use Python's inotify framework. This is a completely different discussion: I'm limiting my question to whether such a pipeline is, or can be made, consistent with things like daemontools-encore, s6, and runit. Thanks, SteveT Steve Litt November 2015 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques
Re: Supervising a pipeline?
On Sat, 26 Dec 2015 18:25:22 +0100
Laurent Bercot <ska-supervis...@skarnet.org> wrote:
> On 2015-12-26 18:09, Steve Litt wrote:
> > #!/bin/sh
> > exec /usr/bin/inotifywait /dev/disk
> > | /usr/local/bin/automounter.py
>
> You can't supervise a pipeline per se; you need to supervise
> both processes in the pipeline independently, and make sure the pipe
> isn't broken when one of them dies.
I've verified that the preceding paragraph to be true. Killing the
Python program left the inotifywait running, and the service
non-restarted. That's unacceptable.
>
> So, have "exec inotifywait /dev/disk" as foobar/run, and have
> "exec automounter.py" as foobar/log/run. This will work with
> daemontools, runit and s6. (You can accomplish the same goal with
> perp and nosh too; the syntax will just be different.)
:-)
Congrats Laurent: You've suggested a kludge that the King of Kludges,
Steve Litt, cannot abide by. I understand it would probably work, but
I'm too much of a prude to use a facility, meant for logging, in this
way. Nice one though: The kludgicity is a thing of panoramic beauty. If
it had been my idea I might have done it just for that reason :-)
>
> Alternatively, you could use s6-rc and create the "inotifywait" and
> "automounter" longrun services in a pipeline; your compiled database
> would then include instructions to set up the supervised pipeline
> for you. This is more complex to set up than just using the integrated
> pipe management in svscan and runsvdir, but it's also more powerful,
> because you can pipeline an arbitrary number of processes that way
> (this is also what nosh does).
I'll keep the preceding in mind when contemplating:
exec a | b | c | d | e
But I can't begin to imagine what would be in the "compiled database"
to which you refer. I can't actually do this for the following reasons:
1) I have absolutely no idea how.
2) Documentation for deployment would be a nightmare.
3) My intended audience would not only laugh, but use such a setup as a
reason not to use my automounter.
At this point I should mention the pipexec described by post-sysv. This
looks promising in a different situation, but it would be a dependency
that would belie my automounter's claim to "simplicity."
So what I did was something like this, in the Python program:
proc = subprocess.Popen(['/usr/bin/inotifywait', \
'-m', '-r', '/dev/disk/by-id'], \
stdout=subprocess.PIPE, bufsize=1)
EOF = False
while not EOF:
line = proc.stdout.readline().decode('utf8')
if line == '':
EOF = True
else:
line = line.strip()
process_line(line)
print('\nCaller loop done, aborting program')
The runit run command becomes:
exec /usr/local/bin/amounter.py
While not as kludgificent as using the logging facility, the preceding
does have a certain charmful kludgicity.
And, as tested with runit, if amounter.py dies, they both die and the
service gets rerun, and if inotifywait dies, they both die and the
service gets rerun.
Thanks everybody!
SteveT
Steve Litt
November 2015 featured book: Troubleshooting Techniques
of the Successful Technologist
http://www.troubleshooters.com/techniques
Re: Entering a passphrase interactively in a runit script
Wow! I didn't understand all of that, but it does point to the fact that there's no simple answer. It sounds like you want a human to type in the password. I can think of two ways to do it, without having the password hanging around on disk or in history or the environent: 1) Have a human invoke the daemon rather than Runit 2) Have the daemon that needs to receive the password listen on a socket, and have a front end program (could be telnet) ask the human for the password. This could be expanded to provide other communication between the daemon and the human. If you do #2, I'd like to hear all about it, to the extent that you can tell given the obviously trade secret nature of what you're doing. SteveT Steve Litt May 2016 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21 -- SteveT Steve Litt May 2016 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21 On Fri, 27 May 2016 10:49:40 +0200 Christophe-Marie Duquesne <c...@chmd.fr> wrote: > Hi, > > So many answers in this thread! I did not think this would generate so > much interest. It looks like I am going to use a completely different > method, but for the sake of interest in the answers, I am dropping > here a more detailed description of what was initially intended. > > What this does: The program runs with a dedicated user with limited > rights. It pulls messages from an input queue system, encrypts them > individually using symetric encryption and a passphrase, and then > forwards them to another output queue system. There are several queues > on the input system, each of these queues gets their individual > passphrase, so I actually simplified when I was saying that I launch > the program with > > exec prog # reads PASSPHRASE from the environment > > because what I actually do looks more like: > > exec chpst -u user env \ > $(cat /etc/passphrases.env) \ > program > > Where /etc/passphrases.env is only readable by root and looks like > this: QUEUE1="mylongrandompassphrase1" > QUEUE2="mylongrandompassphrase2" > > > So the user running the program can read the passphrases from its > environment, but cannot open the file /etc/passphrases.env. So runit > is taking care of reading the file and "dropping privileges". > > Another thing I did not mention is that the input system has some > limitations forcing me to create several instances of the daemon. As a > consequence I have a master daemon that does: > > sv start daemon1 > sv start daemon2 > sv start daemon3 > exec tailf /dev/null > > Now for extra security, we wanted to modify this so that the > passphrases are not on disk. And yes, that means non interactive > restart in case of a problem. This would protect us against physical > theft of a hard drive. > > Funnily enought the plan was already to use pass > (http://passwordstore.org) to store the passphrases, but one needs to > unlock a gpg key in order to use it. So somehow the problem does not > change and a passphrase needs to be entered when the daemon start. > Ideally, the "master daemon" would ask for the passphrase of the gpg > key, unlock it, use it get the relevant passphrases of the queues from > the pass repository, and then start the slaves and somehow transmit > these passphrases to each of them, and manage them from there. > > So here is the problem in full, which I break down to 2 tasks: > - Get a passphrase at startup interactively from a master daemon (to > unlock the gpg key and get the relevant passphrases from pass) > - Transmit environment variables securely to several slaves > > But again, I am fairly certain we are going to use a whole different > approach and just not encrypt these queues like this and rather do > something on disk of the receiving system, so don't put too much > thinking in there. > > Thank you so much for your insights! > > Cheers, > Tof
Re: Entering a passphrase interactively in a runit script
On Thu, 26 May 2016 14:16:16 +0100 Jonathan de Boyne Pollard <j.deboynepollard-newsgro...@ntlworld.com> wrote: > Christophe-Marie Duquesne: > > Any idea how to proceed? > > You're running a daemon. It really shouldn't have an interactive > user interface. Remember the lessons that resulted in Session 0 > Isolation in Windows NT. The more I read of this thread, the more I think it's a bad idea to have a boot-instantiated daemon acquire a password by any means, and the more I think maybe a completely different approach might be more appropriate. So let me ask the original poster a few questions: * What does this daemon do? * How many users does the machine have? - At one time? - Ever? * Would all the machine's users be expected to know the password? * Did you write the daemon yourself? * Why does it need to be a supervised daemon, rather than just a program the user runs? Thanks, SteveT Steve Litt May 2016 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
How to supervise an early process
Hi all, A big objection to most supervision type init systems is that for a given process you must choose between early, like run from the rc script(s) preceding running of the supervisor, and respawning supervision. I just thought of a theoretical hack to have both. Symlink to give the executable a new name. ln -s myapp myapp_sym Run myapp_sym as early as you want in the rc file. Heck, run it in the initramfs for all I care, and let it switch_root over. Then, in the run script for any supervision suite, do this: === #!/bin/sh if ps ax | grep myapp_sym; then killall myapp_sym fi exec myapp === Obviously, for some apps, you'll need to shut down a little more gracefully than killall, but whatever way you need to shut down, you just put it in the if statement or in a shellscript called from within the if statement. This should work on daemontools, daemontools-encore, runit and s6. It might run on more, but those four are the only ones I've used. One of the outstanding benefits of supervision suites is how malleable they are with a little imagination. Thanks, SteveT Steve Litt June 2016 featured book: Troubleshooting: Why Bother? http://www.troubleshooters.com/twb
Crash script for Runit?
Hi all, I use Void Linux, which inits with Runit. Some supervisors have a script that runs when the supervised program terminates, and I was wondering if Runit has such a script. I couldn't find evidence of such with a Google search. What I'm trying to do is be informed every time one of my daemons goes down, whether or not it comes back up (sometimes they just keep changing PIDs and having a 0 uptime). Thanks, SteveT Steve Litt January 2016 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
No DOWN file support in runit?
Hi all, I've been using runit for about 4 months now, and a few days ago I tried to disable a service by placing a file called DOWN in its directory (like you would with s6 or daemontools). However, the service kept coming up at boot. What's the idiomatic way to prevent a runit service from starting at boot? I hope the answer isn't to unlink it. Thanks, SteveT Steve Litt February 2016 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: What's the difference between s6 and s6-rc?
On Thu, 25 Feb 2016 16:02:11 +0100 Laurent Bercot <ska-supervis...@skarnet.org> wrote: > On 25/02/2016 15:47, Steve Litt wrote: > > Could somebody, in one or two sentences, tell me the difference > > between s6 vs s6-init? > > I'm not looking for references to big old documents: I'm looking for > > one or two sentences telling me the difference. > > Still, big old documents have all the information you need, and > more: the first lines of http://skarnet.org/software/s6-rc/why.html > would answer your question. > > s6 is a process supervisor. It manages daemons. > s6-rc is a service manager. It manages the global state of a > machine: what service is up, what service is down, with dependencies > between services, and "services" being implemented by either a daemon > or a one-shot script. > > s6-rc is a management layer running *on top of* s6. It uses the s6 > infrastructure to do its job, but this job is not the same at all. :-) Expecting responses like the preceding, I specified 1 or 2 sentences. Luckily, Jan Bramcamp answered my questions in 2 sentences, and then went on to give even more details, so I now feel confident in understanding the difference between the two. Understanding http://skarnet.org/software/s6-rc/why.html depends on understanding the exact meanings of "supervision suite" and "service manager", two phrases sounding confusingly similar to me (and I doubt I'm the only one). Also confusing is sometimes use of "supervision tree", which I assume is a synonym for "supervision suite." And then in your email reply (above), you refer to "process supervisor", which I assume is yet another synonym for "supervision suite." I wouldn't have understood http://skarnet.org/software/s6-rc/why.html before reading Jan's 2 sentence explanation (with additions). Now I do. I'll write more about this in a separate email... Thanks, SteveT Steve Litt February 2016 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: What's the difference between s6 and s6-rc?
On Thu, 25 Feb 2016 19:26:11 +0100 Laurent Bercot <ska-supervis...@skarnet.org> wrote: > Do you think a glossary page would help? Yes yes yes yes YES!!! SteveT Steve Litt February 2016 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Improving http://skarnet.org/software/s6-rc/why.html
Hi all, I find http://skarnet.org/software/s6-rc/why.html too dependent on phrases that aren't well enough defined, and phrases that are too similar. For the purpose of *this one email thread* I'm going to define a "Process Grand Supervisor" as a grand process that runs a bunch of "Process Supervisors", that in turn run, rerun, turn on and turn off, and output status of the processes they supervise. In other words: Process Grand Supervisor | `Process Supervisor (many) | `Process (1, or 2 if there's a ./log) The Process Grand Supervisor runs the Process Supervisors in parallel, with ordering completely undefined. A "Supervision Suite" should be defined as all the software to implement, run and control both the Process Grand Supervisor and the Process Supervisors. Extreme care should be taken never to use "Supervision Suite" in place of Process Grand Supervisor. In the Runit world, Process Grand Supervisor=runsvdir, while Supervision Suite refers to the entire Runit package. These definitions need to be used consistently. A "Process Ordering Facility" (for the purposes of this one email thread) is software that can be added to a Process Grand Supervisor for the purpose of defining which order Process Supervisors are run. The Process Ordering Facility can take many forms, three of which are: * Shellscripts that run before the Process Supervision Suite is started (like Runit's 1 and 2 files) * Manipulation of down files (like LittKitt) * Another kind of addition, added to an existing Process Supervision From what I understand, a Process Ordering Facility is a small part added to a Process Grand Supervisor. It does not contain a Process Grand Supervisor, nor is it a form of Process Grand Supervisor (has-a relationship). It's something you bolt on to a Process Grand Supervisor. Continuing on definitions, I think things that run should be called either "processes" or "daemons" or "services", but only one of those words should be used throughout, and no attempt should be made to draw any distinction between any of the three. Already, people on all sorts of communication venues use these words interchangeably, and then others try to appoint some sort of arbitrary distinction, and the whole thing is a swamp of misunderstanding. Personally, I'd vote for "process" as the one and only word, but an argument could be made that the Process Supervisor and Process Grand Supervisor are also processes, so the thing supervised by the Process Supervisor should be called the "service." Personally, I see no reason to ever use the word "daemon". For the rest of this email I'll use "service" for the thing run by the Process Supervisor... There are two kinds of services: * Oneshot services * Long-run services These phrases are crystal clear and have been used forever in this community. One could also make the point that there could be foreground services: I'll worry about that later. There are other definitions that need to be clarified and consistently agreed upon: init system (there's a special place in hell for the fool who says "init" when he means sysvinit), PID1, "daemonizing" for those fools who think backgrounding should be evidence of readyness, and probably others. I leave that for another time. WHY THIS MATTERS A lot of people have gone to a lot of trouble to make excellent Supervision Suites and init systems that use Process Supervision Suites. This fact assumes new importance as more people get sidetracked, misled, or dragged kicking and screaming to systemd: http://skarnet.org/software/s6/systemd.html The trouble is this: Nothing in any existing Process Supervision Suite docs of which I'm aware showcase the robust and simple architecture bestowed by Process Supervision Suites and the init systems they [help] facilitate, unless the reader has quite a bit of familiarity with the situation, and the discipline to read *a lot*. Which is bad news, because most people who could benefit from an init featuring a Process Supervision Suite are a lot more like me than they are most of you. If they're lucky, their knowledge of Process Supervision Suites is "Yeah, daemontools is that thing you have to install to get djbdns to run." If they're unlucky, they ask "What's a Process Supervision Suite? What's daemontools?" When confronted with today's Process Supervision Suite documentation, their eyes would quickly blur with all the undefined concepts, the similarly worded undefined concepts, and the lack of diagrams or other explanation of relationships. And they'd figure "Hey, systemd works OK, I'll stay where I am." And then the Redhat/Freedesktop contingent would get even more power to consumer more of the Linux low level programming, to the point where using a Process Supervision Suite would preclude a system most people want to have. SteveT Steve Litt February 2016 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: The plans for Alpine Linux
On Sun, 31 Jan 2016 13:08:33 -0300 Guillermo <gdiazhartu...@gmail.com> wrote: > From this thread: > > http://www.mail-archive.com/supervision@list.skarnet.org/msg01123.html > > 2016-01-27 9:16 GMT-03:00 Laurent Bercot: > > > > The biggest hurdle that *every* distribution faces is that every > > daemon starting script is specific to the service manager, and > > supervision systems do things very differently from non-supervision > > systems. > > > > And so, daemon packages need to be separated into "mechanism" (the > > daemon itself) and "policy" (the way to start it), with as many > > "policy" packages as there are supported service managers. > > > > I plan to do this work for Alpine Linux towards the end of this > > year; I think most of it will be reusable for other distributions, > > including Buildroot. Hi Laurent, The situation you describe, with the maintainer of a distro's maintainer for a specific daemon packaging every "policy" for every init system and service manager, is certainly something we're working toward. But there are obstacles: * Daemon maintainer might not have the necessary time. * Daemon maintainer might not have the necessary knowledge of the init system or service manager. * Daemon maintainer might be one of these guys who figures that the only way to start up a system is sysvinit or systemd, and that any accommodation, by him, for any other daemon startup, would be giving aid and comfort to the enemy. * Daemon maintainer might be one of these guys who tries to (example) Debianize the run script to the point where the beauty and simplicity and unique qualities of the service manager are discarded. SteveT Steve Litt January 2016 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: C API for service management
On Tue, 19 Jan 2016 18:44:54 +0100 Jan Olszak <jan.ols...@vestiacom.com> wrote: > Hi! > I'm developing a daemon that will: > - start/stop > - send signals > - check status > of some services. > > This can be done via s6-svc or s6-svstat but: > 1. Can I use the underlying calls from libs6? > 2. Is this interface likely to change? I use runit, but as I remember from the little s6 that I did, s6 is functionally speaking a superset of runit. If I were going to develop a daemon like you suggest, I'd just use the executables bestowed by the process supervisor. For instance: Start = sv up servicename stop = sv down servicename restart = sv down servicename;sv up servicename status check = sv status servicename The output of sv status can be parsed to get whether it's running, and if so its PID so you can use kill to send it the desired signal. Why can't your daemon be a simple shellscript? SteveT Steve Litt January 2016 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: Determining the running init
On Mon, 02 May 2016 16:20:20 -0400 James Cloos <cl...@jhcloos.com> wrote: > I don't have a runit system up right now. > > Could someone show me what: > > lsof -a -p 1 -d txt > > outputs when run on a runit-based system? > > Thanks! > > -JimC [root@mydesk ~]# lsof -a -p 1 -d txt COMMAND PID USER FD TYPE DEVICE SIZE/OFFNODE NAME runit 1 root txtREG8,1 754872 2364332 /usr/bin/runit [root@mydesk ~]# Beware that this is Runit as installed by Void Linux. I suppose theoretically a different type of Runit installation could yield a different result. SteveT Steve Litt April 2016 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Re: s6 talk at FOSDEM 2017: video
On Mon, 06 Feb 2017 17:03:00 + "Laurent Bercot" <ska-supervis...@skarnet.org> wrote: > Hello everyone! > > I'm back from FOSDEM and it was great! Lots of people to meet, lots > of interesting talks to attend. And I had my own 15 minutes of glory, > complete with French accent and mic failure! The video has arrived and > it's available at: > https://fosdem.org/2017/schedule/event/s6_supervision/ Reading the abovementioned link, I still can't figure out exactly what "service management" is. The part about "bring all services up" and "bring all services down" sound to me like rc scripts. Is that correct? the part about "change services' states" sounds like (in runit) the sv command. Is that correct? You mention that services can be oneshots or longruns. To make it a longrun, would you just call it with the equivalent of runit's runsv, from inside the rc file? You mention the following: === Void Linux uses runit without a service manager; it sometimes needs hacks (longruns doing nothing) to emulate oneshot services. === Isn't the proper way to do a runit oneshot to run it from one of the rc scripts (/etc/runit/1 or /etc/runit/2)? The world needs more explanations like the one you give in this slideshow. One thing I'd like to see is a definitive set of definitions for service managers and supervision suites. Each definition should come with plenty of examples and enough redundancy that even the most unfamiliar can understand exactly what you're talking about. Which do you like more: Anopa, or s6-rc? Thanks for the great doc! SteveT Steve Litt January 2017 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: register runsvdir as subreaper
On Thu, 02 Feb 2017 19:30:17 + "Laurent Bercot" <ska-supervis...@skarnet.org> wrote: > >There's also the possible case of having this for getty/login > >session. So any process spawned from there won't be reparented to > >PID1 but to e.g. the session's supervisor; That can include things > >such as pulseaudio, settings daemon, dbus, window manager, etc > > Those are more examples of what you *can* do, with no precise reason > why you *should*. http://e.lvme.me/u33yl35.jpg > > > >Also besides the visually pleasing pstree, it means one can - thanks > >to a finish script - ensure that upon logout everything gets killed > >before a new getty is respawned. If that ability floats your boat, there's a perfectly wonderful init system that does that: Systemd. As far as the visually pleasing pstree, I'd rather have a messy pstree and simple, robust architecture than an aesthetically beautiful pstree and entangled, messy architecture. > > How would you do that? Unless the subreaper comes with yet another > system call to genocide all its children, you still need some > mechanism to atomically send a signal to everything - which means you > want to have everything into the same process group (which is > unlikely if you have an interactive session), or you need cgroups. Hey, I've heard of this great new init system that's all about cgroups! > > And even if you can, that still doesn't mean you should. Nohup is a > thing. I use it all the time, and half the time pipe its output to /dev/null. I doublefork a lot too. When I make a shellscript or design software, I never depend on the getty or terminal emulator's close closing my software. If I want it closed, I make it close itself. > People may want to leave background processes running after > they log off. Yes! Talk to any tmux user about this. I'd rather have a few orphans and zombies hanging around than foreclose the user's ability to start working from one computer and finish working at one 30 miles away, having logged out of the first one. I don't understand this desire some folks have to implement the marketing points of systemd. Visually pleasing pstree? Logging out kills all processes started during that login? What next, socket activation? Keyless remote with magnesium paddle shifter? In the features vs simplicity tradeoff, why do so few value simplicity? Runit, which I use every day, is so simple I could create a clone of it after reviewing djb's supervisor backgrounding code for about 10 minutes, and Gerrit, this isn't an insult to runit, it's a huge compliment. Runit's enough for me, but for the person wanting more USEFUL features and a guarantee against having an incommunicative, childless PID1, there's s6. And although not on-topic for a supervisor mailing list, Epoch provides wonderful init features in a very simple package. Every time I hear people wanting inits to do more, my question is "why does it have to be done in the init?" Whatever you want to do, there are a million ways to do it, so it's not necessarily the responsibility of already-written, highly functioning software. SteveT Steve Litt January 2017 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: runsvdir polling
On Sun, 15 Jan 2017 01:40:13 -0800 39066...@gmail.com wrote: > On Sat, Jan 14, 2017 at 04:10:07PM -0500, Steve Litt wrote: > > On Fri, 13 Jan 2017 21:12:27 -0800 > > 39066...@gmail.com wrote: > > > > > I'm using runit as my primary init on Linux to good effect but > > > have noticed that it accumulates CPU time even while the system > > > is idle. I > > > > How much time in how much uptime? Which process are you looking at > > with this accumulated time? Here's what I get: > > runsvdir has used 1m 55s over a bit under 5 days. The runsv instances > are way less (but then they have no reason to wake). Load average is > 0.00. > > > On my computer, none of this in any practical way affects my > > computing. I would guess on a more heavily used computer, any > > deleterious effects of runit time consumption could be solved by > > running runsvdir and all the runsv's with a positive nice value. > > > > In my opinion, by far the greatest benefit of runit is its > > simplicity, and there's no way I'd trade that for a theoretical > > efficiency benefit. > > That sounds sensible on a desktop. In my case the motivation is to > trim a source of power draw for an image that's going to run on a > battery-powered device that will be awake but idle a lot of the time. > Why Linux? Pretty much familiarity & tooling, if power is ok then I > get to have my cake and eat it too. Why not try s6? IIRC, s6 doesn't poll, is a little more complex than runit, but like runit, it's a daemontools descendent. I've used it and like it. One thing to consider: What is the relationship between the 1m:55s and power consumed? Is power consumption proportional to CPU time, or does it depend on the kind of activity being done? Are disk accesses more, or less power hungry than loops with calculations or other CPU intensive stuff? How would you even find out the answers to these questons? Another question: What sized battery, is it rechargable, and how long do you expect it to last between replacements or recharges? What I'm getting at is this: If power draw is proportional to time, then in 50 days runsvdir would have consumed just under 20 minutes, which is almost nothing if you're using a good sized battery. Are you going to use a superserver? This sounds like a good case for a superserver. You could use xinetd, or the djb-type superserver (can't remember its name). This sounds interesting. SteveT Steve Litt January 2017 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Runit debian package fix for Ubuntu 16.04
On Mon, 22 Aug 2016 13:44:45 + Gerrit Pape <p...@smarden.org> wrote: > For some reason, interest in the runit project raised again over the > last months, although I didn't do any work on it, In a word, systemd. Expanding, runit makes life easy, predictable, and simple. Thank you for that. The shame is it took a tragedy like systemd for people to look around and discover runit. Now that runit is gaining popularity, I'm sure you'll be flooded with requests to put in new features. Please don't. Runit's quality and usefulness derives straight from its simplicity. Thanks, SteveT Steve Litt A very satisfied runit user the past 10 months
Re: Mass bug filing: use and misuse of dbus-launch (dbus-x11)
ought that address=unix:runtime=yes might, but that did not either. > [snip] > > Simon McVittie: > > > To be brutally honest, there is a fairly low limit to how much > > benefit I can create by giving new things to PC-BSD users, [...] > > > That's not the right way to look at it. This is precisely the right way to look at it, when it pertains to systemd. > You yourself have just said > several times that this is stuff that is supposed to be on "supported > Unix platforms". This isn't giving new things to anyone. This is > making existing things, that you yourself think are existing, work. If these existing things can't be made to work without systemd incorporation, they should be torn out and replaced. Encumbering a good system with systemd is not the answer. > > I shouldn't dismiss PC-BSD so readily, if I were you, either. PC-BSD > (now rebranded as TrueOS Desktop a few days ago -- I just got through > changing a whole load of preset file and -run package names.) is the > BSD that comes in the box with the desktop environments and with all > of the desktop programs that use Desktop Bus. Yes, people can and do > run all of this stuff on FreeBSD and OpenBSD from ports. But > PC-BSD^H^H^H^H^H^H ... Gah! ... TrueOS Desktop is where it comes in > the box and is run as standard in the default install. TrueOS > Desktop is where one ends up choosing from running PCDMd, kdm, lxdm, > or gdm; and where one gets lots of little Desktop Bus brokers all > over the place in various ways from these different systems. TrueOS > Desktop is where the people who are behind the operating system will > be strongly motivated towards improving the desktop subsystems and > the Desktop Bus. To those who care about simplicity and user-maintainable software, the preceding paragraph appears to be one possible strategy to continue eliminating non-systemd choices. Beware. > > You're pushing your new way of per-user Desktop Bus brokers at the > world. I can give the TrueOS Desktop people, and the the UbuntuBSD > people, and the Debian FreeBSD people, a service management system > that I know can run per-user Desktop Bus brokers on a FreeBSD > kernel. It already does. I published it last year. If you, the > Desktop Bus people, can give us these mechanisms in your program > actually working on these operating systems, then the TrueOS people > get the opportunity to simplify some of the scaffolding that they > have had to erect (PCDM-session writing out nonce scripts that invoke > dbus-launch, for example), and you get less of the world still using > your old way of doing things. LOL, per-user desktops. There must be a zillion different ways to have sterminal hung off a Linux box each get their own GUI. I'd do it myself, except that's not my itch. In a world where a COTS desktop is $600 USD, laptop $500 USD, and used but still perfectly functional computers can be had for $50-$200, hanging terminals makes little economic sense. I'm sure the systemd afficianados will find such a use case, and proclaim that use case must be served, but we all know it's FUD. The systemd folks shout from the mountaintops that sysvinit is 32 years old or whatever, and how that alone is enough reason to use systemd, and yet these same monuments to modern software proclaim their multiseat, terminal-enabling technology is a reason to switch to systemd, even though terminals had their heyday in 1984. Talk about greybeards. One more thing: They talk about dbus as if it's a good thing. Even before systemd, I tried to stay away from a bus system that was pretty much like a traffic circle enabling everything to talk to everything else, addressing allowing. What could *possibly* go wrong? The subject of this thread is "Mass bug filing: use and misuse of dbus-launch (dbus-x11)". If you're a software user, use dbus as little as possible. If you're a developer, find other communication methods, and don't incorporate dbus. Because, as evidenced by this thread, dbus is now just a pretty entry point to systemd. SteveT Steve Litt
Re: Runit questions
On Thu, 13 Oct 2016 12:16:52 +0200 Andy Mender <andymenderu...@gmail.com> wrote: > I was recently trying to convert my OpenRC-based Gentoo installation > into one based on runit. However, I noticed that it's quite easy to > send runsvdir or runsvchdir into an infinite loop when there is > something wrong with a service directory or the runlevel dir selected > through runsvchdir is missing. > Is there something potentially wrong with my setup or is this normal? I know nothing about runsvchdir, but with runsvdir, well, it's *supposed* to be an infinite loop. It loops the service directory forever, starting what needs to be started. Could you please elaborate on the symptom that happens to you when there's something wrong with a service directory, and if possible articulate on the flaw in that service directory? Thanks, SteveT Steve Litt September 2016 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: GNU Emacs now runs in foreground
On Tue, 6 Dec 2016 12:15:49 + Jonathan de Boyne Pollard <j.deboynepollard-newsgro...@ntlworld.com> wrote: > Martin "eto" Misuth: > > First, there are two major caveats, > > There are actually three. They break scripting. For example: People > cannot use the GNOME Editor as $VISUAL or $EDITOR because one of the > things implicit in the $EDITOR/$VISUAL mechanism is that when the > program that has been invoked exits, the editing is over and the file > being edited has been saved in the desired form. That is not the > behaviour of the "small and lightweight" GNOME Editor, however. I think I can live without GNOME, so for me this is no problem at all. I think a lot of people on this list can live without GNOME. > > * http://unix.stackexchange.com/questions/201900/ > > * http://unix.stackexchange.com/a/323700/5132 > > * https://news.ycombinator.com/item?id=13056252 > > Other "interesting" problems result from the move that the Desktop > Bus and the Desktop Environment people are making away from > per-session instances of the D-BUS daemon to per-user instances of > the same. This causes fun with deciding what the daemon's $DISPLAY > should be set to. A per-session Desktop Bus obviously has one > $DISPLAY by and large. But a per-user Desktop Bus not only has to > handle multiple logins from a single user, it has to handle that the > per-user session management can be running when there's no X server > at all. Dbus isn't part of my world, and when software tries to make dbus part of my world, I tend to ditch that software. Dbus is a traffic circle allowing everything to talk to everything else, addressing allowing. It's a system-wide global variable on steroids. > (systemd starts its per-user instances via PAM hooks that > act upon every login, including logins over SSH and on terminals.) If dbus isn't part of my world, then it's safe to say systemd isn't part of my universe. If systemd were relevant to my life, I'd still be on the Debian-User mailing list, not on this list. I don't think I'm alone on this. > Even though some daemons try to take the approach that the daemon > supports multiple $DISPLAYs, sent in from multiple clients as part of > the client session, one unfortunately finds that the daemons > themselves still have to have an arbitrary $DISPLAY in order to start > up in their initial, not connected to any clients and their displays > yet, mode. In practice, thus, the implementation of the user-wide > client-server idea is half-hearted and flawed in this respect. Well, maybe. But I'm going to try it. To me, Linux is and should remain a DIY accessible OS, and that requires shunning all things FreeDesktop.Org. SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: GNU Emacs now runs in foreground
On Tue, 6 Dec 2016 10:40:20 +0100 "Martin \"eto\" Misuth" <et.c...@ethome.sk> wrote: > I am immense fan of these user-level daemons (maybe they should have > some more exact name). > > First, there are two major caveats, that I noticed, being seen as > disadvantages, to this design: > > 1 - When such daemon crashes, your whole session is taken out. For > example all terminals die, all shell process trees go out (make), so > you can loose quite a lot of work. > > 2 - There is more moving parts in these setups, and when you are > unawarare of supervision, it might be hard to manage these daemons > efficiently. > > Software designed for this mode of operation obviously includes emacs > now. Teams and authors behind these, usually have pretty good > abilities and experience, as obviously designing this way requires > some kind of architecture. Thus most of these packages dont suck, and > are quite reliable. > > I personaly use/know these: > > - super famous tmux - terminal multiplexer > - you are probably aware of that one, handles lots of terminal > sessions > > - gnu screen - terminal multiplexer > - older and poorer cousin of tmux > > - mpd - the music player daemon - basically music player > - when spawned as it's own service outside of/before X it can > keep music playing even as you are tweaking you xorg.conf :) > - can route music over fifos through network and make many > amchines play same music Thanks Martin, Could you please show me your run scripts for tmux, screen and mpd? These are the three I could see myself using in this manner. SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: GNU Emacs now runs in foreground
On Thu, 1 Dec 2016 20:48:37 +0300 Jean Louis <bugs@gnu.support> wrote: > On Thu, Dec 01, 2016 at 12:41:18PM -0500, Steve Litt wrote: > > > > > > Just before some time, emacs --daemon, would go into background, > > > so it was not feasible to control it with s6 scripts. Now > > > developers changed it, and it is possible to invoke multiple > > > instances by name, and still keep the daemon in foreground. In > > > that mode there is no interface to editor. Clients that access > > > the editor, show the interface. > > > > Fascinating! How do emacs clients communicate with the emacs daemon? > > Sockets? > > > > How many other user programs might be used as daemons. What an > > interesting idea! For some reason, mplayer pops into my mind. > > I have a strange feeling that you are joking... Not at all. You know mplayer can interact via a FIFO. Seriously, I'm not joking. I'm always interested in new ways to use my computer. SteveT -- SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: GNU Emacs now runs in foreground
On Thu, 1 Dec 2016 20:28:46 +0300 Jean Louis <bugs@gnu.support> wrote: > On Thu, Dec 01, 2016 at 12:05:31PM -0500, Steve Litt wrote: > > OK, I'll byte. > > > > I thought emacs was an editor that a human runs in the foreground to > > edit files, so my reaction to this was "of course it runs in the > > foreground!" > > > > What am I missing? > > You are right, it is hard to figure out the context. That is for s6 > services, I am invoking editor as daemon, so that it may be accessed > later by need. > > Buffers remain in memory, and so the IRC, Jabber chat, various > documents from which I yank parts of texts into emails, registers, the > programming languages are invoked within the editor, and shells and > other tools. Invoking single editor each time is a bit slower, than > invoking it as client that accesses the daemon. And that way, all the > goodies would be lost by each new invocation. Some people have > multiple buffers over multiple days. > > Just before some time, emacs --daemon, would go into background, so it > was not feasible to control it with s6 scripts. Now developers changed > it, and it is possible to invoke multiple instances by name, and still > keep the daemon in foreground. In that mode there is no interface to > editor. Clients that access the editor, show the interface. Fascinating! How do emacs clients communicate with the emacs daemon? Sockets? How many other user programs might be used as daemons. What an interesting idea! For some reason, mplayer pops into my mind. Thanks! SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: GNU Emacs now runs in foreground
On Thu, 1 Dec 2016 11:18:29 +0300 Jean Louis <bugs@gnu.support> wrote: > The GNU Emacs now has got a new option: > > emacs --new-daemon=NAME > > that is running emacs daemon in foreground, obviously someone reacted > since last time I wrote to the emacs mailing list. It is in the > development version or git. OK, I'll byte. I thought emacs was an editor that a human runs in the foreground to edit files, so my reaction to this was "of course it runs in the foreground!" What am I missing? Thanks, SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: s6-svscanboot, how to exit?
On Fri, 14 Jul 2017 22:29:04 +0300
Jean Louis <bugs@gnu.support> wrote:
> On Fri, Jul 14, 2017 at 02:31:28PM -0400, Steve Litt wrote:
> > > If you're fine with it not being supervised and insist on
> > > launching s6-svscanboot from /etc/rc.local, then you want to
> > > enclose all its commands in a "background { }" block, which is
> > > execline's equivalent of the shell's &.
> >
> > Or, from sysvinit, you can put it as a respawn in /etc/inittab, so
> > that s6svscanboot itself is supervised, by sysvinit.
>
> On that new Debian VPS that I ordered with Digital
> Ocean, there is no /etc/inittab and I don't know
> nothing much about systemd, so I just used recipe
> from stackexchange.com
So you can either do it like you are now, and if s6svscanboot goes down
your whole supervision tree goes down and stays down, or you can make a
systemd unit file for s6 and run it from there, and if s6-svscanboot
goes down it will come right back up. You can probably experiment at
home with a Debian virtual machine host to get just the right unit file
and how to tell it to run itself.
SteveT
Steve Litt
July 2017 featured book: Quit Joblessness: Start Your Own Business
http://www.troubleshooters.com/startbiz
Re: s6-svscanboot, how to exit?
On Fri, 14 Jul 2017 13:14:36 +
"Laurent Bercot" <ska-supervis...@skarnet.org> wrote:
> >The script does not "exist". That means when I do
> >
> >sudo service rc-local stop
> >
> >it stops, and when I do
> >
> >sudo service rc-local start
> >
> >it is then hanging, or running, it is fine, but I
> >would like to exit back to shell on such run.
> >
> >Adding & on end of line is not helping.
> >
> >How do I make it execute and exit back to shell?
>
> You don't, if you want it supervised. If you want to launch
> s6-svscan under systemd, you should write a proper unit file for it,
> so systemd can manage it as one of its services.
>
> If you're fine with it not being supervised and insist on launching
> s6-svscanboot from /etc/rc.local, then you want to enclose all its
> commands in a "background { }" block, which is execline's equivalent
> of the shell's &.
Or, from sysvinit, you can put it as a respawn in /etc/inittab, so that
s6svscanboot itself is supervised, by sysvinit.
SteveT
Steve Litt
July 2017 featured book: Quit Joblessness: Start Your Own Business
http://www.troubleshooters.com/startbiz
Re: s6-svscanboot, how to exit?
On Sat, 15 Jul 2017 18:53:44 +0300 Jean Louis <bugs@gnu.support> wrote: > Hello Jonathan, > > On Sat, Jul 15, 2017 at 02:47:39PM +0100, Jonathan de Boyne Pollard > wrote: > > > Type=forking > > > > No. > > > > > StandardOutput=tty > > > > No. > > > > > RemainAfterExit=yes > > > > No. > > > > > SysVStartPriority=99 > > > > No. > > I know I know, I did not read the manuals on > systemd. And why should you? You're using daemontools to escape as many processes as possible out of systemd's clutches, and there's nothing wrong with copying off stackexchange and letting the list know what worked. You're not expected to be a systemd expert. > > > You're a long-time daemontools user, you say. > > Why on Earth do you think that Type=forking is > > right? > > Because I don't think, I copied and it works, if > you did not tell me know, it would remain so for > years probably. I know I should be more > responsible, but I have placed before daemontools > in /etc/inittab and just forgot about it, I guess > that is wanted effect. You don't really need to apologize. You apparently weren't completely correct, and it worked well enough to make it seem correct. And some guy comes on, gives you four "no"s without reasons, imply that you're falsely bragging about being a daemontools user, and then gives you one of those "why on earth" type things. Louis, ignore the guy. His motivation obviously isn't to help. And if he again asks "why on earth did you think "Type=forking" is right?", just tell him that for all you knew, that meant it's the type of software that systemd must fork because it doesn't fork itself. That's just as plausible. And when he asks... > > > Why on Earth do you think that good daemons fork > > and exit parent? the answer is that you didn't. > > I don't think really, and I tried the one-shot > method too, but that was not the problem that I > was not exiting. > > This one I changed > > RemainAfterExit=no > > and now after systemd "start" it exits. > > > Why on Earth do you think that svscan -- any > > svscan, from daemontools-encore svscan to > > s6-svscan -- forks and exits parent? Or needs a > > TTY? > > You are right, I did not check enough. I'm sure you *DIDN'T* think that, and you really don't owe this guy an explanation, because he's sure not here to help you. Laurent's right. Go on a systemd list or IRC, tell them you need to spawn daemontools, tell them daemontools does not put itself in the background, and that you want it restarted if it stops. They'll tell you what to do. By the way, you might have to tell it to put /command on the $PATH. If you need that and you cannot get systemd to handle it, I'll give you a shellscript to get it done. SteveT Steve Litt July 2017 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: s6 as a systemd alternative
On Wed, 28 Jun 2017 22:31:12 -0300 Guillermo <gdiazhartu...@gmail.com> wrote: > 2017-06-28 14:40 GMT-03:00 Steve Litt: > > > > On Mon, 26 Jun 2017 14:53:50 + "Laurent Bercot" wrote: > >> > >> The problem with the runit model is that it is pure supervision - > >> it does not provide service management. You have to run all your > >> oneshots _before_ you can start longruns. > > > > Not necessarily true. You can easily run longruns early by creating > > a second service directory and a second link directory. You make the > > service specific directory, create run script and the supervise > > directory within it, and symlink like the following: > > > > ln -s /etc/sv2/gnumeric /var/svlink/gnumeric > > > > Then do the following: > > > > runsv /etc/svlink/gnumeric > > But then you end up with an unsupervised runsv process, disconnected > from the rest of the supervision tree... I'm not sure about the unsupervised part of it, but I know if the executable ends another one is run within a second, and I know that you can turn the executable on and off with sv [up|down] /var/svlink/gnumeric SteveT Steve Litt June 2017 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: s6 as a systemd alternative
On Thu, 29 Jun 2017 19:57:55 -0300 Guillermo <gdiazhartu...@gmail.com> wrote: > 2017-06-29 1:43 GMT-03:00 Steve Litt: > > > > On Wed, 28 Jun 2017 22:31:12 -0300 Guillermo wrote: > >> > >> But then you end up with an unsupervised runsv process, > >> disconnected from the rest of the supervision tree... > > > > I'm not sure about the unsupervised part of it, but I know if the > > executable ends another one is run within a second, and I know that > > you can turn the executable on and off with > > > > sv [up|down] /var/svlink/gnumeric > > The executable (gnumeric) is supervised, the supervisor (runsv) is > not. Sure, all those things are true, but while the supervisor is > alive. Kill runsv (if launched in this way) and see what happens. The runsv executable is pretty robust, so it's unlikely to die. As far as somebody killing it accidentally or on purpose with the kill command, that's a marginal case. But if it were *really* important to protect against, fine, have one link dir per early longrun, and run an individual runsvdir on each of those link directories. So now you have the same opportunity to kill each early longrun's supervision as you have to kill the main supervision. I think that's huge overkill, but it could be done. Which is the real beauty of daemontools-inspired inits: They're just POSIX, so a halfway savvy admin can mold them to his or her exact situation. SteveT Steve Litt June 2017 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: s6 as a systemd alternative
On Fri, 30 Jun 2017 19:50:17 + "Laurent Bercot" <ska-supervis...@skarnet.org> wrote: > >The runsv executable is pretty robust, so it's unlikely to die. > Yadda yadda yadda. Most daemons are also unlikely to die, so > following your reasoning, I wonder why we're doing supervision in the > first place. Hint: we're doing supervision because we are not content > with "unlikely". We want "impossible". You want impossible. I'm quite happy with unlikely. With my use case, rebooting my computer doesn't ruin my whole day. If it *did* ruin my whole day, my priorities would be changed and I'd switch to s6. > > > > As far > >as somebody killing it accidentally or on purpose with the kill > >command, that's a marginal case. But if it were *really* important to > >protect against, fine, have one link dir per early longrun, and run > >an individual runsvdir on each of those link directories. > And you just increased the length of the chain while adding no > guarantee at all, because now someone can just kill that runsvdir > first and then go down the chain, like an assassin starting with the > bodyguards of the bodyguards of the important people. Or the assassin > might just use a bomb and blow up the whole house in one go: kill -9 > -1. > > The main point of supervision is to provide an absolute guarantee > that some process tree will always be up, no matter what gets killed > in what order, and even if everything is killed at the same time. To me, the preceding isn't the main point of supervision. Supervision benefits I value more are: * Run my daemon in foreground, so homegrown daemons have no need to self-background. * Consistent and easy handling of log files. * Under almost all circumstances, dead daemons get restarted. * Simple config and troubleshooting, lots of test points. * POSIX methodologies ensure I can easily do special stuff with it. * Ability to base process dependency on whether the dependee is *really* doing its job. > You > can only achieve that guarantee by rooting your supervision tree in > process 1. Yes. > > With runit, only the main runsvdir is supervised - and even then it > isn't really, because when it dies runit switches to stage 3 and > reboots the machine. Which is probably acceptable behaviour, but > still not supervision. If we're going to get into definitions, then let me start by saying what I want is daemontools that comes up automatically when the machine is booted. Whether or not that's supervision isn't something I care about. > And everything running outside of that main > runsvdir is just hanging up in the air - they can be easily killed > and will not return. Well, if they kill the runsv that's true, but if they kill the daemon, no. Either way, I'm willing to live with it. > > By adding supervisors to supervisors, you are making probabilistic > statements, and hoping that nobody will kill all the processes in the > wrong order. But hope is not a strategy. There is, however, a strategy > that works 100% of the time, and that is also more lightweight because > it doesn't require long supervisor chains: rooting the supervision > tree in process 1. That is what an s6-based init does, and it > provides real, strong supervision; and unlike with runit, the machine > is only rebooted when the admin explicitly decides so. I completely understand your point. I just don't need that level of indestructibility. > > If you're not convinced: *even systemd* does better than your > solution. systemd obviously has numerous other problems, but it does > the "root the supervision tree in process 1" thing right. LOL, my whole point is I don't necessarily think "root the supervision tree in process 1" is right, at least for my use case. I *enjoy* having a tiny, do-almost-nothing PID1. Like I said before, if losing control of the system during special circumstances would ruin my whole day, I'd change my priorities and use s6. > > I appreciate your enthusiasm for supervision suites. I would > appreciate it more if you didn't stop halfway from understanding > everything they bring, and if you didn't paint your unwillingness to > learn more as a technical argument, which it definitely is not, while > flippantly dismissing contributions from people who know what they > are talking about. But I didn't flippantly dismiss anybody or any contributions. I pointed out that one can, and I'll use different verbiage now, respawn daemons early in the boot, before some of the one-shots had started. I'm not an enemy of s6. I'm not an enemy of anything you apply the word "supervision" to. I think I understand your reasons for doing what you do. It's just that with my current use case, I've traded some of s6's proces
Why /command ?
Hi all, I'm writing a document on how to install runit on Devuan, with the hope that some day it will lead to a Devuan package that makes sense and to the best degree possible implements the goals of the software's author. Most of it's pretty straightforward, but the runit install scripts (package/upgrade to be specific) create /command right off the root, and the runit docs suggest I create /package right off the root. These are things that most distros would refuse to do. So I was wondering what the original intent was in having these two directories directly off the root? Is it so the init and supervision can proceed even before partition mounts are complete? Is there some other reason? Can anyone recommend setups that fulfill the reasons for the direct-off-root dirs without having direct-off-root dirs? By the way, if the runit docs go well, I'll do the same thing with s6. Thanks, SteveT Steve Litt June 2017 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: s6 as a systemd alternative
On Mon, 26 Jun 2017 18:17:29 +0300 Jean Louis <bugs@gnu.support> wrote: > I am user of s6 for reason of simplicity and to avoid trouble of > systemd. > > While not being developer, maybe it could be possible to have s6 run > systemd as a service Euuu! > and systemd to run s6 as a service, This actually makes sense. One by one move daemons from the unfathomable systemd to the completely understandable s6, and leave your binary logs behind on the moved daemons :-) SteveT Steve Litt June 2017 featured book: The Key to Everyday Excellence http://www.troubleshooters.com/key
Re: How to use s6 and s6-rc together
On Fri, 19 May 2017 00:02:00 +0200 Luis Ressel <ara...@aixah.de> wrote: > On Thu, 18 May 2017 16:28:17 -0500 > Robert Hill <hilljamesrob...@gmail.com> wrote: > > > The overview for s6-rc > > <http://skarnet.org/software/s6-rc/overview.html> mentions that: > > > > > the chosen init should make sure that a s6 supervision tree is up > > > and running. s6-rc will only work if there is an active s6-svscan > > > process monitoring a scan directory. > > > > This is a little confusing to me, and I am sure I am not > > understanding correctly. If s6-svscan is monitoring a scan > > directory, won't all the processes in that scan directory already > > have been started by the s6-supervise instances that get started by > > s6-svscan? I would like to use s6-rc to start all services and s6 > > to supervise them once started. > > Yes, when your init exec's into s6-svscan, it will immediately start > all services in the scandir (unless the corresponding servicedir > contains a down file). > > However, the scandir will typically be empty at this point (it may > contain an early getty service). It is only populated with services > once you call s6-rc-init (to be exact, s6-rc-init creates the > servicedirs in /run/s6-rc/servicedirs and then symlinks them into the > scandir). s6-svscan will still not start the services, though, since > all the servicedirs created by s6-rc-init will contain a down file. > They will only be started once you call s6-rc -u change ok-all (ok-all > being the s6-rc bundle containing all your services). This sounds something like the way I incorporated determinate startup order to daemontools, runit and s6 using LittKit: http://troubleshooters.com/projects/littkit/README http://www.troubleshooters.com/linux/diy/suckless_init_on_plop.htm#littkit_introduction If you're using down files to determine startup order, do you have a different filename (LittKit calls it 'reallydown") to take over the former functionality of the down file (to make the service down for awhile, through reboots)? Thanks, SteveT Steve Litt May 2017 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: A dumb question
On Mon, 01 May 2017 23:11:25 +0200 Francisco Gómez <especta...@kydara.com> wrote: > Hello there, nice to meet you! If you don't mind, I'll be quite too > straight and tell you what I'm coming for. > > So, a few days ago, I switched to Void Linux. This not only meant that > my hipster non-mainstream levels had increased over 9000, but that I > had switched from Systemd to Runit as well. At first, I found it > amazing that the system and the GNOME desktop could do so much and so > easily without relying on a heavy init overlord. I also am surprised you could get Gnome to work without systemd. Just for fun, try out LXDE, and install dmenu and have it at the ready with an easy hotkey. I think you'll love the productivity enhancement, once you get used to it. > I shared my joy to my > fellow colleagues. And during the process, someone recently told me > something like this. > > "It's old software. Its last version is from 2014. If I have to > choose between a dynamic, bug-filled init like Systemd and a > barely maintained init like Runit, I'd rather use Systemd." You'll meet plenty of people like that. He almost certainly hasn't tried Runit. If "barely maintained" is why he recommends systemd, he's just flapping his gums. Or, maybe, he's a systemd sycophant and you backed him into a corner where he couldn't enthuse about systemd because it's better than sysvinit, as if those were the only two choices. > > That sounds bad, doesn't it? I've heard a lot worse. > > I hence ask: why is it that Runit has no new versions, neither a VCS > repository or bug tracker? Has Runit been so well tested and hardened, > is it such a simple codebase? Pre-cisely! As of 2014, it did everything it was supposed to do, and did it according to specification. The niche it fills is as the simple init, so there's no reason to add features. > Or are there not enough interested, > capable people maintaining the project? Last time I looked, there's one guy maintaining it. That one guy wrote it, and every once in a while he fixes a bug or whatever. But here's the thing: He isn't riding his software to fame and glory, so he feels no need to cram more and more features into it: And that's the way I like it. Runit's so simple I could maintain it that if Gerrit Pape dropped dead tomorrow, I could pick it up and maintain it. That's the benefit of having software developed by one guy. But wait, there's more. Let's say Gerrit Pape and I are on the same airplane and it crashes in the Andes mountains. No problem, there's another init system called s6 that's extremely similar to Runit, and you could switch between the two without breaking a sweat. > > Sorry for asking this, but I'm really curious and interested on using > Runit, and questions need answers. Even if those questions are really > questionable on themselves. Cheers~ Yes. Guys like your friend somehow manage to sound s convincing, even though they give you almost no facts. Runit's an excellent choice, whether your distro shipped with it or not. Meanwhile, Void Linux has been the most trustworthy distro I've ever used. It's a rolling release that almost never has bugs, and you don't get into jams the way Arch people do. And any time you need help, you can go to #voidlinux on FreeNode and talk to lots of experts. Void's the best: Don't lose your enthusiasm. Steve Steve Litt May 2017 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: Gentoo wiki
On Fri, 1 Sep 2017 19:44:07 +0100 Jonathan de Boyne Pollard <j.deboynepollard-newsgro...@ntlworld.com> wrote: > For those people who had not noticed: Someone (I shall say no more. > (-:) has been writing on the Gentoo wiki recently. > > * https://wiki.gentoo.org/wiki/S6_and_s6-rc-based_init_system > > * https://wiki.gentoo.org/wiki/Runit > > * https://wiki.gentoo.org/wiki/S6 > > * https://wiki.gentoo.org/wiki/Daemontools-encore Jonathan, If you wrote those wiki pages, that's a wonderful and needed thing, and not just for Gentoo people. Otherwise, thanks for pointing out these pages to us. I'll share them with others. SteveT Steve Litt September 2017 featured book: Manager's Guide to Technical Troubleshooting Brand new, second edition http://www.troubleshooters.com/mgr
Noclobber supervisor/init installation
oes the $da_dir, $du_dir and $dp_dir vocabulary sound reasonable? I'd like to thank Joel Roth (copied) for first giving me the idea of separated directories. SteveT Steve Litt September 2018 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz
Re: Log rotation issue with runit
On Sat, 29 Dec 2018 18:33:06 + Dmitry Bogatov wrote: > [2018-12-27 08:07] Steve Litt > > > [ Dmitry Bogatov ] > > > No, it is reproducible. See end of bug thread. > > > > Hi Dmitry, > > > > Just so we're all on the same page, do you mean the end of > > https://bugs.debian.org/916230 ? Could you please provide a message > > number? I briefly looked over https://bugs.debian.org/916230, paying > > particular attention to the later half, and could see no > > reproduction sequence articulated. But sometimes I miss things. > > Sure. Message 17: > > Offending .u file is created by rename(2) call at line 532, in > logdir_open() function. It happens, when 'current' file exists, > non-executable and non-empty. > [...] OK, so you can reproduce it with echo bogus > current; chmod a-x current That's pretty bizarre logic. I can understand why an empty current would be overwritten --- what do you lose. But why only if it's non-executable? Are they using the executable bit as some sort of flag to keep processes from overwriting each others' writes to the file? I once wrote a program in which, when an invocation of my program opened a file, it set the file to read-write, and other invocations of my program would decline to try to open a read-write version of the file. Bizarre, but it worked, and no process clobbered the other. I guess what I'm asking is this: Are you sure the original poster's (OP's) .u files were caused by the rename call when non-empty non-exec current exists, or is he experiencing a different reproduction sequence. I still wouldn't mess with the existing code. I don't think anyone here is positive of the purpose of the logic you described, and therefore what side effects would happen if it were modified. The OP didn't have enough .u files to really inconvenience him, and there are other ways of getting rid of .u files. And the OP could switch to s6. SteveT Steve Litt December 2018 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Can s6 be enough?: was s6-ps
On Sat, 5 Jan 2019 10:46:29 + Jonathan de Boyne Pollard wrote: > s6 and s6-rc are actually ports/packages in FreeBSD and s6 is a > package in Debian. Alas, the Debian world has not yet caught up with > the other toolsets, and the third-party Debian packaging for > s6-linux-utils and the others has yet to make it into Debian's own > repository. This brings up a question I've had for a long time. Three years ago I constructed an init consisting of Suckless Init as PID1 and s6 as the supervisor, which was called at the end of a simple rc file forked by Suckless Init. It worked just fine, without s6-rc or any other s6 products. From discussions with Laurent I got the idea that the main added functionalities of s6-rc are: 1) Process ordering during boot 2) Ability to intermix longruns with oneshots Everybody appreciates the preceding two features, but personally, I don't think they're absolutely necessary. Runit has neither, yet it works just fine for most things. When I first switched to runit, with its indeterminate ordering, I figured that with the dependencies I built into the run scripts, boot would resemble a field of mousetraps throwing ping pong balls. But that wasn't the case: runit brought everything up quickly and without trouble. So I'm thinking that although process ordering and intermixing are features we'd everyone likes, all but the most discerning among us could get along without them and therefore init with s6 alone. This is more than theoretical. As you remember, Debian's runit package manager wrote to ask about a minor problem with too many .u files and was contemplating changing some code, but nobody knew the code intimately, and runit is pretty much unmaintained at this point. So I suggested s6, which apparently has a Debian package but not one for s6-rc. My opinion is one can boot just fine with s6 alone, as long as you're willing to forego startup ordering and intermixing of longruns and oneshots, which by definition a runit fan is. So what do you all think? Is s6 a useful init system without s6-rc? Thanks, SteveT Steve Litt January 2019 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: s6-ps
On Sat, 5 Jan 2019 18:00:57 -0300 Guillermo wrote: > I don't know why execline is a 'recommends' though. It should be a > 'depends', I believe. I set up a s6 supervisor using all shellscripts and no execline. You don't absolutely need execline. SteveT Steve Litt January 2019 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: More Answers
On Thu, 20 Dec 2018 10:17:58 + Jonathan de Boyne Pollard wrote: > * https://unix.stackexchange.com/a/489949/5132 > > This may be of interest to people looking for some (brief) > comparative analysis. Including the further reading. (-: > I love being on this mailing list. Both your article and Laurent's comment on the same page were tremendously informative. SteveT Steve Litt December 2018 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Is s6/s6-rc ready to be the ubiquitous init?
Hi all, Months ago IBM bought Redhat, and IBM might not want to throw a million a year at a dev group devoted to keeping the systemd leaky boat afloat. Meanwhile, just today another major systemd snafu emerged. If IBM bails on systemd, is s6/s6-rc ready to take its place? I mean we all know it's ready technically, and is well maintained, but is it ready politically, with help for distro packagers? Perhaps some documentation on best practices and making it easy to install s6/s6-rc. We could soon have a rare opportunity for a Linux-wide init change. I'd hate to see systemd replaced by more snake oil. SteveT Steve Litt January 2019 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Dependencies on shutdown?
Hi all, Let's say I have dependencies so that shutdown must be done in the right order. For instance, I want to have my web app close before my database, so that all final writes get done. How do I do that in runit or s6? Thanks, SteveT
Generic interrupt command?
Hi all, I think a cool addition to runit program sv and s6's s6-svc would be a command to send an arbitrary signal to the daemon being supervised. Let's say a -z was added as an arg to s6-svc or a "genericinterrupt" was added as an arg to sv. Now you could say: sv genericinterrupt SIGIO myspecialdaemon s6-svc -z SIGIO myspecialdaemon The supervisor already knows the PID of what's being supervised, so it would be an easy way to get an arbitrary signal into a daemon, for those daemons that have non-standard signal usage. SteveT -- Steve Litt January 2019 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: Is s6/s6-rc ready to be the ubiquitous init?
On Sat, 12 Jan 2019 11:37:55 + "Laurent Bercot" wrote: > >If IBM bails on systemd, is s6/s6-rc ready to take its place? I mean > >we all know it's ready technically, and is well maintained, but is > >it ready politically, with help for distro packagers? Perhaps some > >documentation on best practices and making it easy to install > >s6/s6-rc. > > I'm very thankful for your generous offer to provide documentation > and tutorials. Help for distro packagers is indeed something s6/s6-rc > needs. Can you give me an outline of your tutorial when it's ready? > I'll be happy to proofread it and make suggestions. You're welcome. The docs and tutorials will roll out slowly, but they're roll out. A couple questions: 1) Where can I find the s6/s6-rc project's preferred directories for everything? I'd prefer not to put anything directly off the root directory: Too many people would object. 2) Does there exist a block diagram of either s6, s6-rc, or both combined, and if so, where? Thanks, SteveT Steve Litt January 2019 featured book: Troubleshooting: Just the Facts http://www.troubleshooters.com/tjust
Re: interesting claims
On Wed, 01 May 2019 18:13:53 + "Laurent Bercot" wrote: > >So Laurent's words from http://skarnet.org/software/s6/ were just > >part of a very minor family quarrel, not a big deal, and nothing to > >get worked up over. > > This very minor family quarrel is the whole difference between > having and not having a 100% reliable system, which is the whole > point of supervision. The preceding's true for you, but not for everyone. Some people, like myself, are perfectly happy with a 95% reliable system. I reboot once every 2 to 4 weeks to get rid of accumulated state, or as a troubleshooting diagnostic test. I don't think I'm alone. Some people need 100% reliable, some don't. My liking of supervision is not 100% reliability, but instead 95% reliability that is also simple, understandable, and lets me write daemons that don't have to background themselves. I don't think I'm alone. > Yes, obviously sinit and ewontfix init are greatly superior to > systemd, sysvinit or what have you. Which is why I call it a family quarrel. Some in our family have a strong viewpoint on whether PID1 supervises at least one process, and some don't. But outside our family, most are happy with systemd, which of course makes most of us retch. > That is a low bar to clear. And > the day we're happy with low bars is the day we start getting > complacent and writing mediocre software. I'd call it a not-highest bar, not a low bar. Systemd is a low bar. > > Also, you are misrepresenting my position - this is not the first > time, and it's not the first time I'm asking you to do better. > I've never said that the supervision had to be done by pid 1, actually > I insist on the exact opposite: the supervisor *does not* have to > be pid 1. What I am saying, however, is that pid 1 must supervise > *at least one process*, which is a very different thing. I'm sorry. Either I didn't know the preceding, or I forgot it. And supervising one process in PID1 makes a lot more sense than packing an entire supervisor in PID1. > s6-svscan is not a supervisor. It can supervise s6-supervise > processes, yes - that's a part of being suitable as pid 1 - but it's > not the same as being able to supervise any daemon, which is much > harder because "any daemon" is not a known quantity. I understand now. > Supervising a process you control is simple; supervising a process > you don't know the behaviour of, which is what the job of a > "supervisor" is, is more complex. I understand now. Thanks, SteveT
Re: what init systems do you use ?
On Mon, 13 May 2019 20:13:29 +0100 multiplexd wrote: > All, > > On Fri, May 03, 2019 at 02:53:21AM +0200, Jeff wrote: > > what init systems do this list's subscribers use ? > > I saw Guillermo's reply up-thread, so I thought I'd add my own two > cents. My main workstation at present is a Debian 9 box booted with > s6 and s6-rc, with system setup and teardown (i.e. stages 1 and 3) > handled by some execline scripts heavily adapted from those generated > by s6-linux-init-maker 0.4.x.x. I use runit on all my Void Linux machines. Before using Void, I used sysvinit plus daemontools-encore. Whenever I had any problem with any daemon, or whenever I made my own daemon, I added it on to daemontools-encore, not to sysvinit's comments-actually-mean-something five function essay length init scripts. Experimentally I've initted with suckless-init plus s6. I even did Felker 16 line init + s6, which worked great if you didn't mind not having zombie-killing and signal based shutdown. I've also experimentally initted with Epoch and Systemd. I like the former and dislike the latter. SteveT
Re: chpst -u and supplementary groups
On Mon, 19 Aug 2019 14:08:07 +0200 Jan Braun wrote: > Hello list! > > Yesterday, I spent way too much time chasing down a permissions > problem caused by the fact that "chpst -u acc prog..." only sets the > account's primary group, and ignores any supplementary groups the > account may be a member of. > > TFM mentions "All initial supplementary groups are removed.", but I > failed to memorize that. (Also, what does "initial" signify here?) > > My inability to see the issue came from the fact that all other > similar programs (I'm aware of) do in fact add the supplementary > groups. Watch: > > | # chpst -u test id > | uid=1003(test) gid=1003(test) groups=1003(test) > | # runuser -u test id > | uid=1003(test) gid=1003(test) groups=1003(test),4(adm) > | # s6-setuidgid test id > | uid=1003(test) gid=1003(test) groups=1003(test),4(adm) > | # su - test -c id > | uid=1003(test) gid=1003(test) groups=1003(test),4(adm) > | # su test -c id > | uid=1003(test) gid=1003(test) groups=1003(test),4(adm) > | # sudo -u test id > | uid=1003(test) gid=1003(test) groups=1003(test),4(adm) > | # > > So now I'm wondering: > What are the use cases for not applying existing supplementary groups? > Should chpst apply them by default? > Should chpst grow an option to (not) apply them? > "chpst -u acc: prog..." is still free. > Or is everything as it's supposed to be, and people might need to > munge the output of "getent initgroups acc" and feed it to the -u > option? > > I'll be happy to try to come up with a patch (even if it's still a > fatter warning in the manpage) if people can agree here what the right > thing to do is. > > regards, > Jan Thanks Jan, I was having a similar problem, and inspired by your post, I solved it the following way, using Runit's chpst: chpst -u slitt:audio:disk /tmp/test.sh In the preceding, /tmp/test.sh was permissioned 750 owner root.audio, and contained the following: #!/bin/sh whoami cat /d/audio.group cat /d/disk.group groups pwd file /d/audio.group was 640 root.audio, while /d/disk.group was 640 root.disk. Both cat statements printed out the contents of the files, proving that I was acting with both groups disk and audio. The output of the whoami proved I was acting as user slitt. You're obviously right that having all the groups slitt belongs to, which include both disk and audio, would have been easier, but as you say, it doesn't work right now, so I used this method for the time being. Thanks for helping me solve my problem. SteveT Steve Litt August 2019 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: chpst -u and supplementary groups
On Wed, 21 Aug 2019 05:22:10 +0200 Jan Braun wrote: > This is what vexes me about the daemontools family. Apparently it's so > easy to reimplement them that people keep doing that. Instead of > working together to get one implementation polished enough to make a > big unix distro use it by default. > > cheers, > Jan Far as I'm concerned, they're *all* polished enough to make a big unix distro use it by default. The problem is politics and money. In other words, the best of them all could be rolled into one software, and it still wouldn't be accepted, because Red Hat has many millions for marketing and halloween code to prevent that. The reason daemontools-inspired init/supervisors weren't used before systemd was because nobody was dissatisfied with sysvinit and/or upstart, so nobody even knew about the others. SteveT Steve Litt August 2019 featured book: Twenty Eight Tales of Troubleshooting http://www.troubleshooters.com/28
Re: runit patches to fix compiler warnings on RHEL 7
On Wed, 4 Dec 2019 10:40:14 -0600 "J. Lewis Muir" wrote: > On 12/04, Jonathan de Boyne Pollard wrote: > > Jan Braun: > > > > > 2) runit has manpages. s6 has HTML. :( > > > > > Daniel J. Bernstein had something to say on that subject, two > > decades ago. See the "Notes" section of > > http://cr.yp.to/slashdoc.html . > > > > I generate both manual pages and HTML from a common DocBook XML > > master in the nosh toolset. And the DocBook XML is itself readable > > directly with a WWW browser. > > http://jdebp.uk./Softwares/nosh/guide/commands/setuidgid-fromenv.xml > > is a copy of one such DocBook XML master, for example. It's on the > > WWW, and the packages also install it locally, for off-line > > reading. > > I still like having man pages. It's often just easier to type "man > " than to find the local (or remote) HTML document and open it > in a web browser. I never thought about man pages until a few days ago I did some experiments with execline and had a quick question about syntax. I did man execlineb and of course got "no entry". So I fired up a browser, did a locate command, and put a path in my browser. The browser is vastly superior for learning all about unfamiliar or moderately familiar software, but for the quick lookup of something you primarily know about, there's no substitute for a quick "man execlineb". SteveT Steve Litt December 2019 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Re: s6 usability
On Mon, 2 Dec 2019 17:17:50 -0600 Samuel Holland wrote: > On 12/2/19 3:32 PM, Laurent Bercot wrote: > >> As a guy who has both daemontools and s6 installed on the same > >> box, I thank you from the bottom of my heart for: > >> > >> 1) Prepending s6- to each command so they don't clash with djb's > >> 2) Except for the s6-, naming them the same as djb's so I have > >> less to remember. > > > > Yes, there are a good number of people, me included, who prefer that > > naming scheme. However, Jan's UX return is valid, and if I want to > > make s6 adoption as easy as possible, it needs to be taken into > > account too. > > From a Linux distribution perspective, there's also the question of > if s6 can be made a drop-in replacement for daemontools, since it > does follow djb's naming scheme. In gentoo, there are various > packages that depend on virtual/daemontools; for example, the > nullmailer test suite uses ipcserver. From a quick comparison of the > documentation, it looks like s6 only adds options, and remains > compatible with the daemontools options. > > So would it be valid/acceptable for a distribution to create > unprefixed symlinks to the s6-* binaries? It looks like this would > mostly only work for the subset of the binaries that implement > daemontools functionality; some others (s6-setsid, s6-sudo) would > have naming conflicts if they were not prefixed. > > Then, with the symlinks, s6 could "provide" virtual/daemontools. > Maybe this would also help discoverability (the issue at hand). Maybe > the inconsistency would cause more harm than good, and the symlinks > should be "for compatibility only". > > Thoughts? In my opinion, 99% of all people currently using daemontools are highly technically proficient and could easily either rename commands or prepath to a directory containing prefixless symlinked synonyms. IMHO if the distro does it, they'll find a way to screw it up. And even if the distro does it right, it will screw that 1 in a million people (like me) who occasionally use daemontools and s6 on the same box, switching between them regularly. Personally, I'd leave well enough alone. SteveT Steve Litt December 2019 featured book: Rapid Learning for the 21st Century http://www.troubleshooters.com/rl21
Re: The "Unix Philosophy 2020" document
On Tue, 22 Oct 2019 23:33:41 +0800 "Casper Ti. Vector" wrote: > On Sun, Oct 13, 2019 at 01:37:43AM +0800, Casper Ti. Vector wrote: > > [...] Consequently I think that, with an appropriate amount > > of publicity for the document, much more people would be willing > > to keep an eye on, migrate to, or even help develop daemontools-ish > > systems, potentially creating a mini-avalanche effect or even > > resulting in a turning point in the "init wars". However, the > > influx of people into our circle will also result in a lot of noise > > (especially the noise from some ill-intentioned systemd proponents) > > and a lot of additional technical workload, so I request Laurent > > for a decision on whether to publicise the document; and provided > > he agrees, I request you to help spread it to appropriate places. > > Through private mail with Laurent, I have confirmed that he agrees to > the publicising (not just publication -- it has been on GitLab/GitHub > for some time!) of this document. So please help me spread the > information to where it should be, and direct any interesting > criticism to me if you find it necessary. Thanks in advance. What URL is the best one for us to publicize? -- Steve Litt Author: The Key to Everyday Excellence http://www.troubleshooters.com/key Twitter: http://www.twitter.com/stevelitt
Process Supervision Rosetta Stone
Hi all, I just today started a Process Supervision Rosetta Stone, which shows names of similarly functioned things in daemontools, runit and s6. It's obviously incomplete and probably contains errors, but it's a start. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr
Re: runit patches to fix compiler warnings on RHEL 7
On Sat, 30 Nov 2019 00:21:59 + Colin Booth wrote: > runit-init is slowly becoming less functional and it wouldn't surprise > me if it fails entirely after Debian 10. By what mechanism is runit-init slowly becoming less functional, and what changes in Debian might cause it to fail entirely? Thanks, SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr
Re: s6 usability (was: runit patches to fix compiler warnings on RHEL 7)
On Sat, 30 Nov 2019 10:15:27 + "Laurent Bercot" wrote: > I hear you. Unfortunately, I have no control over what Debian does. > Debian isn't even able to ship a not-broken execline package, so I'm > at a loss on what to do with them. I'm working on a version of > execline that > they *might* accept to package correctly, but it's doubtful as long as > the people in charge are prejudiced against the "lots of small > binaries in /bin" approach. :( Would it be acceptable to you and them to put the binaries in /bin/s6 and then very early in the boot add /bin/s6 to the path? This isn't a lot different from what djb did with /command, except it's not off the root, which everyone seems to hate. [snip] > >3) s6 executables are somehow worse named than runit's. This may be > >highly subjective, but I can recall and recognize the runit > > commands far easier than the s6 ones. Possibly it's the "s6-" > > prefix getting in the way of my brain pattern matching on visual > > appearance of glyph sequences. > >This point is exacerbated by #2 and the number of s6 executables. > >Compare chpst with s6-envdir s6-envuidgid s6-fghack s6-setsid > >s6-setuidgid s6-applyuidgid s6-softlimit. Yes, I know about the > >historical reasons, but still. > > This is very interesting. I thought that having a s6- prefix was a > *good* > thing, because I valued clarity above everything, and especially above > terseness. As a guy who has both daemontools and s6 installed on the same box, I thank you from the bottom of my heart for: 1) Prepending s6- to each command so they don't clash with djb's 2) Except for the s6-, naming them the same as djb's so I have less to remember. [snip] > > >4) s6 seems more complex (hello execline!), and I don't (yet?) see > >any > >benefit/feature I'd appreciate except minimizing wakeups. > > This, on the other hand, is a misconception that really needs to > disappear. Understanding execline is *not needed* to run s6. s6 > depends on execline in two places (there were more, but I scrapped > them because nobody used the commands that were involved): > - at build-time, in s6-ftrig-listen > - at run-time, in s6-log, for processor invocation > > Would entirely removing s6's dependency on execline help clear that > misunderstanding and help with s6 adoption? This could be made > possible by: > - duplicating el_semicolon() functionality in s6-ftrig-listen.c > (it's not elegant, but clearing the dep may be worth it) > - adding an alternative '?' processor directive to s6-log, that spawns > the processor using /bin/sh instead of execlineb. (The '!' directive > would still be there; processor invocations using '!' would just fail > if execline is not installed.) > > I don't like this, but if "execline" is a scarecrow that keeps people > away from s6 for no other reason than perception, it's a possibility. > Savvy users will still install execline for use in run scripts. I don't think it's necessary to remove the dependency unless ordinary users would be altering the code of s6-ftrig-listen or s6-log. A simple change change I think would do it is to change the documentation to imply that, for the *user*, execlineb is a way to get just a little extra whatever. Currently, when I read it, I thought I'd be missing a lot by using /bin/sh. > > s6-rc, however, absolutely cannot do without execline, since it uses > autogenerated execline scripts. But s6-rc is a different beast, that > requires a lot more involvement than s6 anyway, and that isn't needed > at all if we're just talking about runit-like functionality. Does the *user* need to code execline scripts, or is it just something the program does? If the former, then make a point that one doesn't need to use execline for s6-rc to be a very powerful startup system. If anybody would make an execline tutorial, that would help a lot. For a guy like me who only does procedural programming (C, C++, Pascal, Perl, Python, Ruby, Lua, etc), execline is difficult to understand. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr
Re: runit patches to fix compiler warnings on RHEL 7
On Thu, 28 Nov 2019 19:04:37 + "Laurent Bercot" wrote: > - We on the list will gladly help with any question with runit, but > to be honest, I'm not exactly sure what to do with patch upstream > requests for runit. Is anyone processing them and integrating them > into a new release? For submitting patches, I'd recommend working with the Void Linux project. They can be found at #xbps on Freenode IRC. Void Linux has used runit as their init system for the past 5 years: Their implementation is very reliable and mature. > > - I host this list. I'm also the author of s6, a supervision > software suite that is very similar to runit in many ways. s6 is > actively maintained and has a public git repo, and we generally have > a quick response time with requests. > > - My opinion is that the most sustainable path forward, for runit > users who need a centrally maintained supervision software suite, is > to just switch to s6 - and it comes with several other benefits as > well. IMHO not necessarily. There are people whose top priority is simplicity. They value simplicity over guaranteeing against a machine whose supervisor has died, and is now incommunicado. They value simplicity over PID1's ability to supervise one program; the process supervisor (did I get that right?). Such people would prefer runit. Additionally, if a person uses sysvinit as PID1 and only PID1, and puts "respawn runsvdir" in /etc/inittab, then they do get PID1 supervising the supervisor. One other observation: If I wanted the Cadillac of the industry, I'd go with s6. But on a day to day basis, the Chevy of the industry, runit, is good enough for the driving I do. Which leads to the next point: One reason runit has such a large mindshare is because Void Linux and maybe some others ship with runit as their init. s6 has an opportunity to leapfrog. Right now, the Devuan project is discussing supplying run scripts for runit and for s6. Assuming Debian ship with a working s6 (only has to work as a supervisor: sysvinit could be PID1), if the s6 run scripts arrive first, I think s6 would be in a position to become Devuan's default supervisor a year or two from now. I spoke the preceding sentence as an individual, not as a member of the Devuan community. Anyway, if you have a bunch of known-good s6 run (and finish) scripts curated somewhere, everyone would be pleased if you let the Devuan user mailing list know where they are. Thanks, SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr
Re: The "Unix Philosophy 2020" document
On Sun, 13 Oct 2019 01:37:43 +0800 "Casper Ti. Vector" wrote: > However, people told me that the document is not quite accessible to > those who know really little about systemd: one example is they do not > even know much about how the modules are organised in systemd, so the > claim that the systemd architecture has how cohesion and high coupling > may seem unfounded; Hi Casper, About not knowing how systemd modules are organized: NOBODY knows that except Poettering et. al. To my knowledge, there has never been published a systemd block diagram with both the blocks and the interaction lines between those blocks. Systemd "block diagrams" are typically a bunch of blocks in layers, which indicates nothing about how they're organized. So if you defined how the modules were organized, as a block diagram, you would be the first. Contrast this situation with sane init or supervision systems. Here's my block diagram of daemontools: http://www.troubleshooters.com/linux/djbdns/daemontools_intro.htm#daemontools_mental_model If I were to modify that block diagram for runit, the "system boot" and "inittab" would be replaced by runit-init, /etc/runit/1, and /etc/runit/2, with the latter exec'ing (being replaced by) the runit equivalent of svscanboot. With my understanding of s6, if I were to modify it for s6, I'd have the s6 PID1 do some initial stuff, then exec (be replaced by) an executable that does exactly two things: 1) Listen for and act on appropriate signals 2) Supervise the s6 supervisor, which on my diagram is svscanboot. So with s6, PID1 becomes a supervisor that supervises one program, the main supervisor (did I finally get that right, Laurent?) Look at the situation. For daemontools type stuff, a guy who is a Troubleshooting Process Trainer, an office automation Programmer, a tech writer and an author (in other words, NOT an expert on inits) can draw a complete and reasonably accurate block diagram including interaction lines, whereas with systemd the millions of dollars Red Hat spends on the "best and brightest" to produce, maintain, and evangelize systemd cannot produce such a diagram for systemd. This is telling. So when the systemd fanboiz tell you that you haven't provided systemd module interaction, tell them that information is not available, and that's excellent evidence of cohesion, high coupling, and gratuitous complexity. Here's another diagram I use when speaking to those who claim systemd is modular because it has modules: http://troubleshooters.com/linux/systemd/lol_systemd.htm And when speaking with somebody knowledgeable enough to say that all that gratuitous crosstalk goes through one channel, namely dbus, tell them that doesn't matter a bit, because the crosstalk still happens. SteveT Steve Litt Author: The Key to Everyday Excellence http://www.troubleshooters.com/key Twitter: http://www.twitter.com/stevelitt
