Re: Could s6-svscan ignore non-servicedir folders?
On 01/22/15 00:17, Laurent Bercot wrote: On 21/01/2015 21:47, Olivier Brunel wrote: The thing is, that you've referred to services oneshots, whereas I would refer to longrun services oneshot services resp., i.e. I see those as two types of services. They are both services from a system management, higher-level, point of view. From s6-svscan's point of view, longruns are services, oneshots are something it's entirely irrelevant for. In fact, a reason I want/need one folder with everything (servicedirs for both oneshot longrun) is to avoid issues of two services with the same name or such complications when resolving dependencies/ordering. Refer to services as oneshot/foo or longrun/foo, from your system manager's main directory. Now users can name their oneshots and their longruns the same thing, and your dependency manager doesn't care. :) I can't do that, because I need to refer to services by their names, and said name must be (able to be) a filename. Particularly useful for setting things up in subfolders needs/wants/after/before. Otherwise it would complicate things too much, for users possibly, in my code without a doubt, so no. To maybe explain/detail a bit more how what I'm planning would work: my service manager is asked to start some services, it pulls dependencies and ordering from subfolders (of servicedirs) needs/wants/after/before. It then starts everything in order, waiting on some services to be started before others can be. Starting a oneshot service would mean running the start script, and waiting for it; Starting a longrun service would mean sending the command to its supervise process, and waiting for the event on its fifodir. So while the service manager does start services, when it comes to longrun ones it always delegate to s6, as it should. That sounds perfectly reasonable. Make sure to also have a stop script in your oneshot directories, for when you deactivate the service. (To be more precise, all longrun servicedirs will include a down file when created (into /run), to make sure they're started as/when needed. So starting a longrun service actually also includes removing the down file once the event has been received (or when sending the start command, I'm not sure yet which is best) to reflect the change of state.) Yeah, that's ugly, but that's because down files are inherently ugly. I'd say a better solution for the initial boot (which is the only time where you'd need down files) would be to start with an empty, or almost empty, scandir, and have your dependency manager auto-populate it if the longrun service it wants to start isn't being supervised yet. That could be a solution, but I don't really like it. Or, it doesn't play nice with what I wanna do, so I'll stick with down files I think. I understand what you're saying regarding the scandir though. Now I'm thinking I would have a folder with all servicedirs, oneshots longruns, and use a folder .scandir that would only contain symlinks for all the longrun servicedirs. That way, the scandir only has the servicedirs that needs to be supervised, and the service manager still has a single place for all services. That can work, but your ls output will still be confusing, and there *will* come a day when you'll accidentally link a oneshot directory into the scandir. And that will be fun. ls output is mixing oneshots longrun in the main folder, but in the scandir there's only the longrun. Hopefully it won't be that confusing... As for linking a oneshot in there, I plan on having this done automatically, so there shouldn't be such an issue. (But if it happens, I'll just have a supervise failing every 10s to start a non-existing ./run; nothing dramatic.) What do you have against subdirectories ? Nothing at all. It's just that if I don't have one folder with all (oneshots longruns) services, it complicates things in my code, and I don't want that. (I might also say, that I'm planning of having this folder of servicedirs be created on boot into /run, for all enabled services. So it is all runtime information.) Well, you only need the oneshot information during state changes, i.e. mostly boot and shutdown, and on admin intervention in any case. There's no live, automatic state to maintain, so why have oneshot directories eat precious tmpfs space when they could just sleep on your rootfs ? Well, I do in fact have state for oneshots as well, e.g. is it started, failed to start, stopped, etc
Re: Could s6-svscan ignore non-servicedir folders?
On 21/01/2015 18:24, Olivier Brunel wrote: I'll have to setup some scripts for different init stages, using s6-svscan as stage 2, as you've described elsewhere. But I also want to have a system to start (and stop) services in order. I see this whole idea of order/dependency is something that is being talked about, but currently not supported. Yes. A dependency manager is something fundamentally different from a process supervisor, and I definitely do not want to mix the two. They can be tightly coupled, but I'm not going to make the process supervisor more complex to accommodate the dependency manager: all the necessary hooks are already there. nosh gets it right. I don't agree with all of nosh's design principles - especially running the system manager as process 1, which needlessly makes it more complex and increases the height of the supervision tree - but I absolutely agree with its concept of system manager changing the global state by, among others, sending commands to the supervision tree (which nosh calls service manager). This is the right way to perform dependency management with a supervision infrastructure. And it's also a real project, not something I can add to s6 in a week. It needs a lot of work, and it's in my long-term plans, but not in my short-term ones. Furthermore, I want this system of mine to include other kinds of services, that is one-time process/scripts that needs to be run once (on boot), and that's it. And to make things simpler, I want to have it all work together, mixing longrun services (s6 supervised) and oneshot services when it comes to dependency/order definition. So I'll have servicedir of sorts, for oneshot services. And I'm planning of having one folder, that I tend to call runtime repository, but that would also be the scandir for s6-svscan. Obviously though, those aren't servicedirs in the s6 meaning, they shoudln't be supervised, so I'd like for s6-svscan to check if a folder does in fact have a file run, and if not to simply skip/ignore it. Sorry, but it's going to be a frank no. The scandir is the place where s6-svscan does its stuff. It belongs to s6-svscan. Private property. Don't use it for other purposes, and don't mess with it, or you're going to get into trouble. That's intentional, for several reasons. * s6-svscan is a very, very vital and very, very low-level piece of infrastructure. Which means: - The simpler it is, the better. Every line of extra code in it is potentially a bug in your process 1. It's not the place to handle corner cases or convenience issues. s6-svscan and s6-supervise are the two programs where I'm *the most* paranoid about feature creep; everything I can take out of them and put somewhere else instead, I will. - You should not mix interfaces. Clear separation between what is lower-level and what is higher-level is the key to good architecture. s6-svscan should not have to know or care about what a one-shot is. The facts that one-shots and supervisable services should both be handled by the same system manager entity is of no concern to the supervision tree. * Don't mix config-time and run-time. Your one-shot scripts, or directories, belong to a collection of scripts that will be called whenever the global state changes, for instance at boot time or shutdown time. They will only be used when the state changes, not in normal system operation. As such, they belong in your service repository, along with the entirety of your service directories, even those that are not active. That's read-only configuration information. The scandir, on the other hand, is a snapshot of what is currently live; it's run-time information. That's not the same thing, so it doesn't belong in the same place. (That is also why I advise making a live copy of the active service directories: to separate runtime data from offline data.) * Clarity. - In a scandir, it's easy to identify the servicedirs: those are the non-hidden subdirectories of the scandir. Period. If you change that, it will be more complex to detect what is a servicedir and what is a one-shot. ls won't tell you at a glance. Remember how much of a pain it was, figuring out what exactly happened when entering a SysV runlevel, which scripts were one-shots and which ones spawned a daemon ? I don't want to make the same mistake. Especially with a supervision infrastructure, that has the concept of live data that SysV didn't have. So, what do you think of this? Would you be willing to have s6-svscan ignore folders not containing a run file? No. Keep out of the scandir. I swear that your system, and also the software you're developing, and its users, will thank you. Directories are not a scarce resource: you can always make a deeper hierarchy to properly categorize what you're doing. In your case, I would make at least 4 subdirectories of my main work directory: * .../service, the live scandir, containing only symlinks * .../services, which contains the
Re: Could s6-svscan ignore non-servicedir folders?
On 21/01/2015 21:47, Olivier Brunel wrote: The thing is, that you've referred to services oneshots, whereas I would refer to longrun services oneshot services resp., i.e. I see those as two types of services. They are both services from a system management, higher-level, point of view. From s6-svscan's point of view, longruns are services, oneshots are something it's entirely irrelevant for. In fact, a reason I want/need one folder with everything (servicedirs for both oneshot longrun) is to avoid issues of two services with the same name or such complications when resolving dependencies/ordering. Refer to services as oneshot/foo or longrun/foo, from your system manager's main directory. Now users can name their oneshots and their longruns the same thing, and your dependency manager doesn't care. :) To maybe explain/detail a bit more how what I'm planning would work: my service manager is asked to start some services, it pulls dependencies and ordering from subfolders (of servicedirs) needs/wants/after/before. It then starts everything in order, waiting on some services to be started before others can be. Starting a oneshot service would mean running the start script, and waiting for it; Starting a longrun service would mean sending the command to its supervise process, and waiting for the event on its fifodir. So while the service manager does start services, when it comes to longrun ones it always delegate to s6, as it should. That sounds perfectly reasonable. Make sure to also have a stop script in your oneshot directories, for when you deactivate the service. (To be more precise, all longrun servicedirs will include a down file when created (into /run), to make sure they're started as/when needed. So starting a longrun service actually also includes removing the down file once the event has been received (or when sending the start command, I'm not sure yet which is best) to reflect the change of state.) Yeah, that's ugly, but that's because down files are inherently ugly. I'd say a better solution for the initial boot (which is the only time where you'd need down files) would be to start with an empty, or almost empty, scandir, and have your dependency manager auto-populate it if the longrun service it wants to start isn't being supervised yet. I understand what you're saying regarding the scandir though. Now I'm thinking I would have a folder with all servicedirs, oneshots longruns, and use a folder .scandir that would only contain symlinks for all the longrun servicedirs. That way, the scandir only has the servicedirs that needs to be supervised, and the service manager still has a single place for all services. That can work, but your ls output will still be confusing, and there *will* come a day when you'll accidentally link a oneshot directory into the scandir. And that will be fun. What do you have against subdirectories ? (I might also say, that I'm planning of having this folder of servicedirs be created on boot into /run, for all enabled services. So it is all runtime information.) Well, you only need the oneshot information during state changes, i.e. mostly boot and shutdown, and on admin intervention in any case. There's no live, automatic state to maintain, so why have oneshot directories eat precious tmpfs space when they could just sleep on your rootfs ? -- Laurent
