Re: Could s6-scscan ignore non-servicedir folders? [provides-needs deps]
On 01/22/2015 01:33 AM, Avery Payne wrote: This brings to mind the discussion from Jan. 8 about ./provides, where a defining a daemon implies: * the service that it actually provides (SMTP, IMAP, database, etc.); think of it as the doing, the piece that performs work * a data transport (pipe, file, fifo, socket, IPv4, etc.); think of it as how you connect to it * a protocol (HTTP, etc.); think of it as a grammar for conversing with the service, with vertical/specific applications like MySQL having their own grammars, i.e. MySQL-3, MySQL-4, MySQL-5, etc. for each generation that the grammar changes. I'm sure there are other bits and pieces missing. With regard to relationships, if you had a mapping of these, it would be a start towards a set of formal (although incomplete) definitions. From that you could say I need a database that speaks MySQL-4 over a file socket and you could, in theory, have a separate program bring up MySQL 4.01 over a file socket when needed. But do we really need this? The provides-needs relationship is one I've pondered myself (and it's how GNU dmd works), but once again it stumbles conceptually the more I think of it. How do you adequately encapsulate service categories, for instance? Do you actually do any testing (e.g. connect to socket to see if daemon providing service X is up), or do you simply have some sort of queue where you perform name lookups? Such as, mysqld being up causes the database name to be registered globally for all dependent services to be aware? But then what you're doing is poorly reinventing service discovery (like Bonjour/DNS-SD, Consul, etcd, SSDP and so on). Service discovery also implies you're dictating a distributed architecture with a particular configuration scheme (obtaining K-V pairs, or whatever). Unless you check provides using the file system, but that sounds clumsy, racy and again superfluous. And then actually trying to do service availability testing (e.g. via connections)would require a lot of explicit, brittle configuration. But back to service categories, things like database are too generic. You can have multiple DBs of various types - relational, document-oriented, key-value store, on-disk hash table and so on. Such an approach isn't scalable *unless* you design your platform in a specific manner, by having the luxury (or arrogance) of dictating tons of policy, as CoreOS can do, for instance. Once again, provides-needs can be adequately hacked in by doing start-single-instance foobard || exit 1, something to the effect of it. You're adding a thin and useless layer of sugar just to avoid mixing some code with your configuration. Or you just order your services descendingly in the dependency chain, yet again. Provides-needs would work fine for a service-oriented architecture with explicit design and policies, but in a generic supervision framework it just sounds unwieldy. Do try to describe a model where it isn't, though.
Re: Could s6-scscan ignore non-servicedir folders?
On Wed, 21 Jan 2015 18:24:58 +0100 Olivier Brunel j...@jjacky.com wrote: Hi Laurent, So you mentioned breaking compatibility recently, and I figure that might be a good time for me to mention something. I'd like to set up my system around s6, and have been working on this lately. I'll have to setup some scripts for different init stages, using s6-svscan as stage 2, as you've described elsewhere. But I also want to have a system to start (and stop) services in order. I see this whole idea of order/dependency is something that is being talked about, but currently not supported. Can't you do something like this: http://smarden.org/runit/faq.html#depends Furthermore, I want this system of mine to include other kinds of services, that is one-time process/scripts that needs to be run once (on boot), and that's it. And to make things simpler, I want to have it all work together, mixing longrun services (s6 supervised) and oneshot services when it comes to dependency/order definition. I do too. If you have a run-once thing that quickly returns, couldn't you just not exec the thing in the run script, and then have the last statement in your run script write a down file to the service? I'm assuming that s6 does down files the same way as daemontools. Then, all that remains is to have stage 1 of your boot delete all the down files that were put there to achieve run-once. If it really is a daemon, but you don't want it respawned, couldn't you have the finish script (they have those in runit, I don't know about s6) write the down file? You know what you could do? You could make a $servicedir/oneshot shellscript that does something like this: touch ./down cat rm $scriptname/down $whatever/enable_oneshots.sh Then all you'd need to do is run $whatever/enable_oneshots.sh in stage 1, and right after that truncate and make executable $whatever/enable_oneshots. SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance
Re: Could s6-scscan ignore non-servicedir folders?
On 01/21/15 19:03, Steve Litt wrote: On Wed, 21 Jan 2015 18:24:58 +0100 Olivier Brunel j...@jjacky.com wrote: Hi Laurent, So you mentioned breaking compatibility recently, and I figure that might be a good time for me to mention something. I'd like to set up my system around s6, and have been working on this lately. I'll have to setup some scripts for different init stages, using s6-svscan as stage 2, as you've described elsewhere. But I also want to have a system to start (and stop) services in order. I see this whole idea of order/dependency is something that is being talked about, but currently not supported. Can't you do something like this: http://smarden.org/runit/faq.html#depends No. By which I mean, I do not want to use the run script to handle dependency/order, for a few reasons. Such as, I don't think it's its place; It makes things harder to tweak/change, I'll simply use files in subfolder needs/wants/after/before; It also produces things I don't like, where a service is seen as up because its run script is indeed running, but the service hasn't even began to start yet, as the run script is just waiting for another service to be up. I don't like that, I'd rather have something (i.e. external tool) start one service, then waits until it's up to start the other one. I feel a run script should only set things up (limits, environ, etc) and exec into the actual service/daemon, nothing else. It should be as quick as possible, much like the finish script is supposed to be (in s6, 5s until it's killed). Furthermore, I want this system of mine to include other kinds of services, that is one-time process/scripts that needs to be run once (on boot), and that's it. And to make things simpler, I want to have it all work together, mixing longrun services (s6 supervised) and oneshot services when it comes to dependency/order definition. I do too. If you have a run-once thing that quickly returns, couldn't you just not exec the thing in the run script, and then have the last statement in your run script write a down file to the service? I'm assuming that s6 does down files the same way as daemontools. Then, all that remains is to have stage 1 of your boot delete all the down files that were put there to achieve run-once. Well, it might work, but it feels muck hackier to me. Not to mention a down file means the service is meant to be down, whereas the oneshot service would actually be meant to be up (and be active, as in it did start and completed its run); Also that means a supervise running all the time for no good reason for each oneshot service... I don't like that. If it really is a daemon, but you don't want it respawned, couldn't you have the finish script (they have those in runit, I don't know about s6) write the down file? You know what you could do? You could make a $servicedir/oneshot shellscript that does something like this: touch ./down cat rm $scriptname/down $whatever/enable_oneshots.sh Then all you'd need to do is run $whatever/enable_oneshots.sh in stage 1, and right after that truncate and make executable $whatever/enable_oneshots. SteveT Steve Litt* http://www.troubleshooters.com/ Troubleshooting Training * Human Performance
Re: Could s6-scscan ignore non-servicedir folders?
On 1/21/2015 7:19 PM, post-sysv wrote: I'm not sure what effective and worthwhile ways there are to express service *relationships*, however, or what that would exactly entail. I think service conflicts and service bindings might be flimsy to express without a formal system, though I don't think it's anything that pre-start conditional checks and finish checks can't emulate, perhaps less elegantly? This brings to mind the discussion from Jan. 8 about ./provides, where a defining a daemon implies: * the service that it actually provides (SMTP, IMAP, database, etc.); think of it as the doing, the piece that performs work * a data transport (pipe, file, fifo, socket, IPv4, etc.); think of it as how you connect to it * a protocol (HTTP, etc.); think of it as a grammar for conversing with the service, with vertical/specific applications like MySQL having their own grammars, i.e. MySQL-3, MySQL-4, MySQL-5, etc. for each generation that the grammar changes. I'm sure there are other bits and pieces missing. With regard to relationships, if you had a mapping of these, it would be a start towards a set of formal (although incomplete) definitions. From that you could say I need a database that speaks MySQL-4 over a file socket and you could, in theory, have a separate program bring up MySQL 4.01 over a file socket when needed. But do we really need this?
Re: Could s6-scscan ignore non-servicedir folders?
On 01/21/2015 06:09 PM, Wayne Marshall wrote: 4) in general, folks here are letting their panties get far too twisted with the dependency problem. Actual material dependencies are relatively few and can be easily (and best) accomodated directly in the runscript of the dependent service. See the perpok(8) utility for a way to handle dependencies that is suitable in practice for most all installations: I'd like to second this notion, as well. The core issue, the way I interpret it, is that dependencies within the context of services and of libraries, are quite different. A library will at the least need a stub that exports the expected symbols to resolve a dependency. In contrast, at its most primitive, services simply need to be started in an order that descendingly satisfies the dependency chain. Thus, if a dependency system is too weak, then it becomes scantly more than an idealized way of expressing startup ordering, one with a little less administrator effort, but making the feature conceptually uninteresting and of little use. But if it is too powerful, then it incurs a maintenance and complexity cost, ends up requiring complicated scheduling semantics, and thus the whole design starts to suffer. I'm not sure what effective and worthwhile ways there are to express service *relationships*, however, or what that would exactly entail. I think service conflicts and service bindings might be flimsy to express without a formal system, though I don't think it's anything that pre-start conditional checks and finish checks can't emulate, perhaps less elegantly?
