[PATCH] s6-log: Fix tainstamp being truncated
Signed-off-by: Olivier Brunel j...@jjacky.com --- TIMESTAMP is 25 char long, so just enough for the tainstamp with its @-prefix. This was actually overwriting the last character in the tainstamp! src/daemontools-extras/s6-log.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/daemontools-extras/s6-log.c b/src/daemontools-extras/s6-log.c index d460a90..8a5afc0 100644 --- a/src/daemontools-extras/s6-log.c +++ b/src/daemontools-extras/s6-log.c @@ -912,11 +912,11 @@ static void script_run (scriptelem_t const *script, unsigned int scriptlen, char int flagselected = 1, flagacted = 0 ; unsigned int i = 0, hlen = 0 ; char hstamp[32] ; - char tstamp[TIMESTAMP] ; + char tstamp[TIMESTAMP+1] ; if (gflags 1) { timestamp_g(tstamp) ; -tstamp[TIMESTAMP-1] = ' ' ; +tstamp[TIMESTAMP] = ' ' ; } if (gflags 2) { @@ -953,7 +953,7 @@ static void script_run (scriptelem_t const *script, unsigned int scriptlen, char for (j = 0 ; j script[i].actlen ; j++) { act_t const *act = script[i].acts + j ; -siovec_t v[4] = { { .s = tstamp, .len = act-flags 1 ? TIMESTAMP : 0 }, { .s = hstamp, .len = act-flags 2 ? hlen : 0 }, { .s = (char *)s, .len = len }, { .s = \n, .len = 1 } } ; +siovec_t v[4] = { { .s = tstamp, .len = act-flags 1 ? TIMESTAMP+1 : 0 }, { .s = hstamp, .len = act-flags 2 ? hlen : 0 }, { .s = (char *)s, .len = len }, { .s = \n, .len = 1 } } ; switch (act-type) { case ACTTYPE_FD1 : -- 2.3.0
loopwhilex
Hi, Couple of things regarding loopwhilex: - First off, I found the doc to be a bit confusing, because you alternate between talking about when loopwhilex keeps running prog, and when it stops/exits. In fact, it's confusing enough that you yourself got it wrong :p -n : negate the test: run prog... as long as it exits non-zero (or exits a code that is *not* listed in breakcodes). See that not that's in italic on the HTML page, well I believe it should not actually be there, as -n actually means to run prog... as long as it exits a code that *is* listed in breakcodes. Again, might feel odd/confusing because the line above also mentions something when it *is* listed in breakcodes, but it speaks about exiting the loop instead of ending it there. I feel picking one and only talking about that one, to make things clearer/easier to follow, could be a good idea. I'd suggest going with the exits when; say loopwhilex runs prog until it exits non-zero, or exits one of the breakcodes. Then -n means to end the loop if it exits zero, or not listed in breakcodes. Even makes it easier to read: -x 2 means stop on 2; -n -x 0 means stop on non-zero. It might feel a bit odd because loopwhilex -x 2 doesn't mean while it returns 2, but while it doesn't, but that's how it works I guess (loopuntilx might have been a better name?). - The phrasing on the doc led me to think that if prog is killed by a signal, loopwhilex exits (that signal number). But looking at the code I see that that's not the case, instead it will treat it as if prog had returned 126. Why that magic number? Also, shouldn't it always exit when prog was killed by a signal? - And then, I was gonna ask for something, but as I kept thinking about things I went another way, and in the end won't need/use loopwhilex after all. But let me just mention the idea anyhow, just in case: I was gonna say I'd like to get the exit code from prog that caused the loop to end. So, I was thinking an option to make loopwhilex return the exit code from prog, and maybe be killed by the same signal if prog was signaled. But then I thought as an alternative/better choice, how about a new loopuntilx that works the same as loopwhilex, but that takes another block as input: loopuntilx [ -n ] [ -x breakcodes ] { progloop... } prog... It would run progloop until it exits non-zero (or one of the breakcodes) (unless -n ofc) and then exec into prog, having set the ? environment variable to the (last) exit code of progloop, or 256 if signaled. But again, I won't need this after all, so I'm fine if you don't wanna add this. (As a side note though, if foreground would set that ? environment variable to 256 instead of 111 when signaled it might be better, since 111 is a valid exit code after all.) Cheers, -j
Re: loopwhilex
On 02/18/15 01:17, Laurent Bercot wrote: On 18/02/2015 00:22, Olivier Brunel wrote: - First off, I found the doc to be a bit confusing, because you alternate between talking about when loopwhilex keeps running prog, and when it stops/exits. Yes, it's confusing, but it's still accurate. In fact, it's confusing enough that you yourself got it wrong :p Nope, the doc is correct here. At least in intent; if loopwhilex's behaviour is different, then it's a bug. Alright, so let me quote you from that very mail, about behavior when using both -n and -x : With the -x option, the exit loop codes are the listed argument and the continue looping code is all the rest. With -x, breakcodes are when loopwhilex exits. So with -n, they are when it continues, since it negates it, correct? Let me repeat: you're saying with both -n -x, breakcodes are when it *does* loop. Ok, now let me quote you the doc for the same case, using both -n -x : It will run prog... as long as it exits a code that is not listed in breakcodes (there's even emphasis on the not). Let me repeat: the doc is saying with both -n -x, breakcodes are when it does *not* loop. Clearly, one of you is wrong. I still say the doc is wrong there. The problem is, loopwhilex behaves differently when the -x option is given and when it's not. Without the -x option, the continue looping code is 0 and the exit loop code is all the rest. With the -x option, the exit loop codes are the listed argument and the continue looping code is all the rest. That is why it's confusing: the explicit set of codes vs. all the unlisted codes meaning is reversed when you give the -x option. The -x option was added as an afterthought, and the -n option after that one again. I only later realized how confusing it was. The right solution is to replace the -x option with another one, where you list the ok codes, not the exit ones, and if you want -x behaviour, you add -n. I'll do that for the next release. - The phrasing on the doc led me to think that if prog is killed by a signal, loopwhilex exits (that signal number). But looking at the code I see that that's not the case, instead it will treat it as if prog had returned 126. Why that magic number? Also, shouldn't it always exit when prog was killed by a signal? Hmmm, it probably should. I'll check and fix it accordingly. The magic number comes from a time when I didn't have a clear vision of what to do when programs get killed, and I didn't want to think about it too much, so I treated a death by signal like an exit with failure by default. I was gonna say I'd like to get the exit code from prog that caused the loop to end. So, I was thinking an option to make loopwhilex return the exit code from prog, and maybe be killed by the same signal if prog was signaled. Signal propagation is dangerous, and I don't like to use it when it's not strictly necessary. loopuntilx [ -n ] [ -x breakcodes ] { progloop... } prog... That was actually how it worked in early versions of execline; the command was called loopwhile. But with use, I realized that implementing a sequence of operations where it's not necessary led to code duplication; add a foreground around the loopwhilex if you want that behaviour. Yes, except when you want the exit code from progloop of course, which was what I was trying to address here in the first place... I like chain loading, a lot, but sometimes exiting is just the right thing. :) That's why there's forx (replaced for), forbacktickx (replaced forbacktick), and ifte (replaced ifthenelse, that is still around because the order of blocks is more intuitive and I actually still have a script somewhere relying on ifthenelse -s *shudder*, but that should go the way of the dodo at some point). But yes, even with a foreground construct, your following comment is valid: (As a side note though, if foreground would set that ? environment variable to 256 instead of 111 when signaled it might be better, since 111 is a valid exit code after all.) Absolutely, it's the right thing, like the fix to finish scripts.
Re: How to report a death by signal ?
On 02/18/15 14:20, Laurent Bercot wrote: On 18/02/2015 14:04, Olivier Brunel wrote: I don't follow, what's wrong with using a fd? It needs a convention between G and P. And I can't do that, because G and P are not necessarily both execline commands. They are normal Unix programs, and the whole point of execline is to have commands that work transparently in any environment, with only the Unix argv and envp as conventions. But isn't the whole anything = 128 will be reported as 128, and anything higher is actually 128+signum also a convention that both needs to agree upon? P will do something to report info about C to G, and P and G needs to agree about how said info will be reported. Using 128+signum is one way, using an fd for the full/correct info is another. The later being an option, it wouldn't change what P returns, but be an additional means to provide accurate information to grandprocess G should they need to. Or just, like shells, assume it's not needed and simply only do the 128+signum convention. Noting that shells do not actually clamp the exit code to 128. As illustrated by Peter's example, shells return the exit code (up to 255 included), or 128+signum. So assuming no signal, you get the accurate exit code. But of course, with anything higher than 128 there's no way of knowing if it was an exit code or a signal (unless you know exit codes don't go that high). (Clamping provides better results though, so I'm not saying don't do it; Just the difference shall probably be pointed out/documented.) Cause that was my idea as well: return the exit code or 255. I was considering it for a while, then figured that the signal number is an interesting information to have, if G remotely cares about C crashing. I prefer to reserve the whole range of 128+ for something went very wrong, most likely a crash at some point, and if you get 129+, it was directly below you and you get the signal number. Though if you want shell compatibility you could also have an option to return exit code, or 128+signum when signaled, and similarly one would either be fine with that, or have to use the fd for full/complete info. Programs that can establish a convention between one another are easy to deal with. If I remember to document the convention (finish scripts *whistle*)
Re: Feature requests for execline s6
On 01/26/15 23:18, Laurent Bercot wrote: - execline: I'd like the addition of a new command, e.g. readvar, that would allow to read the content/first line of a file into a variable. IOW, something similar to importas (including an optional default value), only instead of specifying an environment variable one would give a file name to load the value from (in a similar manner as to what s6-envdir does, only for one var/file and w/out actually using environment variable). How about backtick -n SOMETHING { redirfd -r 0 FILE cat } import -U SOMETHING ? Sure it's longer, but it's exactly what the readvar command would do. You can even program the readvar command in execline. ;) That looks good, except that I need a default value, in case the file doesn't exist. If there's a way to work a default value in that case then that'd be fine, yes. ...is there a way to add a default value if FILE doesn't exist? If for some reason it's inconvenient for you, I can envision writing something similar to readvar, but it *is* going to use the environment: I've been trying to reduce the number of commands that perform substitution themselves, so that passing state via the environment is prioritized over rewriting the argv, and import is kind of the one-stop-shop when you need substitution. I don't want to go back on that intent and add another substituting command. Is it a problem for you to use the environment ? No it's not, it just felt unneeded when thinking of readvar, is all. But i could combine it with import, as I said what I really want is to have a default value if the file doesn't exist, instead of nothing/empty string, skipping/not using the environment was/is not a requirement at all. - s6-log: I think a new action to write to a file could be useful. The problem with that is the whole design of s6-log revolves around controlling the size of your logging space: no uncontrolled file growth. Writing to a file would forfeit that. What I'm considering, though, is to add a spawn a given program action, like a !processor but every time a line is selected and triggers that action. It would answer your need, and it would also answer Patrick's need without fswatch. However, it would be dangerous: a misconfiguration could uncontrollably spawn children. I'll do that when I find a safe way to proceed. Yeah that would work for me... I thought the writing to file action was maybe more generic, but I get your point re: controlling the logging size. Running an external program would work for me just as well, yes. Looking forward to this nice little addition. (re: misconfiguration, in what way though? It could lead to the program being triggered for each new line, but that doesn't mean uncontrollably spawning children? It might be a lot, but with a rotation every 4096 bytes one might already have frequent calls to the processor, no?) -j
[PATCH] envuidgid: Fix error message
Signed-off-by: Olivier Brunel j...@jjacky.com --- src/daemontools-extras/s6-envuidgid.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/daemontools-extras/s6-envuidgid.c b/src/daemontools-extras/s6-envuidgid.c index 3a58c3c..3712411 100644 --- a/src/daemontools-extras/s6-envuidgid.c +++ b/src/daemontools-extras/s6-envuidgid.c @@ -67,7 +67,7 @@ int main (int argc, char const *const *argv, char const *const *envp) if (n 0) strerr_diefu2sys(111, get supplementary groups for , argv[0]) ; } - else if (insist) strerr_dief2x(1, unknown user: , argv[1]) ; + else if (insist) strerr_dief2x(1, unknown user: , argv[0]) ; { unsigned int pos = 0 ; -- 2.3.3
s6-devd, stdin stdout
Hi, I have a question regarding s6-devd: why does it set its stdin stdout to /dev/null on start? This seems non-obvious to me, and isn't documented. I'm specifically wondering about stdout, since it means all the children/helpers launched will also have their stdout be /dev/null, so anything they printed actually gets lost, whereas I was expecting to find it logged, since I set s6-devd to have its std{out,err} sent to its s6-log. Unless there's a reason I'm missing, maybe those forced redirections should be removed? Also, completely unrelated, but I noticed a bug in the doc for s6-setuidgid: As it turns out, this only calls s6-envuidgid s6-applyuidgid, but - according to the doc - there's a difference in behavior. Specifically, the doc for s6-setuidgid says: If account contains a colon, it is interpreted as uid:gid, else it is interpreted as a username and looked up by name in the account database. This doesn't seem to be true (anymore?). Cheers, -j
[PATCH] examples: Fix syslog LOGSCRIPT
Log lines are actually prefixed with uids from $IPCREMOTEEUID $IPCREMOTEEGID, so they should be acocunted for in the regexs. Signed-off-by: Olivier Brunel j...@jjacky.com --- Also note the need to use \s because, AFAIK, there's no way to use spaces in the regex then, as space is a delimiter for splitting. This is probably important enough to be noted alongside in some README in fact, since that's why I've personally kept them in the run file. .../ROOT/img/services-local/syslogd-linux/log/env/LOGSCRIPT| 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/examples/ROOT/img/services-local/syslogd-linux/log/env/LOGSCRIPT b/examples/ROOT/img/services-local/syslogd-linux/log/env/LOGSCRIPT index 65c53e6..e7a1eb7 100644 --- a/examples/ROOT/img/services-local/syslogd-linux/log/env/LOGSCRIPT +++ b/examples/ROOT/img/services-local/syslogd-linux/log/env/LOGSCRIPT @@ -1,6 +1,6 @@ -- +^error\. t /var/log/syslogd/error -- +^authpriv\. t /var/log/syslogd/auth -- +^user\. t /var/log/syslogd/user -- +^messages\. t /var/log/syslogd/messages -- +^daemon\. t /var/log/syslogd/daemon +- +^[0-9]+:\s[0-9]+:\serror\. t /var/log/syslogd/error +- +^[0-9]+:\s[0-9]+:\sauthpriv\. t /var/log/syslogd/auth +- +^[0-9]+:\s[0-9]+:\suser\. t /var/log/syslogd/user +- +^[0-9]+:\s[0-9]+:\smessages\. t /var/log/syslogd/messages +- +^[0-9]+:\s[0-9]+:\sdaemon\. t /var/log/syslogd/daemon f t /var/log/syslogd/misc -- 2.3.1
Re: s6-rc design ; comparison with anopa
On 04/23/15 17:40, Laurent Bercot wrote: (...) Three kinds of services --- Like anopa, s6-rc works internally with two kinds of services: longrun, which is simply defined by a service directory that will be directly managed by s6, and oneshot, which is defined by a directory containing data (a start script, a stop script, and some optional stuff). s6-rc allows the user to provide a third kind of service: a bundle. A bundle is simply a set of other services. Starting a bundle means starting all the services contained in the bundle. A bundle can be used to emulate a SysV runlevel: the user can put all the services he needs into a single bundle, then tell s6-rc to change the machine state to exactly that bundle. Bundles can of course contain other bundles. A oneshot or a longrun are called atomic services, as opposed to a bundle, which is not atomic. Bundles are useful for the user, because oneshot and longrun are often too small a granularity. For instance, the Samba service is made of two longruns, smbd and nmbd, but it's still a single service. So, samba would be a bundle containing smbd and nmbd. FYI, this is kinda possible with anopa, since one can have empty services, i.e. a servicedir without any script (no run, start nor stop). Those can be used to order things, but one could also have such a service (treated as oneshot internally, just without start or stop script) with a few dependencies. Then, starting it will start all the dependencies in order, as you'd expect. However, there isn't anything runlevel-like, as your switch to that bundle command. (One can add aa=foobar on the kernel cmdline to use /etc/anopa/onboot/foobar as listdir to aa-start (during stage 2) instead of /etc/anopa/onboot/default but that's it). On that note, one thing you've apparently done/planned is auto-stopping, whereas there is no such thing in anopa. This is because I always felt like while auto-starting can be easily predictable/have expected behavior, things aren't the same when it comes to stopping. That is, start httpd and it will auto-start dependency php which will auto-start dependency sqld; fine. But I'm not sure stopping sqld should stop php, despite the dependency. Maybe one just wants to shut down sqld, not the whole webserver. Then there's also the case of, imagine you start foobar web. web is a bundle with httpd, php sqld, foobar is a service w/ a dependency on sqld. Now you stop web; should sqld be stopped? about foobar? What is the expected behavior here? I'm not sure there's one, different people/scenario will have different expectations... it always seemed too complicated so I went with no auto-stopping at all. Also, the smbd daemon itself could want its own logger, smbd-log. Correct daemon operation depends on the existence of a logger (a daemon cannot start if its logger isn't working). So smbd would actually be a bundle of two long-runs, smbd-run (which is the smbd process itself) and smbd-log (which is the logger process), and smbd-run would depend on smbd-log. Just so I understand: why are you talking about smbd-log as a separate service, and not the logger of service smbd, as usual with s6? Or is that what you're referring to here with smbd-log? Users who want to start Samba don't want to deal with smbd-run, smbd-log, nmbd-run and nmbd-log manually, so they would just start samba, and s6-rc would resolve samba to the proper set of atomic services. Source, compiled and live - Unlike anopa, s6-rc does not operate directly at run-time on the user-provided service definitions. Why ? Because user-provided data is error-prone, and boot time is a horrible time for debugging. Also, s6-rc uses a complete graph of all services for dependency management, and generating that graph at run-time is costly. Instead, s6-rc provides a s6-rc-compile utility that takes the user-provided service definitions, the source, and compiles it into binary form in a place in the root filesystem, the compiled. At run-time, s6-rc ignores the source, but reads its data from the compiled, which can be on a read-only filesystem. It also needs a read-write place to maintain information about its state; this place is called the live. Unlike the compiled, the live is small: it can reside in RAM. I'm not sure where you put the scandir in this? I believe original servicedirs where taken from the source and put into the compiled, and s6-rc will create the actual servicedirs in the scandir with s6-rc-init, correct? So that would be part of live? How about the oneshot scripts? The point of this separation is multifold: efficiency (all checks, parsing and graph generation performed at compile-time), safety (the compiled can be write-protected), and clarity (separation of user- modifiable data, current configuration data, and current live data). Atomic services can be very small. It can be a single
[PATCH] devd: Fix invalid option used for s6-uevent-listener
Signed-off-by: Olivier Brunel j...@jjacky.com --- src/minutils/s6-devd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/minutils/s6-devd.c b/src/minutils/s6-devd.c index 9df5c80..74f2d41 100644 --- a/src/minutils/s6-devd.c +++ b/src/minutils/s6-devd.c @@ -63,7 +63,7 @@ int main (int argc, char const *const *argv, char const *const *envp) } if (kbufsz != 65536) { - newargv[m++] = -k ; + newargv[m++] = -b ; newargv[m++] = fmt + pos ; pos += uint_fmt(fmt + pos, kbufsz) ; fmt[pos++] = 0 ; -- 2.3.6
build system license
Hey Laurent, Quick question regarding your build system, that is ./configure or tools/gen-deps.sh and the likes: how are they licensed? Most of your source files have a ISC license mention, but those do not. Are they also released under the same license, or? (Asking cause I'd like to use them in a little project of mine, but wanna make sure it's alright.) Cheers, -j
[PATCH] forstdin: Fix possible error/hanging w/ -p
Basically there's a race condition with signals, and it goes something like this: - we launches child1, and add pid1 to the array of pids - we launches child2, when suddenly! - SIGCHLD happens: in the handler, we wait() for pid1, removing it from pids, then wait for pid2, ignoring it since it's not (yet) in the array, wait again and be done - back in the normal path, we now add pid2 to pids - later on, we believe there's still a reason to wait (pid2) when in fact, there's not. Leading to e.g. ECHILD in waitn() (Noting that this wasn't limited to a single extra pid in the end.) Blocking SIGCHLD until we've added the pid to the array ensures this doesn't happen. (Thus we bring back the call to waitn removed in 5053ea39, which didn't really fix the issue.) Signed-off-by: Olivier Brunel j...@jjacky.com --- Hey Laurent, So I have been hit by a bug in forstdin for some time, and finally took some (time that is) to look into it. Then saw you (tried to) address it recently (in 5053ea39) as I pulled to maybe rebase my fix... However, I'm not sure your fix really does the trick, plus (as you've seen by now) I have an explanation for the bug, so there. This bug would manifest in two ways for me, either forstdin would exit 111 (w/ ECHILD in waitn()), or it would just hang indefinitely. I believe this was two manifestations of the same issue, and due to some compilation options or something I'd either get one or the other. I never really looked into it more, honestly, and the one I faced working on it was the exit 111 (I got the other from my package compilation, where things are compiled w/ different options possibly.) Anyhow, I'm not sure about your fix with sig_pause(), but I did a quick try and got it to hang indefinitely I'm afraid. So here's a proper fix (I hope), also restoring the waitn() call. Cheers, -j PS: If I may, little tip/advice re: git: this is why it's better to separate things when you commit. That is, if you'd made a commit to address this bug only, one for the bugfix for forbacktickx and another one to prepare for the rc 2.1.2.2, it might be better/make things easier. That is, one commit only means/relates to one thing, so when mentioning it it's clearer what issue we're dealing with. Also it's easier when using blame later on, or to e.g. revert it, as might have been done here. src/execline/forstdin.c | 14 -- 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/src/execline/forstdin.c b/src/execline/forstdin.c index 3fb52eb..b620c89 100644 --- a/src/execline/forstdin.c +++ b/src/execline/forstdin.c @@ -128,12 +128,14 @@ int main (int argc, char const **argv, char const *const *envp) if (!stralloc_0(modif)) strerr_diefu1sys(111, stralloc_0) ; if (!env_merge(newenv, envlen+2, envp, envlen, modif.s, modif.len)) strerr_diefu1sys(111, merge environment) ; + if (pids.s) sig_block(SIGCHLD) ; pid = el_spawn0(argv[1], argv + 1, newenv) ; if (!pid) strerr_diefu2sys(111, spawn , argv[1]) ; if (pids.s) { if (!genalloc_append(pid_t, pids, pid)) strerr_diefu1sys(111, genalloc_append) ; +sig_unblock(SIGCHLD) ; } else { @@ -147,11 +149,11 @@ int main (int argc, char const **argv, char const *const *envp) stralloc_free(modif) ; } if (pids.s) -for (;;) -{ - sig_block(SIGCHLD) ; - if (!pids.len) break ; - sig_pause() ; -} + { +if (sig_restore(SIGCHLD) 0) + strerr_diefu2sys(111, restore, SIGCHLD handler) ; +if (!waitn(genalloc_s(pid_t, pids), genalloc_len(pid_t, pids))) + strerr_diefu1sys(111, waitn) ; + } return 0 ; } -- 2.4.2
[PATCH] s6-uevent-spawner: Fix possibly delaying uevents
Because of the buffered IO, the possible scenario could occur: - netlink uevents (plural) occur, i.e. data ready on stdin - iopause triggered, handle_stdin() called. The first uevent is processed, child launched, we're waiting for a signal - SIGCHLD occurs, we're back to iopausing on stdin again, only it's not ready yet; Because we've read it all already and still have unprocessed data (uevents) on our own internal buffer (buffer_0) This could lead to e.g. plug an USB drive, and see only half the uevents be processed. Generate a new uevent, and see all the old uevents then be processed, and maybe the new ones too, maybe only part of them. Rince, repeat. We now call handle_stdin() if the buffer isn't empty, so it'll process any such buffered uevents, knowing that it will read if needed (and we can pretty much assume the data will be there in this case). Signed-off-by: Olivier Brunel j...@jjacky.com --- src/minutils/s6-uevent-spawner.c | 25 - 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/src/minutils/s6-uevent-spawner.c b/src/minutils/s6-uevent-spawner.c index 40f088e..f7e2456 100644 --- a/src/minutils/s6-uevent-spawner.c +++ b/src/minutils/s6-uevent-spawner.c @@ -219,17 +219,24 @@ int main (int argc, char const *const *argv, char const *const *envp) while (cont || pid) { -register int r = iopause_g(x, 1 + (!pid cont), deadline) ; -if (r 0) strerr_diefu1sys(111, iopause) ; -else if (!r) handle_timeout() ; +register int ready = !pid cont ; + +if (ready !buffer_isempty(buffer_0)) + handle_stdin(sa, linevar, argv, envp) ; else { - if (x[0].revents IOPAUSE_EXCEPT) -strerr_diefu1x(111, iopause: trouble with selfpipe) ; - if (x[0].revents IOPAUSE_READ) -handle_signals() ; - else if (!pid cont (x[1].revents IOPAUSE_READ)) -handle_stdin(sa, linevar, argv, envp) ; + register int r = iopause_g(x, 1 + ready, deadline) ; + if (r 0) strerr_diefu1sys(111, iopause) ; + else if (!r) handle_timeout() ; + else + { +if (x[0].revents IOPAUSE_EXCEPT) + strerr_diefu1x(111, iopause: trouble with selfpipe) ; +if (x[0].revents IOPAUSE_READ) + handle_signals() ; +else if (ready (x[1].revents IOPAUSE_READ)) + handle_stdin(sa, linevar, argv, envp) ; + } } } if (verbosity = 2) strerr_warni1x(exiting) ; -- 2.4.3
Re: [PATCH] s6-uevent-spawner: Fix possibly delaying uevents
On 06/14/15 21:11, Laurent Bercot wrote: On 14/06/2015 14:37, Olivier Brunel wrote: Because of the buffered IO, the possible scenario could occur: - netlink uevents (plural) occur, i.e. data ready on stdin - iopause triggered, handle_stdin() called. The first uevent is processed, child launched, we're waiting for a signal - SIGCHLD occurs, we're back to iopausing on stdin again, only it's not ready yet; Because we've read it all already and still have unprocessed data (uevents) on our own internal buffer (buffer_0) Right, thanks for the catch. I usually avoid that trap, but meh. I committed a simpler change than your patch, please tell me if it fixes things for you. Haven't tested it, but I'm sure it'll work. In fact I had done something similar at first, but changed it because I don't think this is quite correct. That is, in your test now you're using x[1] even though it might not have been used in the iopause call before, so while I guess this isn't random memory, it doesn't really feel right either. Moreover, imagine this very likely scenario: - uevent comes in, handle_stdin is called, children forked, and we're iopausing in x[0] (selfpipe) only. - SIGCHLD occurs, handle_signals is called, then you test on x[1] which should still be as it was last time it was actually used, so stating stdin is readable even though it isn't anymore... and we're gonna block in handle_stdin. Not that it's a big deal since at this point we were gonna iopause to wait for that fd to become readable anyways, since we're not expecting any signal (as we don't have children anymore). So in effect this works basically the same, but you're using memory (x[1]) that you probably shouldn't, and effectively trying to read (and blocking) in handle_stdin despite having an iopause in place. You might as well change your test to if (cont) handle_stdin(...) ; then, it'll surely work just as well. But it doesn't feel right, to me at least, since we have a loop w/ iopause, hence why I went with the bit longer (and maybe less elegant) version. -j
bitarray: Add first{clear,set}_skip
Hey Laurent, So skalibs contains a few functions to work with bit arrays, but only some of the supported operations can be done with an offset (unless I missed it). One can obviously clear, set or peek at any bit within the array, but also set/clear a group of bits from an offset via bitarray_clearsetn(). However, to get the first bit clear/set within a group, the functions bitarray_first{clear,set}() do not support an offset, even though it might be useful -- I certainly had the need for it recently. I think it'd be a nice addition to support this. Since I had to do it, you'll find below the code for it, feel free not to use it & implement this differently as you wish ofc (I'm using git to send this as that's better to properly send code, but this isn't a patch per-se, since I didn't do touch doc/deps or nothing; Also this would make a terrible commit message :p), or point me to what I missed to do so already, if that's the case. For the record, I believe there might be more functions that do not support use with such an offset, e.g. bitarray_countones(), but I didn't need any of that so I didn't look further. Cheers, -j --- src/libstddjb/bitarray_firstclear_skip.c | 22 ++ src/libstddjb/bitarray_firstset_skip.c | 22 ++ 2 files changed, 44 insertions(+) create mode 100644 src/libstddjb/bitarray_firstclear_skip.c create mode 100644 src/libstddjb/bitarray_firstset_skip.c diff --git a/src/libstddjb/bitarray_firstclear_skip.c b/src/libstddjb/bitarray_firstclear_skip.c new file mode 100644 index 000..1361fb2 --- /dev/null +++ b/src/libstddjb/bitarray_firstclear_skip.c @@ -0,0 +1,22 @@ + +#include + +unsigned int +bitarray_firstclear_skip (register unsigned char const *s, unsigned int max, unsigned int skip) +{ +unsigned int n = bitarray_div8(max) ; +register unsigned int i = bitarray_div8(skip) ; +if (i && s[i - 1] != 0xffU) +{ +register unsigned int j = skip ; +skip = i << 3 ; +if (skip > max) skip = max ; +while ((j < skip) && bitarray_peek(s, j)) ++j ; +if (j < skip) return j ; +} +for (; i < n ; ++i) if (s[i] != 0xffU) break ; +if (i == n) return max ; +i <<= 3 ; +while ((i < max) && bitarray_peek(s, i)) ++i ; +return i ; +} diff --git a/src/libstddjb/bitarray_firstset_skip.c b/src/libstddjb/bitarray_firstset_skip.c new file mode 100644 index 000..5d77a0c --- /dev/null +++ b/src/libstddjb/bitarray_firstset_skip.c @@ -0,0 +1,22 @@ + +#include + +unsigned int +bitarray_firstset_skip (register unsigned char const *s, unsigned int max, unsigned int skip) +{ +unsigned int n = bitarray_div8(max) ; +register unsigned int i = bitarray_div8(skip) ; +if (i && s[i - 1]) +{ +register unsigned int j = skip ; +skip = i << 3 ; +if (skip > max) skip = max ; +while ((j < skip) && !bitarray_peek(s, j)) ++j ; +if (j < skip) return j ; +} +for (; i < n ; ++i) if (s[i]) break ; +if (i == n) return max ; +i <<= 3 ; +while ((i < max) && !bitarray_peek(s, i)) ++i ; +return i ; +} -- 2.6.4
[PATCH] bitarray_not: Fix skipping a byte
Signed-off-by: Olivier Brunel <j...@jjacky.com> --- src/libstddjb/bitarray_not.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libstddjb/bitarray_not.c b/src/libstddjb/bitarray_not.c index 4bd95ad..de37b9f 100644 --- a/src/libstddjb/bitarray_not.c +++ b/src/libstddjb/bitarray_not.c @@ -12,7 +12,7 @@ void bitarray_not (register unsigned char *s, register unsigned int a, register { register unsigned int i = (a>>3) + 1 ; s[a>>3] ^= ~((1 << (a & 7)) - 1) ; -for (; i < (b>>3) - 1 ; i++) s[i] = ~s[i] ; +for (; i < (b>>3) ; i++) s[i] = ~s[i] ; s[b>>3] ^= (1 << (b & 7)) - 1 ; } } -- 2.8.2
Re: [announce] Fall 2016 release
On Fri, 28 Oct 2016 15:46:25 + "Laurent Bercot"wrote: > * execline-2.2.0.0 > > > - Compatibility to skalibs-2.4.0.0. > - New option "-I" to backtick and withstdinas. > - New option "-t" to wait: it can now wait for children with a > timeout. > - Minor bugfixes. I believe there was also a "-s" option added to execlineb ;)
backtick segfault
Hey, So first off, quick question about the doc of multisubstitute, where it reads under Options: "If a backtick directive was given with the -i option..." I don't get that. There's no backtick directive listed above, and trying it use one doesn't seem to work ("unrecognized directive backtick"). Am I missing something? Is this line not supposed to be here, or? Then, speaking of backtick, seems it segfaults if ran w/out a prog to exec into, e.g: backtick var { pwd } Cheers,
Re: How to write an execline helper in execline?
On Sun, 16 Oct 2016 17:10:23 + "Laurent Bercot" <ska-skaw...@skarnet.org> wrote: > Ew. Shows how much that code is used - in more than two years, > nobody (including me) has noticed the segfault. > > Since it's so widely used, and since diving back into environment > frame pushing/popping gives me headaches, I have half a mind to just > remove runblock in the next release (which is a major version bump so > compatibility isn't guaranteed). I'll try fixing the bug for a little > more > time, and if I can't find it, I'll scrap the whole thing. Had a quick look, and from what i could gather the issue is that in our case, nothing gets put into v, so by the time pathexec_run() gets called, v only contains a NULL, and therefore v[0] (well, it's v.s[0] but hopefully you get what I'm saying) - which is given as first arg - is indeed, NULL. So the "file" argument for pathexec_run() is NULL, which ends up in execvpe() as a call to strchr() with NULL as string/1st arg, and that's your segfault (in libc, in musl it actually happens in strchrnul). The attached patch seems to fix it, and also address the other case: $ ./helper foo bar baz runblock: fatal: unable to exec bar: No such file or directory $ ./helper foo bar runblock: fatal: unable to exec bar: No such file or directory $ ./helper foo runblock: fatal: too few arguments The "return 0" is what made it return w/out even trying to exec into anything, I'm guessing this was a mistake trying to catch when there was no rest of command line to exec into? > I have implemented your -s suggestion in the latest execline git > head; that one was easy. Please test it and tell me if it works for > you. Yep, works great! Thanks! Cheers, >From d01f2fe92be9d50d5592786b5aec08cabb41e001 Mon Sep 17 00:00:00 2001 From: Olivier Brunel <j...@jjacky.com> Date: Sun, 16 Oct 2016 20:38:55 +0200 Subject: [PATCH] runblock: Fix segfault Signed-off-by: Olivier Brunel <j...@jjacky.com> --- src/execline/runblock.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/execline/runblock.c b/src/execline/runblock.c index a654023..78a4b59 100644 --- a/src/execline/runblock.c +++ b/src/execline/runblock.c @@ -77,7 +77,7 @@ int main (int argc, char const *const *argv, char const *const *envp) if (flagr) /* put remainder envvars into v */ { -if (++m == sharp) return 0 ; +if (++m > sharp) strerr_dief1x(100, "too few arguments") ; if (!genalloc_ready(char const *, , sharp - m + 2)) strerr_diefu1sys(111, "genalloc_ready") ; for (; m <= sharp ; m++) -- 2.10.0
emptyenv doc bug
Hey, Going through some execline doc these days, and I noticed the emptyenv doc states: -o : pop environment variables starting with ELGETOPT_. You might want to do this before executing a final program from a script that uses elgetpositionals. I think this is wrong, as it should be elgetopt instead of elgetpositionals. Cheers,
Re: [PATCH] s6-supervise: Optionally run child in a new pid namespace
On Sat, 15 Jul 2017 20:24:25 + "John O'Meara"wrote: > You can achieve a PID namespace (and others) using the unshare > program from util-linux without patching s6. Put the following at the > top of your run script: > > unshare -fp --mount-proc > > this also has the advantage of clearly showing which services are in > their own namespaces when looking at a ps listing, especially for > forest views ("ps f" or "s6-ps -H") > Though as Jesse explained, this requires some sort of exit/signal proxing, which isn't the case here. Here the direct child of s6-supervise remains the daemon itself - in its own pid ns - which is much better. As far as showing which services are in their own ns, most namespaces (i.e. all but pid) won't require a fork and usually you'd get unshare/nsenter to just exec into the daemon, again to get proper supervision. So FWIW I'm very much for this patch myself :) It is a linux-specific thing, but done as it is w/ a compile-time option this shouldn't be an issue I'd hope, and it allows simple "proper" supervision of services running (as pid1) in their own pid ns. So, thanks Jesse! Cheers,
Re: [PATCH] Fix openreadnclose failing if errno was nonzero before
On Mon, 22 May 2017 00:44:28 + "Laurent Bercot"wrote: > Does this bug manifest in serious ways, for instance making a s6 > program > fail when it should not? If it does, I'll need to release a new > skalibs version just for this >.> I didn't notice it from a s6 program, but from anopa where e.g. aa-status would fail since it tries to read e.g. fd-notification before calling s6_svstatus_read(), so when the file doesn't exist the later call would fail. I didn't notice issues w/ s6 programs, but as soon as I fixed this I recompiled everything (execline, s6, etc) to avoid any issues, so I didn't look further.
[PATCH] Fix openreadnclose failing if errno was nonzero before
Signed-off-by: Olivier Brunel <j...@jjacky.com> --- src/libstddjb/openreadnclose.c | 1 + 1 file changed, 1 insertion(+) diff --git a/src/libstddjb/openreadnclose.c b/src/libstddjb/openreadnclose.c index aa61ece..39bb3b9 100644 --- a/src/libstddjb/openreadnclose.c +++ b/src/libstddjb/openreadnclose.c @@ -7,6 +7,7 @@ static ssize_t readnclose (int fd, char *s, size_t n) { + errno = 0 ; size_t r = allread(fd, s, n) ; if (errno) { -- 2.13.0
s6-envdir bug/behavior change
Hey, Not sure if this was intended or not, but the latest s6-envdir has some behavior change that at the very least would need to be documented: Doc says that "s6-envdir reads files in DIR"; And before, if DIR contained one or more directories they were simply ignored. Now however, it will fail (111) : s6-envdir: fatal: unable to envdir: Is a directory (I think this might be linked to the changes in skalibs, specifically openreadnclose and error handling.) Cheers,
Re: [announce] mdevd-0.0.1.0 - a mdev-compatible uevent manager
On Wed, 10 Jan 2018 00:35:33 + "Laurent Bercot"wrote: > Question that may help: > - does your /sys/devices/pci:00/:00:01.0/:01:00.1 > directory contain a "dev" file: before you manually trigger the add? > afterwards? Nope, afraid not; neither before nor after. What happens is that, in each of the case I tried (usb mouse, audio card, pc speaker), after I write "add" to the uevent file, a new directory will have appeared in the folder, e.g. mouse0 or input/input5/event5, which will be the one containing a dev file. (Of course by then there's also a corresponding syslink in /sys/dev/char). Sorry, not sure how to avoid using a big hammer... Cheers,
Re: [announce] mdevd-0.0.1.0 - a mdev-compatible uevent manager
On Mon, 15 Jan 2018 11:39:19 + "Laurent Bercot"wrote: > The change is obviously incompatible with mdevd-0.0.1.0. It appears > to work for me, but please test it - especially Olivier who was > missing events with the old coldplug. If no problems are found, I'll > cut the 0.1.0.0 release. The good news is, it appears to work fine here as well, even with my fancy hardware :p As a side note however, note that when I updated I also reverted to the default buffer size (which I noticed you upped), but then not only did my audio card not show up (amongst other things, this one is noticable because of a service waiting for it...) but some hard disks seemed not to have been processed either. A quick look in the log showed my old friend was back: mdevd: fatal: unable to receive netlink message: No buffer space available And to answer your question, about what could cause such a burst of events, this time I have an easy answer: mdevd-coldplug ! My first try when I noticed the error was simply to re-run mdevd-coldplug, and I got the buffer error again (and still no audio/etc) Easy fix, I upped the buffer back to 1MB and everything went fine this time. Rebooted since, with the buffer set to 1MB, and everything works fine right away. So yay (but my fancy hardware requires a "larger-than-normal" buffer for mdevd it seems). Anyhow, it all works fine now, with mdevd & mdevd-coldplug. Thanks!
avltree / genalloc
Hey, So I've been using avltree & gensetdyn (amongst others) lately in a little project of mine, and I have some questions: - When using avltree_iter() we need to pass a avliterfunc_t_ref, which takes 3 arguments. If I understand correctly, first is the uint32_t index, then the tree's "internal" (unsigned int) index, and my pointer. So if I wanted, from there, to remove an item from the avltree, I could either use avltree_deletenode(tree, ARG2) or avltree_delete(tree, KEY) where KEY is the one from dtok(ARG1,p) -- would all that be correct, or did I miss/misunderstood something? - So it means avltree_deletenode() needs the avltree's own/internal index, whereas gensetdyn_delete() needs "our" uint32_t index, correct? I.e. if I have that uint32_t and want to remove things from both the avltree & gensetdyn (I'm using the later to store the actual data indexed via the former, btw) then for avltree I need to get the key (e.g. via t->dtok(u,p)) - I think there's a bug in skalibs right now, avltree_deletenode() is broken and things don't even compile; I believe the attached patch is a correct fix for it. (I ended up using avltree_delete directly, guessing you usually do that as well maybe?) - And on a completely unrelated topic, is there really no way (macro/function) in skalibs to remove an item from a genalloc? I get it for stralloc, it's not really thought of as an array but a string, you rarely want to remove one char from somehere in the middle, whatever. But with a genalloc, I'd expect one to (often) be wanting to remove an item from the array, and not always/only the last one, yet I can't see how that's supposed to be done? (Again, as in via a macro or something, of course one can - as I do - handle the memmove oneself) Maybe I'm the odd one for wanting to do such a thing on a regular basis, but something like in the (other) attached patch, I feel, might be useful. I know it certainly wasn't the first time I have to write that sort of things... Thanks for the help! Cheers, >From c9d5bb194408f22db8ef859203f28c6748e78ab9 Mon Sep 17 00:00:00 2001 From: Olivier Brunel <j...@jjacky.com> Date: Thu, 12 Apr 2018 18:05:14 +0200 Subject: [PATCH] Fix avltree_deletenode Signed-off-by: Olivier Brunel <j...@jjacky.com> --- src/include/skalibs/avltree.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/include/skalibs/avltree.h b/src/include/skalibs/avltree.h index 1ce7385..8234d95 100644 --- a/src/include/skalibs/avltree.h +++ b/src/include/skalibs/avltree.h @@ -48,7 +48,7 @@ extern int avltree_newnode (avltree *, uint32_t, uint32_t *) ; #define avltree_insertnode(t, i) avltree_setroot(t, avlnode_insertnode(avltree_nodes(t), avltree_totalsize(t), avltree_root(t), i, (t)->dtok, (t)->kcmp, (t)->external)) extern int avltree_insert (avltree *, uint32_t) ; -#define avltree_deletenode(t, i) avltree_delete(t, (*(t)->dtok)(avltree_data(t, i))) +#define avltree_deletenode(t, i) avltree_delete(t, (*(t)->dtok)(avltree_data(t, i),(t)->external)) extern int avltree_delete (avltree *, void const *) ; #define avltree_iter(t, f, p) avlnode_iter(avltree_nodes(t), avltree_totalsize(t), avltree_root(t), f, p) -- 2.15.1 >From 0c8186bfedc2dfa6d1bd083dafc3ec9a7c46828d Mon Sep 17 00:00:00 2001 From: Olivier Brunel <j...@jjacky.com> Date: Thu, 12 Apr 2018 18:51:40 +0200 Subject: [PATCH] Add genalloc_delete Signed-off-by: Olivier Brunel <j...@jjacky.com> --- src/include/skalibs/genalloc.h | 2 ++ src/libstddjb/genalloc_delete_size.c | 12 2 files changed, 14 insertions(+) create mode 100644 src/libstddjb/genalloc_delete_size.c diff --git a/src/include/skalibs/genalloc.h b/src/include/skalibs/genalloc.h index da83c2f..acf2bec 100644 --- a/src/include/skalibs/genalloc.h +++ b/src/include/skalibs/genalloc.h @@ -28,6 +28,8 @@ typedef stralloc genalloc, *genalloc_ref ; #define genalloc_reverse(type, g) stralloc_reverse_blocks((g), sizeof(type)) #define genalloc_insertb(type, g, offset, s, n) stralloc_insertb((g), (offset)*sizeof(type), (char const *)(s), (n)*sizeof(type)) #define genalloc_insert(type, g1, offset, g2) stralloc_insert((g1), (offset)*sizeof(type), (g2)) +#define genalloc_delete(type,g,i) genalloc_delete_size((g),sizeof(type),i) +extern void genalloc_delete_size (genalloc *, size_t, size_t) ; #define genalloc_deepfree(type, g, f) genalloc_deepfree_size(g, (freefunc_t_ref)(f), sizeof(type)) extern void genalloc_deepfree_size (genalloc *, freefunc_t_ref, size_t) ; diff --git a/src/libstddjb/genalloc_delete_size.c b/src/libstddjb/genalloc_delete_size.c new file mode 100644 index 000..a289000 --- /dev/null +++ b/src/libstddjb/genalloc_delete_size.c @@ -0,0 +1,12 @@ +/* ISC license. */ + +#include +#include +#include + +void genalloc_delete_size (genalloc *g, size_t s, size_t i) +{ + size_t len = g->len / s; + if (len
Re: avltree / genalloc
> Then genalloc is indeed the right data structure from you, but > I don't have helpers to arbitrarily remove an object in the middle. > Note that if you don't need index stability, it's trivial, and O(1): > you can just overwrite the ith object with the (n-1)th, and decrement > the length. No need to copy len-i-1 objects, just copy one. Ah, yes. But when I said I don't need index stability, I meant current 4 can become 2, but I actually want to preserve the order of elements within the array... hence moving the whole data block. (I guess for a generic genalloc_delete_size an extra flag indicating whether to move all elements or just move the last one might be good.) Cheers,