Re: newsyslog(8) should wait(2) for children
Maxim Konovalov wrote: > > > b) exit after all children have done only. > > > > > > In the current implementation newsyslog(8) forks and execs gzip(1) or > > > bzip2(1) and exits immediately. If a log file(s) is big enough the > > > compress_log() process(es) will work after newsyslog's death and there > > > is no clear way to get know when it(they)'s done. > > > > Your (a) is statisfied with either approach. > > > > I don't understand why you think (b) is a requirement. > > Let's imagine: > > # /usr/sbin/newsyslog && ./make_something_with_compressed_log > > newsyslog exited but there are several compress_log() processes are > still running and make_something_with_compressed_log will get a > half-compressed logs. You probably meant to say "half the logs compressed", not "half-compressed logs". As far as "half-compressed logs", where a single log file can end up partially compressed -- that can't happen. If it fails to successfully compress, gzip will revert and not compress the file. Overall, allowing multiple processes, and waiting for them to finish still doesn't accomplish anything: "exited" != "exited successfully" If any process has failed to exit successfully, then you can end up with half your logs compressed and half not. If you do the NetBSD approach and serialize, you actually *do* accomplish something: you guarantee that the list of files will be processed in a specific order, and if one fails processing, no subsequent processing will be attempted. This is the most recoverable case, in terms of script based automatic recovery. > > > > OpenBSD: > > > > > > a) SIGCHLD signal handler: waitpid(2) loop, do not examine "status", > > > b) the same waitpid(2) loop before exit(2). > > > > > > I do not think we need a) at all. newsyslog forks/execs all his > > > children and enters into the reap loop like SIGCHLD signal handler > > > does. > > > > The point of this is to not reap until you have to; the default > > case will be no reaping necessary, so you are adding overhead > > unnecessarily by atttempting to reap non-existant children. > > As you see, OpenBSD has (a) *and* (b). You misunderstand. The reaping should be accomplished in the SIGCLD handler. > > There are arguments for both approaches, but if you want to > > wait for the operation to complete, the OpenBSD approach is > > better than a reap-loop. > > Again, OpenBSD has a reap loop. A reap-loop in a SIGCLD handler is a better idea. I still don't understand why you want to delay exiting of the parent until the child processes are finished. I also don't understand why you want a zombie to pile up for each process, until you hit your "reap loop", rather than handling them as you get SIGCLDs. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: newsyslog(8) should wait(2) for children
On 01:04-0700, May 2, 2002, Terry Lambert wrote: > Maxim Konovalov wrote: > > > [ ... patch to wait for children, but do nothing with the result ... ] > > > > > > Why not just set the signal handler for the child process > > > termination to "ignore", so that the child processes do > > > not become zombied in the first place, so it's not ever > > > necessary to do a useless loop whose only purpose is to > > > reap zombies without examining their exit status? > > > > There are two purposes: > > > > a) reap zombies, > > b) exit after all children have done only. > > > > In the current implementation newsyslog(8) forks and execs gzip(1) or > > bzip2(1) and exits immediately. If a log file(s) is big enough the > > compress_log() process(es) will work after newsyslog's death and there > > is no clear way to get know when it(they)'s done. > > Your (a) is statisfied with either approach. > > I don't understand why you think (b) is a requirement. Let's imagine: # /usr/sbin/newsyslog && ./make_something_with_compressed_log newsyslog exited but there are several compress_log() processes are still running and make_something_with_compressed_log will get a half-compressed logs. > > OpenBSD: > > > > a) SIGCHLD signal handler: waitpid(2) loop, do not examine "status", > > b) the same waitpid(2) loop before exit(2). > > > > I do not think we need a) at all. newsyslog forks/execs all his > > children and enters into the reap loop like SIGCHLD signal handler > > does. > > The point of this is to not reap until you have to; the default > case will be no reaping necessary, so you are adding overhead > unnecessarily by atttempting to reap non-existant children. As you see, OpenBSD has (a) *and* (b). > > NetBSD: > > > > a) waitpid(2) for a child right after fork/exec, > > b) examine "status" and print an exit code. > > > > As you see, NetBSD newsyslog serializes fork/exec and there is only > > one gzip process at the same moment. We can take this way but IMHO it > > will be a POLA violation. [ NetBSD approach arguments ] > There are arguments for both approaches, but if you want to > wait for the operation to complete, the OpenBSD approach is > better than a reap-loop. Again, OpenBSD has a reap loop. [...] -- Maxim Konovalov, MAcomnet, Internet Dept., system engineer phone: +7 (095) 796-9079, mailto:[EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: newsyslog(8) should wait(2) for children
Maxim Konovalov wrote: > > [ ... patch to wait for children, but do nothing with the result ... ] > > > > Why not just set the signal handler for the child process > > termination to "ignore", so that the child processes do > > not become zombied in the first place, so it's not ever > > necessary to do a useless loop whose only purpose is to > > reap zombies without examining their exit status? > > There are two purposes: > > a) reap zombies, > b) exit after all children have done only. > > In the current implementation newsyslog(8) forks and execs gzip(1) or > bzip2(1) and exits immediately. If a log file(s) is big enough the > compress_log() process(es) will work after newsyslog's death and there > is no clear way to get know when it(they)'s done. Your (a) is statisfied with either approach. I don't understand why you think (b) is a requirement. > OpenBSD: > > a) SIGCHLD signal handler: waitpid(2) loop, do not examine "status", > b) the same waitpid(2) loop before exit(2). > > I do not think we need a) at all. newsyslog forks/execs all his > children and enters into the reap loop like SIGCHLD signal handler > does. The point of this is to not reap until you have to; the default case will be no reaping necessary, so you are adding overhead unnecessarily by atttempting to reap non-existant children. > NetBSD: > > a) waitpid(2) for a child right after fork/exec, > b) examine "status" and print an exit code. > > As you see, NetBSD newsyslog serializes fork/exec and there is only > one gzip process at the same moment. We can take this way but IMHO it > will be a POLA violation. I think the reason NetBSD does this is that there is a sizeof(uncompressed) + sizeof(compressed) window during the operation itself. If you run out of disk space while this is going on, it's a bad thing. So serializing gets you: i. No contention for the remaining disk space between more than one process ii. The ability to not attempt things you know will fail (average compressability will remain constant over a given log file contents, statistically) iii.A reduction in the maximum instantaneous CPU load committed to the newsyslog operation (by serializing, the total load is spread over time) There are arguments for both approaches, but if you want to wait for the operation to complete, the OpenBSD approach is better than a reap-loop. If you are going to interlock, then interlock. If you aren't, then don't bother explicitly reaping. Realize also that the zombied processes shouldn't hang around forever: newsyslog isn't a daemon. And the zombied processes only take up 64 bytes of memory until they are reaped on the exit of newsyslog. Not that I'm arguing that the processes shouldn't be reaped, or at least ignored. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: newsyslog(8) should wait(2) for children
Terry, On 23:11-0700, May 1, 2002, Terry Lambert wrote: > Maxim Konovalov wrote: > > [ ... patch to wait for children, but do nothing with the result ... ] > > Yes. > > Why not just set the signal handler for the child process > termination to "ignore", so that the child processes do > not become zombied in the first place, so it's not ever > necessary to do a useless loop whose only purpose is to > reap zombies without examining their exit status? There are two purposes: a) reap zombies, b) exit after all children have done only. In the current implementation newsyslog(8) forks and execs gzip(1) or bzip2(1) and exits immediately. If a log file(s) is big enough the compress_log() process(es) will work after newsyslog's death and there is no clear way to get know when it(they)'s done. OpenBSD: a) SIGCHLD signal handler: waitpid(2) loop, do not examine "status", b) the same waitpid(2) loop before exit(2). I do not think we need a) at all. newsyslog forks/execs all his children and enters into the reap loop like SIGCHLD signal handler does. NetBSD: a) waitpid(2) for a child right after fork/exec, b) examine "status" and print an exit code. As you see, NetBSD newsyslog serializes fork/exec and there is only one gzip process at the same moment. We can take this way but IMHO it will be a POLA violation. -- Maxim Konovalov, MAcomnet, Internet Dept., system engineer phone: +7 (095) 796-9079, mailto:[EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: newsyslog(8) should wait(2) for children
Maxim Konovalov wrote: [ ... patch to wait for children, but do nothing with the result ... ] Yes. Why not just set the signal handler for the child process termination to "ignore", so that the child processes do not become zombied in the first place, so it's not ever necessary to do a useless loop whose only purpose is to reap zombies without examining their exit status? -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: newsyslog(8) should wait(2) for children
Maxim Konovalov <[EMAIL PROTECTED]> writes: > Does anyone object to the next patch: while (wait(NULL) > 0 || errno == EINTR) /* nothing */ ; DES -- Dag-Erling Smorgrav - [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
newsyslog(8) should wait(2) for children
Does anyone object to the next patch:
Index: newsyslog.c
===
RCS file: /home/ncvs/src/usr.sbin/newsyslog/newsyslog.c,v
retrieving revision 1.41
diff -u -r1.41 newsyslog.c
--- newsyslog.c 10 Apr 2002 10:38:44 - 1.41
+++ newsyslog.c 1 May 2002 19:15:40 -
@@ -38,6 +38,7 @@
#include
#include
+#include
#include
#include
#include
@@ -135,6 +136,12 @@
p = p->next;
free((char *) q);
q = p;
+ }
+ for (;;) {
+ if (wait(NULL) < 0) {
+ if (errno != EINTR)
+ break;
+ }
}
return (0);
}
%%%
--
Maxim Konovalov, MAcomnet, Internet Dept., system engineer
phone: +7 (095) 796-9079, mailto:[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
