Re: [Libevent-users] regress application
William Ahern wrote: Though, the only practical question, I suppose, is whether supporting the manifestly non-portable posix_spawn() interface is more or less work than not using it. And perhaps ultimately it is preferable to use posix_spawn() I think this: http://www.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html says that it is de-jure portable, and systems that don't have it yet can be expected to get it in future. Granted some of the systems that you mention have dubious (at best) commitment to the actual standards as published; I don't know why you accuse Solaris of dragging its feet. The platforms you mention were also tardy implementing decent threading and aio support. OpenBSD still is. (And, I have to say, I trust the Solaris engineering team much more for this sort of detail than the Linux 'team' in particular, and I'd take FreeBSD's and NetBSD's development processes over Linux's too.) But no matter - I think that: - posix_spawn should be used if available - failing that, try to do async signal safe fork/exec and hope that similar dynamic loader issues don't bite Unless you can be sure that the process is single threaded and you really need a forked process for C-o-W state snapshot etc. I'm not going to argue that fork/vfork etc are necessarily valueless now or in the past - though its certainly the case that its necessity is a lot less than it was when I started (and I felt spoiled at the time since I had my own Sparc IPX and SunOS 4.1). In the case in point for running the regression tests, I can't see why there is a real need to do any address-space cloning - while it may have been 'not broke so don't fix it' and an easy way to pass the test parameters to the child, a need to implement an exec-and-pass-params-and-resources solution for Windows means that a single model can be used for both: it just needs to be the Windows one. Why maintain two mechanisms? Its just asking for one or other to suffer bitrot. James ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
On Fri, Jun 05, 2009 at 11:04:18PM +0100, James Mansion wrote: > Do I need to produce an example? No. The standard says 'don't do it' > and that's that, > surely? > > Most code that's threaded will expect the standards to be followed and > surely all the > gross ugliness necessary to unwind everything with pthread_atfork isn't > always there > and is unlikely to be bullet-proof. > > What's the problem? You have the 'run in child' mechanism there for > Windows, so > use it in POSIX too and its job done, whether or not there are any > threads in play. > If you *know* you only have one thread then its fair game to do the old > thing, but > we're talking about a process that has regress_pthread in it. > > Surely it used to be that as an app developer I was pretty much in > control of whether > my process was threaded, but now its increasingly hard to write > applications where that's > the case unless you are prepared to rebuild a lot of support libraries, > and its hardly > worth the effort any more. What's the point? Well, one point is that posix_spawn() isn't portable. It's not available in OpenBSD, for instance; nor does it appear available in NetBSD. Another point is that other systems--like Linux (and perhaps every system _but_ Solaris)--put more emphasis on guaranteeing the safety of those corner cases rather than offloading the effort to the developer; I'd put money down that in 10 years all the uncertainty regarding threads and the remainder of the unspecified behavior wrt POSIX interfaces will have evaporated. Mostly the issue is that POSIX is meant to follow, not lead, in application design (notwithstanding the POSIX threading interface itself). And though there's ample example to follow, big players like Solaris prefer to drag their feet, and they tend to be rather incorrigible. Though, the only practical question, I suppose, is whether supporting the manifestly non-portable posix_spawn() interface is more or less work than not using it. And perhaps ultimately it is preferable to use posix_spawn(). ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
Nick Mathewson wrote: Do you want to write the patch to make the unit tests use the right interface as appropriate? No. I've posted a couple of patches into your patch system. These help but I've not attempted to get to a polished state - just a slightly better one. I think, for example, that the places where you include windows.h should probably be winsock2.h or ws2tcpip.h, so there are detail changes that could be made to tidy things up somewhat, and that's outstanding. My waf build system is running OK on Ubuntu and Vista but I need to test it on Solaris and at least one BSD, so I haven't uploaded it yet. It will provide a wscript and an event-config.h that includes a generated event-config-generated.h. I can't justify spending time on the libevent core, so don't expect much more from me except some testing, (apart from a waf script - I'm interested in exercising waf currently as a learning exercise) for the moment. James ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
William Ahern wrote: I'm as much a standards pendant as the next guy, but can you point out any implementation that forks threads (i.e. the fork'd process keeps the threads), or where otherwise forking (other than from a signal handler), actually creates issues? I even expect a positive response, and I think actually pointing them out would be productive. But the vfork() issue Actually the rationale I meant was this: https://sun.systemnews.com/articles/99/5/sw/16419 Sorry about that. Do I need to produce an example? No. The standard says 'don't do it' and that's that, surely? Most code that's threaded will expect the standards to be followed and surely all the gross ugliness necessary to unwind everything with pthread_atfork isn't always there and is unlikely to be bullet-proof. What's the problem? You have the 'run in child' mechanism there for Windows, so use it in POSIX too and its job done, whether or not there are any threads in play. If you *know* you only have one thread then its fair game to do the old thing, but we're talking about a process that has regress_pthread in it. Surely it used to be that as an app developer I was pretty much in control of whether my process was threaded, but now its increasingly hard to write applications where that's the case unless you are prepared to rebuild a lot of support libraries, and its hardly worth the effort any more. What's the point? ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
On Fri, Jun 05, 2009 at 10:19:46PM +0100, James Mansion wrote: [...] > >(While some Windows people consider Unix's fork() call a mistake, Unix > >people generally consider CreateProcess() to be an odd hybrid of > >fork(), exec*(), and a few other syscalls. _De gustibus non est > >disputandum_, I suppose.) > > > Yeah, right, hence posix_spawn. > > I know you're trying to be helpful and you can't possibly know what > UNIX experience I have. Sorry there. I really *don't* know what Unix experience you have, and I really _do_ want to be helpful. Do you want to write the patch to make the unit tests use the right interface as appropriate? > There, now we can both be condescending. ;-) Thanks for the ;-) , and sorry for my tone. Apparently I have no idea of when I need to teach Unix 101, and when I need to learn Unix 201. yrs, -- Nick ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
On Fri, Jun 05, 2009 at 10:19:46PM +0100, James Mansion wrote: > Nick Mathewson wrote: > >To me, the Windows code is the special case: Emulating it on Unix > >would require replacing CreateProcess with an explicit or implicit > >fork-and-exec() pair... but once we have forked, there is not really > >any point in calling exec(), since we are already in an isolated > >process, which was what we wanted. > > > If any API has created a thread implicitly then you're in big trouble > and it can go wrong > for you if any additional thread is in use. And compiling with the > threaded settings > makes that fair game. I'm as much a standards pendant as the next guy, but can you point out any implementation that forks threads (i.e. the fork'd process keeps the threads), or where otherwise forking (other than from a signal handler), actually creates issues? I even expect a positive response, and I think actually pointing them out would be productive. But the vfork() issue mentioned in the link is inconsequential in this case, for many reasons, not the least of which is because vfork() needn't be used. If there's memory pressure issue (if fork() fails where vfork() would exceed, there's no reason to believe the exec() wouldn't also fail in the same situation; indeed, there's more reason to think a successful fork() would ultimately fail because there wouldn't be as many shared pages; of course, neither process would be particularly big, anyhow). I don't refuse to use strlcpy() because it violates the C namespace--whereby "anything can happen". Even considering that Solaris' strlcpy() is not compatible w/ the OpenBSD's strlcpy(). > The number of system/library calls you can legally make in a threaded > application after a fork is very limited indeed. You can legally make all the same calls, the behavior just might not be well-defined. > >(While some Windows people consider Unix's fork() call a mistake, Unix > >people generally consider CreateProcess() to be an odd hybrid of > >fork(), exec*(), and a few other syscalls. _De gustibus non est > >disputandum_, I suppose.) > > > Yeah, right, hence posix_spawn. > posix_spawn() isn't portable. There's portable (SUSv3) and then there's portable (not available in some widely used BSDs). Does your implementation have stpcpy() and stpncpy()? Those are in POSIX/SUSv3-2008, too. ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
Nick Mathewson wrote: To me, the Windows code is the special case: Emulating it on Unix would require replacing CreateProcess with an explicit or implicit fork-and-exec() pair... but once we have forked, there is not really any point in calling exec(), since we are already in an isolated process, which was what we wanted. If any API has created a thread implicitly then you're in big trouble and it can go wrong for you if any additional thread is in use. And compiling with the threaded settings makes that fair game. The number of system/library calls you can legally make in a threaded application after a fork is very limited indeed. (While some Windows people consider Unix's fork() call a mistake, Unix people generally consider CreateProcess() to be an odd hybrid of fork(), exec*(), and a few other syscalls. _De gustibus non est disputandum_, I suppose.) Yeah, right, hence posix_spawn. I know you're trying to be helpful and you can't possibly know what UNIX experience I have. It used to be reasonable to concentrate on fork and vfork as appropriate but the issues with async signal safety and the rational for posix_spawn mean that its much, much safer to exec than to make any assumptions about whether you can safely fork and use arbitrary code. The most accessible rationale for posix_spawn is probably here: http://developers.sun.com/solaris/articles/subprocess/subprocess.html There, now we can both be condescending. ;-) James ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] regress application
On Mon, Jun 01, 2009 at 09:21:20PM +0100, James Mansion wrote: > The WIN32 code path execs a copy of the tester as a subprocess to > isolate it. > > Why isn't that the norm? In the presence of pthreads you pretty much > have to do > the same thing, and it can hardly be worth having a special case for the > legacy single > thread UNIX behaviour now that threading is so pervasive. It depends how you look at it. If the testing code used pthreads in its main process thread, then calling fork() would be relatively hairy... but it doesn't, so fork() is safe. To me, the Windows code is the special case: Emulating it on Unix would require replacing CreateProcess with an explicit or implicit fork-and-exec() pair... but once we have forked, there is not really any point in calling exec(), since we are already in an isolated process, which was what we wanted. (While some Windows people consider Unix's fork() call a mistake, Unix people generally consider CreateProcess() to be an odd hybrid of fork(), exec*(), and a few other syscalls. _De gustibus non est disputandum_, I suppose.) yrs, -- Nick ___ Libevent-users mailing list [email protected] http://monkeymail.org/mailman/listinfo/libevent-users
