Re: kqueue and libev
Kip Macy wrote: he's just plain misinforme Until we know what he is referring to we can't actually say that. -Kip OK he said I could post from our private email so here goes. There were bits in and around relating to the Solaris /dev/poll support (and the mechanism's limitations) which I've elided. I think the most telling thing is probably that drivers need to provide support and that a single mechanism in the driver doesn't support select and poll at the same time - which I guess lines up with the reported failure with USB serial. Does kqueue work with USB for example? How about an AIO request to read from a USB endpoint? It may well just be a case of 'fessing up to system limitations. James Compile and install rxvt-unicode on freebsd and run it with: urxvt # works, uses select (or maybe poll) LIBEV_FLAGS=8 urxvt # acts weird, uses kqueue (note: only works when urxvt isn't set[gu]id) The typical symptoms are either delayed notificatrions, no notifications at all or _continuous_ notifications and read failing with EAGAIN. Here is a ktrace showing the latter: http://ue.tst.eu/45eb8a3c3e812933cbe3172af2ee4a6c.txt kqueue works well with sockets (or with about anything on netbsd), but fails on more exotic types such as ptys. (I test on Freebsd 6.2 RELEASE, but got reports about problems with earlier and later versions, too, as well as on openbsd (which I didn't verify) and on darwin (which is completely broken)). You normally don't get useful writeable/readable state for files, No, I only want the same readyness notifications as with select or poll, as is documented in the manpage. (even on platforms where kqueue works this requires some heuristics and workarounds with kqueue due to design limitations (for example problems with close() or fork() that force constant rearming), but thats common in interfaces like kqueue, and by now well understood and handled by libev). Actually, until recently it was broken on pipes. We've never received any PRs to that effect so there is no way of knowing. You'll have better luck asking the author himself. Well, one should better document the types with which it works (which on freebsd apparently includes sockets and nothing else). I also think one should rethink the internal design if every driver needs its own kqueue support, as that will always force kqueue into a second-class citizen not suitable as replacement for select, as it's unreasonable to expect kqueue to just work when its so little used and requires such a high maintainance (linux' epoll for example works fine with everything because it doesn't require drivers to support epoll specifically, so it is unlikely that epoll fails when select would work for example, which is the case on freebsd and darwin). The fact that it works fine on everything I threw at it on netbsd is probably not the result of better design, but more better maintainance, so I wouldn't be surprised if some future version of netbsd failed in similar ways (OTOH, in the past, netbsd consistently was the less buggy platform regardless of topic, wether it was threads, ptys or kqueue, so I might get quite disappointed if that happened :) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
On Dec 17, 2007 1:25 PM, James Mansion [EMAIL PROTECTED] wrote: Kip Macy wrote: he's just plain misinforme Until we know what he is referring to we can't actually say that. -Kip OK he said I could post from our private email so here goes. There were bits in and around relating to the Solaris /dev/poll support (and the mechanism's limitations) which I've elided. I think the most telling thing is probably that drivers need to provide support and that a single mechanism in the driver doesn't support select and poll at the same time - which I guess lines up with the reported failure with USB serial. Does kqueue work with USB for example? How about an AIO request to read from a USB endpoint? It may well just be a case of 'fessing up to system limitations. James Compile and install rxvt-unicode on freebsd and run it with: urxvt # works, uses select (or maybe poll) LIBEV_FLAGS=8 urxvt # acts weird, uses kqueue (note: only works when urxvt isn't set[gu]id) The typical symptoms are either delayed notificatrions, no notifications at all or _continuous_ notifications and read failing with EAGAIN. Here is a ktrace showing the latter: http://ue.tst.eu/45eb8a3c3e812933cbe3172af2ee4a6c.txt kqueue works well with sockets (or with about anything on netbsd), but fails on more exotic types such as ptys. (I test on Freebsd 6.2 RELEASE, but got reports about problems with earlier and later versions, too, as well as on openbsd (which I didn't verify) and on darwin (which is completely broken)). You normally don't get useful writeable/readable state for files, No, I only want the same readyness notifications as with select or poll, as is documented in the manpage. (even on platforms where kqueue works this requires some heuristics and workarounds with kqueue due to design limitations (for example problems with close() or fork() that force constant rearming), but thats common in interfaces like kqueue, and by now well understood and handled by libev). Actually, until recently it was broken on pipes. We've never received any PRs to that effect so there is no way of knowing. You'll have better luck asking the author himself. To be more precise, this only manifested itself in erlang. Well, one should better document the types with which it works (which on freebsd apparently includes sockets and nothing else). I also think one should rethink the internal design if every driver needs its own kqueue support, as that will always force kqueue into a second-class citizen not suitable as replacement for select, as it's unreasonable to expect kqueue to just work when its so little used and requires such a high maintainance (linux' epoll for example works fine with everything because it doesn't require drivers to support epoll specifically, so it is unlikely that epoll fails when select would work for example, which is the case on freebsd and darwin). The fact that it works fine on everything I threw at it on netbsd is probably not the result of better design, but more better maintainance, so I wouldn't be surprised if some future version of netbsd failed in similar ways (OTOH, in the past, netbsd consistently was the less buggy platform regardless of topic, wether it was threads, ptys or kqueue, so I might get quite disappointed if that happened :) Interesting, that has been completely counter to my experience. However, I rely on a completely different set of subsystems. Do you have a set of regression tests for libev? It sounds like they would worth having to regression test kqueue. Thanks for your feedback. -Kip ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
Kip Macy wrote: Do you have a set of regression tests for libev? It sounds like they would worth having to regression test kqueue. I would have thought that libevent and libev should both the checked against kqueue. Also APR and everything else that has support. I'm not the author of libev though, so I'm the wrong guy to ask. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
On Dec 15, 2007, at 08:47 , Kip Macy wrote: On 12/15/07, James Mansion [EMAIL PROTECTED] wrote: Any idea what the author of libev is on about here (from http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod): unsigned int ev_recommended_backends () Return the set of all backends compiled into this binary of libev and also recommended for this platform. This set is often smaller than the one returned by |ev_supported_backends|, as for example kqueue is broken on most BSDs and will not be autodetected unless you explicitly request it (assuming you know what you are doing). and |EVBACKEND_KQUEUE| (value 8, most BSD clones) Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). For this reason its not being autodetected unless you explicitly specify it explicitly in the flags (i.e. using |EVBACKEND_KQUEUE|). It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? James Actually, until recently it was broken on pipes. We've never received any PRs to that effect so there is no way of knowing. You'll have better luck asking the author himself. -Kip How recently? I have been using kqueue with pipes in several programs for the last year or so. Bert JW Regeer
RE: kqueue and libev
Julian Elischer wrote: Julian Elischer wrote: James Mansion wrote: [ On the libev being unhappy with kqueue ] ... It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? he's just plain misinformed kqueue works well with aio to files and raw devices for example. (Only using AIO really makes sense in these cases anyhow, so I've never really tried using kqueue with non-aio calls.) It also depends what version of FreeBSD. For example, in FreeBSD 4, kqueue was non-functional with USB serial devices. I ran into that exact problem with kqueue when porting my event library. Problems like that can make life difficult for a library author; having a special case for one kind of handle is a pain, to the point of leading to comments like this. Of course, in this case the best thing to do is to ask the author, and to see if the situation has changed. Regards, Jan Mikkelsen ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
kqueue and libev
Any idea what the author of libev is on about here (from http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod): unsigned int ev_recommended_backends () Return the set of all backends compiled into this binary of libev and also recommended for this platform. This set is often smaller than the one returned by |ev_supported_backends|, as for example kqueue is broken on most BSDs and will not be autodetected unless you explicitly request it (assuming you know what you are doing). and |EVBACKEND_KQUEUE| (value 8, most BSD clones) Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). For this reason its not being autodetected unless you explicitly specify it explicitly in the flags (i.e. using |EVBACKEND_KQUEUE|). It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? James ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
On 15/12/2007, James Mansion [EMAIL PROTECTED] wrote: |EVBACKEND_KQUEUE| (value 8, most BSD clones) Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). For this reason its not being autodetected unless you explicitly specify it explicitly in the flags (i.e. using |EVBACKEND_KQUEUE|). Does he define broken anywhere? It seems to work great for Squid and Varnish.. -- Adrian Chadd - [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
On Sat, Dec 15, 2007 at 09:51:20AM +, James Mansion wrote: Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). You normally don't get useful writeable/readable state for files, if that is what the author means. This is the same situation as you have with poll(2) and select(2) though. Otherwise I can't think of what he could mean. Joerg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
Actually, until recently it was broken on pipes. We've never received any PRs to that effect so there is no way of knowing. You'll have better luck asking the author himself. -Kip On 12/15/07, James Mansion [EMAIL PROTECTED] wrote: Any idea what the author of libev is on about here (from http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod): unsigned int ev_recommended_backends () Return the set of all backends compiled into this binary of libev and also recommended for this platform. This set is often smaller than the one returned by |ev_supported_backends|, as for example kqueue is broken on most BSDs and will not be autodetected unless you explicitly request it (assuming you know what you are doing). and |EVBACKEND_KQUEUE| (value 8, most BSD clones) Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). For this reason its not being autodetected unless you explicitly specify it explicitly in the flags (i.e. using |EVBACKEND_KQUEUE|). It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? James ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
James Mansion wrote: Any idea what the author of libev is on about here (from http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod): unsigned int ev_recommended_backends () Return the set of all backends compiled into this binary of libev and also recommended for this platform. This set is often smaller than the one returned by |ev_supported_backends|, as for example kqueue is broken on most BSDs and will not be autodetected unless you explicitly request it (assuming you know what you are doing). and |EVBACKEND_KQUEUE| (value 8, most BSD clones) Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). For this reason its not being autodetected unless you explicitly specify it explicitly in the flags (i.e. using |EVBACKEND_KQUEUE|). It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? he's just plain misinformed James ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
Julian Elischer wrote: James Mansion wrote: Any idea what the author of libev is on about here (from http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod): unsigned int ev_recommended_backends () Return the set of all backends compiled into this binary of libev and also recommended for this platform. This set is often smaller than the one returned by |ev_supported_backends|, as for example kqueue is broken on most BSDs and will not be autodetected unless you explicitly request it (assuming you know what you are doing). and |EVBACKEND_KQUEUE| (value 8, most BSD clones) Kqueue deserves special mention, as at the time of this writing, it was broken on all BSDs except NetBSD (usually it doesn't work with anything but sockets and pipes, except on Darwin, where of course its completely useless). For this reason its not being autodetected unless you explicitly specify it explicitly in the flags (i.e. using |EVBACKEND_KQUEUE|). It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? he's just plain misinformed kqueue works well with aio to files and raw devices for example. (Only using AIO really makes sense in these cases anyhow, so I've never really tried using kqueue with non-aio calls.) James ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED] ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: kqueue and libev
It looks like a decent library, but these comments seem unfortunate. Does anyone know what the author is concerned about? he's just plain misinformed Until we know what he is referring to we can't actually say that. -Kip ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to [EMAIL PROTECTED]
