[Libevent-users] libev-1.85 release
Libev 1.85 has just been released (together with the corresponding perl module). Distribution: http://dist.schmorp.de/libev/ Documenttaion: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod The biggest change is that libev now comes in two tarballs: one libevent+libev tarball containing a patched libevent, and one with the standalone libev version. Other major changes (since 1.7) include: - the epoll backend uses some heuristics that avoid superfluous EPOLL_CTL_DEL calls, which can majorly speed up common usage patterns (the same technique could be applied to the kqueue backend). stopping and starting a watcher is now very cheap and will not result in additional syscalls. - a bug in the heap management functions has been fixed that could rarely cause timers to be delayed. - the long term numeric stability of periodic watchers has been improved, as well as timing accuracy in general. - ev_signal_start wasn't in fact async-signal safe and will now block all signals during installation of the watcher. - the API version has been set to 1:0 and will be maintained from now on. - the C++ callback API has changed considerably, and will now be zero-overhead w.r.t. to watcher size, and can be zero-overhead when the compiler is able to inline the method callback into the C thunking code (use gcc :). I do expect some packaging problems, as this is the first tarball release ever. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] libev-1.7 release
I have made a new release of libev + libevent: Tarball is at: http://dist.schmorp.de/libev/ Documentation is linked from: http://software.schmorp.de/pkg/libev.html Changes: - the tarball now reflects both the libevent and libev version. - the patched libevent will install itself under the same names as libevent itself (but is not ABI compatible to it). - minimal mode has been further improved: rxvt-unicode 8.7 (with libev) is 10k smaller than rxvt-unicode 8.4 (with the older iom). - the priority semantics have been finalised (idle watcher semantics have changed considerably, but overall the system stayed the same as implemented). - the C++ callbacks have changed considerably: a callback now uses no additional memory in the watcher struct over a C callback, and can be as fast as a direct C callback. One can also use static emthods and plain functions as callbacks now, making the C++ watchers some fluffier inteface to libev. - the documentation has been extended with the priority changes and the C++ API change. - the libev core has seen considerable fine-tuning. while no new bugs have been found, a few might have been introduced, as usual... -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] libev update (new minimal mode, better libevent emu, ev_stat and ev_fork watchers)
I have updated the patched libevent tarball at http://dist.schmorp.de/libev/ Documentation is linked from http://software.schmorp.de/pkg/libev.html No new bugs have been identified in the libev core proper, which is good, meaning its reaching the point of stability :) Changes in this release: - a workaround for a linux 2.4 bug w.r.t. child watchers has been implemented. - the libevent emulation has been tweaked. common programs such as crawl now seem to work fine with it (I find it rather hard to replicate libevents exact behaviour on when it is safe to call event_set on an event already added). - there is an EV_MINIMAL mode that can reduce codesize and memory consumption by up to 30% (with gcc), at the expense of some speed. - the documentation once more has been improved and expanded. - a lot of tweaks to improve performance or reduce codesize have been implemented. - all watcher structs are now typedefed, too (struct ev_stat = ev_stat). - the c++ api has become much leaner (admittedly by using a dirty trick that only works with standards-compliant compilers) and therefore faster. There are two new watcher types: - ev_fork watchers. these are very small and simple and can be used to hook into the fork support mechanism of libevent (they are basically called before the next event loop iteration after a fork). - ev_stat watchers. these basically stat a path periodically and call a watcher on any changes (including if the path comes into existance or goes out of existance) - a linux inotify backend is available that is used to supply hints to ev_stat watchers (and avoid periodically stat'ing in common cases). (while a kqueue backend should not be difficult to write, the ev_stat semantics cannot be replicated with kqueue, unfortunately). All the new watcher types and features can be configured out when embedding libev, so there is no overhead for features you don't need (rxvt-unicode, in fact, uses this a lot to stay lean). Enjoy, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] libev-1.4.0-beta.tar.gz update
I have updated http://dist.schmorp.de/libev/libev-1.4.0-beta.tar.gz Documentation: http://cvs.schmorp.de/libev/ev.html Changes in this release include: - stopping idle, check and prepare watchers could lead to data corruption, other watchers are unaffected. - due to the rxvt-unicode releases, a lot of quirks in various backends have been identified (and documented so people don't need to find out the hard way). - the documentation has been improved quite a bit: there is now a host of examples, there have been many clarifications, the c++ interface has been documented, as well as the embedding features. - there is a new watcher type named ev_embed. its purpose is to embed another libev loop into a loop (this currently works only for ev_io watchers). There are two main purposes for this: - on platforms where kqueue is broken as a generic mechanism, but works for e.g. sockets (and you know that), you can embed a kqueue loop into your poll-based mainloop (for example, on all current freebsds, only select and poll support generic fds). while this costs (some constant) performance, it still scales just as well as kqueue alone and provides correctness for all types of fds. - this can be used for strict prioritising of events, similarly to the libevent priority system, by invoking sweep in an idle watcher and putting all your low-priority into the embedded loop. this also solves a problem with event starving in libevent: when some high-priority event preempts a lower-priority io watcher 100 times, then the kernel will be queried 100 times for the low-priority fd and the resulting readyness notification must be handled, too. Enjoy! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] evhttp and pre-fork model
On Sat, Nov 24, 2007 at 10:57:26PM +0100, Nicolas Favre-Félix [EMAIL PROTECTED] wrote: Actually, I get some unpredictable behavior: some of the processes receive the callback in which accept() is done, and some don't. It is quite possible that some of your processes receive the wakeup, one gets to accept, and the other processes simply never see the event, because they were busy doing other stuff etc. This depends very much on the backend in use and your server design, but isn't really a problem as long as _one_ process gets the connection. kernels (e.g. freebsd and linux) this will only wake up one process, but of course doesn't allow event handling. When you refer to bad scalability, are you saying that having lots of connections will result in multiple processes trying to accept simultaneously, which would impact the performance, all but one failing instead of processing client? Yes, thats exactly what I meant. Depending on your application, this might be acceptable however, as its also a very simple design. If so, would it be better to have 2 threads per server process, one blocking on accept() and the other running an event loop? If your accept rate is very high (like, say, hundreds to thousands of requests/s), then this sounds certainly better, but you still need an effective and fats way to then communicate this fd to your worker threads, preferably from a thread pool for example. However, since you have a pre-fork model, threads are of no relevance. Apache actually experimented a lot on how to best accept, but that always refers to blocking processes - you might use event handling while serving a request, but not when waiting for an accept. I see. Accepting myself is not a problem here. strongly preferable, in fact .) while (fd == accept) evutil_make_socket_nonblocking (fd); evhttp_get_request(http, fd, sockaddr, addrlen); Thanks! This works fine. Actually, I had tried this function after Oh, thats good, because I only read over the code and intended only to give you some starting hint, but its even better when it actually works out of the box. Well, they are exported, just not declared, if you will :) Should this be declared, then? Well, I have no say in policy issues (or any issues, really) regarding libevent (I am only the libev guy). I do think the various libevent APIs are sometimes severely limited, hiding important functionality. For example, there is no way to make MX lookups using the documented dns code (because there is no generality at all, just three or so predefined request functions), although the underlying code clearly can deal with other resource records. I find that the libevent API usually exposes what the libevent author(s) currently needed but nothing else. (This is actually known as the bsd API style, it isn't necessarily bad!). Or will the upcoming libev enable fork()-safe event handling once evhttp has been added to it? libev, unlike libevent, supports forking. it is also not upcoming anymore, as its fully released. However, this just means that you can reuse event loops from your parent process in the child. It does not magically create some IPC channel to distribute events betweeen processes, and from what I could read in your mail so far, it looks as if you do not even need this capability, as your style of forking and only _then_ initialising libevent is *perfectly* safe and adequate for and with libevent and your program :) like to avoid is my code breaking if evhttp_get_request changes or is renamed/removed. Then you need the libevent maintainers make a committment to both expose this API and keep it in the future. Who knows, they might just do that, and you are exactly in the right list to make them hear you :) Thank you very much for your help. Greast that it was actually helpful, after a rough start of misreading your actual problem :) -- The choice of a -==- _GNU_ ==-- _ generation Marc Lehmann ---==---(_)__ __ __ [EMAIL PROTECTED] --==---/ / _ \/ // /\ \/ / http://schmorp.de/ -=/_/_//_/\_,_/ /_/\_\ XX11-RIPE ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] evhttp and pre-fork model
On Fri, Nov 23, 2007 at 04:03:52PM +0100, Nicolas Favre-Félix [EMAIL PROTECTED] wrote: libevent doesn't support forking (unless you force it to use select or poll) and continue using it in the child. That's actually what I'm doing at the moment, calling listen(), then fork(). Each child adds a callback for EV_READ on the socket and runs its event loop. I hope you also initialise libevent only in the child, too (or indeed use fork or select), as kqueue etc. will not work in the child processes. It tries to accept() on it when the callback function is executed, and handles the client. This has been tested quite heavily without problem... Could there be one? No, not if you only touch libevent in one process that doesn't fork (e.g. in each child). The only problem I see with your approach is that it scales badly, as all processes got woken up that way and then fight for the new connection. But thats not really a problem unless you want high performance, and is a correct approach, which is most important. If you can block when idling (as opposed to having to handle events) you could also just call accept() in each child - on the more prestigious kernels (e.g. freebsd and linux) this will only wake up one process, but of course doesn't allow event handling. Thank you for your precisions. I don't have an event loop in the parent, so this is not a problem. Ah, your mentioning of pre-fork and having a problem made me think that were experiencinging problems with that, my bad :) I'd like to integrate HTTP support - using evhttp. this was my main question... Is it possible to give evhttp a bound socket to monitor? Not with the public API. It is also not sure wether this is what you want, you want to handle a request on a socket you already accepted (unless you want evhttp to do its accepting). If you are logging, libevent will also flood you with false warning messages because it doesn't expect accept not returning a new connection). evhttp internally calls evhttp_get_request, but obviously needs a webserver to be created already. Looking at the code, this could do what you want: struct evhttp *http = evhttp_new_object (); while (fd == accept) evutil_make_socket_nonblocking (fd); evhttp_get_request(http, fd, sockaddr, addrlen); This is of course not prototyped in evhttp.h and undocumented, but it seems quite fashionable to work around the limitations of the public libevent API by calling those private (but non-static) functions. YMMV. I checked http.c but couldn't find any exported function doing this. If this isn't implemented, would it be OK to submit a patch to add this feature? Well, they are exported, just not declared, if you will :) -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] new libev release and mailinglist available
I have updated the libev release tarball: http://dist.schmorp.de/libev/libev-1.4.0-beta.tar.gz changes since the last release: - a solaris port-based backend has been added. - more configurability for win32 platforms. - some of minor portability fixes have been applied. - minor doc updates. - embedding (including autoconf embedding) options have been cleaned up and tested in a number of embedding situations. As the interest from the libevent maintainers seems to have dried out, I decided to maintain libev as a separate package from libevent (but still embedding it into libevent proper for releases). A mailinglist is now available for libev discussions, see http://lists.schmorp.de/mailman/listinfo/libev). Enjoy! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent vs libev
On Tue, Nov 13, 2007 at 01:53:33PM -0800, Larry Lewis [EMAIL PROTECTED] wrote: 1. What is this libev library? Its a drop-in replacement for libevent. While it provides a libevent emulation API, it also has a new api that fixes most (maybe all) of the actual and perceived api problems with libevent. It is also faster, at least with epoll and timers being involved, uses less memory etc. etc. Benchmark and documentation is currently linked from here: http://libev.schmorp.de/ 2. How does it differ from libevent? It offers a completely new, incompatible API with different design goals but also emulates libevent, at leats its event core parts. It is also easily embeddable for those people who need this functionality and wildly configurable. It does not feature evbuffer, evtags, evhttp or evdns, but the libevent source files implementing those can be used without any changes with libev. In fact, the only release file so far is actually a libevent tarball patched by replacing the libevent event handling with the libev files and keeping all else. The high degree of compatibility also makes it trivial to do the same for any upcoming 1.4 release. Which is recommended for general usage on Linux? Decidedly both of them, depending on your goal and needs: - libevent has seen vastly more testing than libev (especially outside linux, which should always be an option). - libev has a much more efficient epoll implementation, and much faster timers, and less arbitrary limitations (as in the type of combinations of fd events are allowed etc.) - libevent isn't embeddable, libev is - libevent is 100% compatible to libevent, libev isn't, only emulating the most visible parts - if libevent works fine for you, its a safe, available fallback - if the libev api appeals to you more than the libevent api, its the way to go. etc. etc. If libevent works for you, you could try out libev, too, using its emulation API. In fact, when you keep developing for libevent you lose nothing, as you can use either library as you want. 3. If it is a separate library, why is it being discussed on the libevent mailing list? Shouldn't there be a separate list for this (libev-users)? The plan by both the libev developer (me) and the libevent maintainers at the moment is to integrate libev (ideas, parts, or full) into libevent, which makes it on-topic on libevent. Also, the new API was developed together with the existing libevent users, as libev does not (at the moment) compete as a separate product or solution, but is aimed at providing a better libevent for the future. If this plan dies, then it would be more than appropriate to move discussions to another mailinglist, as libevent would then not progress towards the better API developed here but instead keep its interface, and discussions about changes would not belong on this mailinglist anymore (as in the past). On the other hand, most of the design is finished by now, and so the amount of discussion has gone down quite a bit. Let me thank the libevent users again here for helping me design the new API, all your input was very important. Niels et all, great work on the 1.4.0 release -- especially the separate libraries and doxygen documentation. Hmm.. is 1.4.0 even released yet? I only see the beta atm. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] libev-1.4.0-beta release
As the subject says, a port of libev to libevent-1.4.0-beta is now available: http://dist.schmorp.de/libev/libev-1.4.0-beta.tar.gz Apart from all the goodies from libevent-1.4.0-beta, this also contains all the patches sent to me for integration in the libev parts, which include: - a small bugfix to kqueue (also present in the libev-1.3e release from yesterday) - the number of warnings have been reduced considerably, as a way was found to keep both msvc compatibility and not create warnings with gcc. - portability problems to solaris and some other OSs lacking standard C header files have been fixed. - documentation is now part of the release and will install itself as the ev(3) manpage, in addition to event(3). Thanks a lot for your contributions! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libev-1.3e demo release - kqueue ident/fd issue?
On Sun, Nov 11, 2007 at 04:42:41PM +0100, Chris Brody [EMAIL PROTECTED] wrote: I suspect there is a very strange error event if there is a small delay between write()/shutdown() and event_add()/event_dispatch(). Ok, if the patch indeed is correct, then it isn't (because it doesn't range-check the fd, which is the real problem). But the real issue is why kqueue returns errors to us for requests we never made, this feels like quite the bug in kqueue (I hope you don't use mac osx, I just found out that both poll and kqueue are broken and the libevent test doesn'T detetc this brokenness on macosx). I tried adding a check of the fd, and this seems to work well. I must admit that I am no kqueue expert, but I hope this information is helpful. Possible patch below. Could you make it crash again and send me the contents of res and kqueue[i]? Its quite possible that I don't use qkueue right, or something else goes bad. In which case ignoring the error would never give us the chance to find out. Thanks a lot! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] C++ interface(s) [was: libev-1.3e...]
On Sun, Nov 11, 2007 at 08:04:10PM +0100, Chris Brody [EMAIL PROTECTED] wrote: My first comment is that building with EV_MULTIPLICITY=1 (the default value) caused a few problems since there is both a struct ev_loop and Thanks a lot (I really never tested with EV_MULTIPLICITY, so I am pleasently surprised you even got some results... :) an ev_loop() function call. In addition, the EV_P (and EV_P_) macros require the user code to use using namespace ev... user code should not need to use those macros... when does the need come up? (in fact, there should not be any macros defined, but I guess thats not reasonable, so maybe in the long run ev will have to end up in the std namespace) does exhibit a few quirks. For any C++ class that is called by interpreted code, CINT has to parse the header file and make a dynamic library that is called in reaction to interpreted C/C++ code. It is only 85% compliant and has real trouble with significantly templated code. Ok, I wonder wether ev++ counts as heavily templated or not :) But if I make a C++ subclass (or several C++ subclasses) that are less complex, these should be more straightforward to be used dynamically. I will try this and provide some more feedback. Thanks! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] libev-1.3e beta release
Hi! Over the last few days, I took all of your feedback that arrived in time or was actable upon and updated libev. Its available at http://dist.schmorp.de/libev/libev-1.3e.tar.gz It has now been tested on freebsd, linux, windows and os x, and is used by a number of gvpe and urxvt installations by now. Here are the highlights/changes: - an enourmous number of bugs have been fixed. I really wonder how it ever worked before (yeah, obviously, because the basic stuff worked :). - the select backend has been completely rewritten and now supports FD_SET and direct-access modes (as well as win32). - the select backend can optionally be told to use the raw winsocket select, and libev will cache osf handles in this case (fd-based selects are still supported). - poll has been bugfixed. - the code has been compiletested as c++. - kqueue has been considerably improved. - the libevent emulation now declares and manages the ev_flags member (in a half-assed way, more work would be neeccessary to really support this member perfectly). - whoever had issues with the word shit in the source, I found a simpler, less annoying way to work around some bugs in msvc, and consequently could avoid using shit (not sure wether the new version is any better, though :) - both poll and kqueue have been found to be defective on os x, and the libevent test for broken kqueue does not detect this condition, so for the time being those backends are disabled. - there is the beginning of a c++ adaptor which is designed to replace the older io_manager event loop to some extent. - a number of embedding features such as being able to override more behaviour, better autoconfiguration or beginning autoconf-support (example usage is in both urxvt and gvpe). Thanks again for all your feedback, but especially to Emanuele Giaquinta, who constantly helped me check the design for flaws and tested, and tested, and tested... If this continues for a few months, I think libev will be quite capable of replacing libevent completely. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [OT] craigslist: libevent programmer wanted
On Fri, Nov 09, 2007 at 12:39:37AM +0100, Hannah Schroeter [EMAIL PROTECTED]
wrote:
I see less problems with the writing away of the data sucked from the
web servers, as most Unix like systems write stuff asynchronously, so
the open(..., O_CREAT...), write() and close() calls won't be too slow.
Most unix systems cache data for quite long, butwhen they write, usually
user mode apps also halt. For throughput this is of little concern, but
in a game server I wrote, even an fsync could freeze the server for 15-20
seconds(!) when another sync was in progress at the same time, or when
some othe rprogram geenrated lots of I/O (for example a backup/restore).
(I hear the linux is abysmal w.r.t. writing out data (and I agree :),
but I found similar problems with freebsd, too, so I guess it is quite
common).
And if they should be slower than the network interfaces, combine
things with I/O worker {threads,processes}. BTDT (main program using
event multiplexing on network + socketpairs to I/O helper processes).
If anybody uses perl, there is the IO::AIO module which provides this
quite efficiently (using only a single pipe to report results, and it is
only written/read once per poll, not per result).
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED]
-=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [OT] craigslist: libevent programmer wanted
On Sun, Nov 11, 2007 at 09:46:43PM -0800, Christopher Layne [EMAIL PROTECTED] wrote: Most unix systems cache data for quite long, butwhen they write, usually user mode apps also halt. For throughput this is of little concern, but in a game server I wrote, even an fsync could freeze the server for 15-20 seconds(!) when another sync was in progress at the same time, or when some othe rprogram geenrated lots of I/O (for example a backup/restore). BTW: This isn't a global Linux issue, it's specifically an issue with ext3 and the way it handles fsync() on a global scale. I am specifically not using ext3 anywhere on any of my systems, so, no, this has nothing whatsoever to do with ext3 and its many deficencies. http://kerneltrap.org/node/14148 Personally, I use XFS (awesome design). Yeah, and even slower than ext3. By far. And this issue happens with xfs just the same. When memory grows tight and linux needs to flush, the system more or less freezes (w.r.t. I/O). I even see operations such as utime() freeze, even when everything is in the cache. (Ok, XFS is in fact the fastest filesystem when all you want to do is stream very large files, it can be very space-efficient, but at *anything* else it rather sucks, speed-wise). (And it fragments like hell, but at least it has an online defragmenter, which helps those very large files stream even faster). :) -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] C++ interface(s) [was: libev-1.3e...]
[note that i replied to the list, I hope this was ok, as I think a c++
interface *might* interest other users too. If anybody finds it too
off-topic, tell me and I will repent :]
On Fri, Nov 09, 2007 at 07:32:13PM +0100, Chris Brody [EMAIL PROTECTED] wrote:
start ()
stop ()
and would complement libev nicely as its quite object-oriented in design.
Yes-I definitely like this idea, with such an immediate re-use for a
wide-spread application.
I tried it, but found it was quite the nuisance and made the header file
rather ugly.
Since I am in immediate porting needs to replace my older iom.C solution in
gvpe and urxvt, I started a c++ interface.
It is very similar to iom.C (and also very similar to libev, of course),
but provides a bit more fluffyness (you can call -set anytime and it will
automatically stop/restart the event watcher).
Its currently only useful for embedding (its in the files ev++.h and
ev++.C), but I didn't chose to override the callback proper, so one can mix C
users of libev and c++ users of the same.
I am thinking about how to make some kind of thunk callback
connector to go directly between libev and eventxx.
This is what ev++.h also does, to keep full ABI compatibility. ev++.h is
currently more or less providing simple wrappers around the watchers allowing
object methods as callbacks.
What is missing would be a nicer namespace cleanup (wrapper functions so
ev_xxx becomes ev::xxx not ev::ev_) and a nice event loop and default
loop abstraction.
(I will probbaly switch to using enums for constants in ev.h at one point,
although this is not quite as helpful as one would assume :)
If you do make this one, I will also consider how to use it with eventxx.
Not having analyzed eventxx in detail, I believe the interfaces to be
quite different in style, with ev++ being quite a bit more primitive
(except in its callback system which is more complex).
Here is an example use (more or less straight from rxvt-unicode):
struct term {
void cursor_blink_cb (ev::timer w, int revents); ev::timer
cursor_blink_ev;
...
};
term::term ()
: cursor_blink_ev (this, term::cursor_blink_cb)
{
...
if (option (Opt_cursorBlink))
cursor_blink_ev.start (CURSOR_BLINK_INTERVAL, CURSOR_BLINK_INTERVAL);
}
void
rxvt_term::cursor_blink_cb (ev::timer w, int revents)
{
hidden_cursor = !hidden_cursor;
want_refresh = 1;
}
As you can see, there are aliases for start that also set the time - I did
this because I had lots of code using the convention, and it doesn't seem to
hurt the interface.
For multiple event loops (not really tested), the constructor has an
additional argument for the event loop to associate with (using the
default event loop if not given), and also has a set method that changes
association.
This is neccessary because I decided that the destructor has to call
stop(), which requires storing the event loop inside the watcher
somewhere.
Anyways, this is just a start to convert my existing projects, but it
seems to work quite well.
--
The choice of a Deliantra, the free code+content MORPG
-==- _GNU_ http://www.deliantra.net
==-- _ generation
---==---(_)__ __ __ Marc Lehmann
--==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED]
-=/_/_//_/\_,_/ /_/\_\
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libev-1.3e demo release - small u_char event_active() issues
On Fri, Nov 09, 2007 at 12:47:03AM +0100, Chris Brody-GMail [EMAIL PROTECTED] wrote: First, when I built on OSX, u_char was not defined in event.c. I fixed this problem by #include sys/types.h before ev.h event.h. Hmm, u_char is not used by anything in libev itself, but it is defined in event_compat.h, which should get included. Oh right, it only defines it on win32 and assumes its there on all other systems. I think I will include a typedef for it unconditionally in event_compat.h. Thanks for pointing this out! (googling around, this seems to be an already-known bug in libevent-1.3e, so I guess I wait till a new release comes out with a fix for this). Its there now (I updated the tarball), the libev equivalent is ev_feed_event. I don't really feel well about exposing it, though, without actually having a use case that isn't served better by simply calling the callback immediatelly. Small problem with MULTIPLICITY-apparently forgot dLOOPev some EV_A_ defs. Oh indeed, sorry. I replaced the tarball after adding the new functions without testing them with mulcitplicity defined. Its been updated by now (including a temporary include sys/types.h in event_compat.h, whivch shouldn't hurt, although sys/types.h is not really required to support u_char in the first place, afaicr). Given the patch below, I was able to build and run the regression Sorry for having you go through the code and add this. The patch is just perfect :) tests on OSX. In addition, all test from eventxx were working except for bench.cpp, which uses a strange copy constructor, Do you have a pointer to the source? and priority test. thats expected indeed. Thanks a lot for testing! The tarball at http://data.plan9.de/libev-1.3e.tar.gz has been updated with everything that came up so far. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libev-1.3e demo release
On Thu, Nov 08, 2007 at 03:03:40PM -0500, Nick Mathewson [EMAIL PROTECTED] wrote: - You probably want event.h to include event_compat.h inside angle brackets, not inside quotes. Its not in fact a problem with embedding as EV_STANDALONE prevents it from being included anyways (so its just a well spotted :). It is a problem with ev.h, though. - There's no event_active compatibility function defined. I looked Its there now (I updated the tarball), the libev equivalent is ev_feed_event. I don't really feel well about exposing it, though, without actually having a use case that isn't served better by simply calling the callback immediatelly. Internally, it queues the event into the pending list by separating on the three principal signal sources (EV_TIMEOUT or EV_READ/WRITE or EV_SIGNAL), but that could likely be improved. (Incidentally, ncalls is also being ignored this way, but as others found it, its not a very useful feature of libevent. I atcually wanted to provide something like this for repeating timers, as I found that event loops either do the wrong thing (call hard-realtime-scheduled timers in a loop for each skipped tick) or do the wrong thing (not telling the app how many ticks were being skipped), but none of the timer types in libev have a sensible definition for it). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] libevent win32 issues
Hi! While trying to port libev to win32, I ran into a number of issues with libevent proper (which of course still is the major part of the sources). Specifically: - evdns.c, httpc and other files include include files such as unistd.h, which are not available under windows. consequently, they do not even compile on win32 (tested with the visual c compiler). - is WIN32-Code/win32.c really the production-ready win32 backend? I can hardly believe this, as its basically not doing anything that the select backend doesn't do better and faster (for example, the win32 backend does a linear search over all(!) registered event watchers on each event_add/event_del/select call). additionally, signals are not being handled correctly (race conditions) on win32, but thats likely not a big deal on that platform. Looking around with google it seems that indeed, evdns et. al does not build on windows (except when using e.g. cygwin, but thats trivial). So I wonder if libevent as a whole is supported under windows at all in current versions? (and yes, this is with libevent-1.3e proper, not anything I wrote, to avoid the off-topic argument again). PS: the good news is, the libev core is now fully working on cygwin and native win32 (and the EV module, too, except the DNS part, which I didn't want to port to win32). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 09:56:44PM -0800, Scott Lamb [EMAIL PROTECTED] wrote: returning from event_loop, leaving the app unclear on what has happened and what to do. In any case, you can get the same behaviour as libevent by calling unloop in case of an error, so the interface is strictly more powerful. Another reason this is undesirable: to get the libevent behavior, I'd have to have this logic in *every callback* to get the same behavior. Hmm, one could actually roll this into the libevent compatibility layer. I'll think about this (didn't occur to me before). Note, however, that I think the libevent interface is undesirable, as it leaves me unclear on what happened and no reasonable way to recover (e.g. in the case of EBADF in select). Also, with libevent, you would need similar logic around each call to event_add/del and so on, because those can all fail in libevent, so its just a different place, really. (And I do think I will provide an oom-error handler for this specific case, as it is about the only generic error that isn't specific to some watcher). Thanks for these ideas, -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 10:04:05PM -0800, Scott Lamb [EMAIL PROTECTED] wrote: Realistically, I think unit tests and bug reports/workarounds are about the only reason to blacklist specific event dispatchers. select() sucks, but well, that's why it's at the bottom of the list, right? There is a hardcoded priority list that only incidentally has the same order as the constants, but yes, select comes last, even if its usually faster than poll :) If you are really concerned about syscall overhead (you mentioned it in another context, though I don't think it should be a major concern), you might want to prefer poll() to epoll() or kqueue() if you're going to Yes, I thought about dynamically using select for a small (~50 or so) fd sets as select is faster than epoll in many common small scenarios, but that is mostly an epoll vs. poll issue. For kqueue, I can't quite see the overhead, as kqueue has the same number of syscalls per iteration as select/poll (namely one). the sysclal is more complicated, but is likely faster than poll in all cases (I have zero benchmark data on that, maybe the bsd people fucked it up, but likely, they didn't :) But that's a situation where you'd want a little more coarseness: low startup overhead methods vs. low per-socket overhead ones. Resorting to naming specific event mechanisms seems undesirable in terms of older programs taking advantage of newer event facilities. It is meant strictly as a benchmark, bug workaround, tuning mechanism. Less being able to programmatically decide which is best but more offering the user a mechanism to influence based on e.g. a config file, so one can configure your mythical big app to use select because it performs better in practise. Choice isn't evil, as long as there is an obvious default choice if you don't care (which is 0/EV_METHOD_AUTO). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 09:47:59PM -0800, Christopher Layne [EMAIL PROTECTED] wrote: What is my program supposed to do? It can't distinguish them, and the correct behavior in each of these conditions is totally different. Also, in the program I'm thinking of, libev chose to kill this file descriptor probably means a network link just went down. Ergh. Great point. It should back away and leave things alone - notifying to the caller (or whoever else is listening) this is not acceptable to me - I suggest you fix it - because I won't (aka unix way). The caller in this case is the callback, however, because its the continuation of whatever code requested to watch e.g. a bad fd in the first place. Also, there is no way around an error status for the callback, one simply *must* provide a sensible status to the callback when something goes wrong, because that might be a long time after the watcher was added. So instead of notifying both the caller of the start funktion and the callback later, I don't see why notifying the callback in all cases would be worse, in fact, it lets you have errors easier. And yes, if you don't check for errors in your callback you are doomed. Returning errors to the caller of the event only requires additional checking code, and I have yet to see any code that actively checks for problems while calling event_once or event_add. But most callbacks do the sensible thing when called with EV_ERROR even when they don't care, because in the case of I/O errors they will try to read or write and see it doesn't work. No, on error your library may have muddied the water already by screwing with the file descriptors. libevent also makes errors clearer by simply returning error from the failed function (I'm thinking of event_once() vs ev_once() here.) Yes, closing is not a good idea, reporting an error and removing the event form the fd set is enough. However, I still maintain concentrating error handlign in the one place where its likely to be present already as opposed to reporting errors to places where nobody cares (show me the code that catches errors after making event_(once|del|add) calls) is the right thing to do. Wether one reports more detailed errors is then another question. And might be solved as easily as giving errno a defined value to use. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Re: Heap-based timer implementation (benchmark)
On Mon, Nov 05, 2007 at 01:46:19PM -0500, Nick Mathewson [EMAIL PROTECTED] wrote: In case anybody here isn't watching the commits list [1], Niels applied a patch from Maxim Yegorushkin to make the timeout implementation based on a heap, rather than a RB-tree. Thanks, Maxim! Thats great news! I just ran a course benchmark against libevent trunk (which has the changes entry for the min_heap code and also references to it in the code). Its the same benchmark, except that I used less repeats and less x-axis sampling points. timers 1.3e vs. libev: http://libev.schmorp.de/dat.t1.png trunk vs. libev: http://data.plan9.de/trunk.t1.png The libev results are virtually the same (ignoring some spikes). It looks as if the libevent growth went down, however: libevevent now has similar growth characteristics as libev (probably due to the heap). Unfortunately, absolute timings are much worse now for libevent, the 1.3e version is definitely faster. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] sensible thread-safe signal handling proposal
On Sat, Nov 03, 2007 at 03:45:39PM -0700, William Ahern [EMAIL PROTECTED] wrote: Curious how you managed to do this. Are you checking the process PID on each loop? I considered that, but I think its too slow (one also needs to be careful that watchers don't change e.g. epoll state until the getpid check is done), or at leats I think I don't want that speed hit, no matter what. After giving signal handling and threads a lot of thought, I came to these conclusions: - requiring pthreads or windows mutexes by default is not acceptable, but thats the only way to distribute signal events among event loops properly, or globally among many threads if signal handling were global. - the only way to do it without locking is to only allow a single loop to handle events. This is the interface I came up with to manage multiple loops (which I think makes more sense than the interface currently in libevent): struct ev_loop *ev_default_loop (int methods); void ev_default_destroy (void); void ev_default_fork (void); this would create the default loop (event_base). ev_default_loop would always create the same loop, and it would be the one to use for third-party libraries in general, too. The fork method can be called in the parent or child (or even in both, or without forking), and it would destroy and recreate the kernel state but keep all the watchers for the default loop. struct ev_loop *ev_loop_new (int methods); void ev_loop_destroy (EV_P); void ev_loop_fork (EV_P); This would create additional loops (event_bases). The difference is that these cannot handle signals (or child watchers) at all, with the default loop being the only one to do signal handling. This would be consistent with how signals are usually handled in a pthreads environment: block signals in all threads and in one thread handle them all (sigwait, or using the default mainloop). No locking inside libevent would be required this way. I'll implement this in my libev replacement code, unless somebody else comes up with a better idea. One such idea that isn't better, but different, would be to require the user to provide mutex support, such as in ev_init_locking (size, init_cb, lock_cb, unlock_cb, free_cb) or similar, then use locking and let any event loop handle the signals and distribute signal events to the relevant loops. But I am not sure how much locking would be required and I assume it would be a lot, as one would need to handle the case where one thread handles a signal for an event_base currently in use by another thread. Looking at the code in libevent, it seems that signals get handled by whatever loop was started last, so signal handling is not reliable at all unless one registers the signal handlers in all threads, which is hard to do in a thread-safe manner (for the user code). Having a deterministic model where one loop handles all that would definitely an improvement over this. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
no change for changes sake, I can explain the rationale behind each and every change (I hope :). A few notes: * what is the purpose of EV_COMMON? Allowing customised event watchers. If you are concerned, treat it as a an internal symbol. Its use is documented here: http://cvs.schmorp.de/libev/README.embed From first glance, I'm concerned that it could not be used properly unless libev.so and all callers are compiled with the same flags, which seems impractical if the library ever gains wide use. This is true, but its an optional feature you don't have to use. In case you wonder, EV, the perl interface to libev, uses this feature. It makes most sense when embedding, of course (not all the world is an .so :). * on ev_once failure, you're calling the callback with EV_ERROR? Yuck. That's quite surprising behavior, and I could see it leading to stack overflows as each ev_once tries to issue another one. All callbacks will be called with EV_ERROR when an error occurs. And yes, if you don't do error handlign and endlessly retry the same operation in a loop, you run into problems. But as that is an obvious programming bug, I don't see any problem here. Besides, if you cannot malloc the few bytes ev_once requires you need a *lot* of good error handlign code to continue sensibly. * What's your use case for ev_loop_new() and ev_loop_default()'s bitmask of allowed implementations? libevents unconditional use of getenv raised concerns with me and apperently some users on this list, too, so this is one way to disable this (EVMETHOD_ANY instead of EVMETHOD_ALL). Also, I am sure some apps want control over the allowed event loops, e.g. to rule out select becasue it is known to be not wrokign for them. * (again, just skimming) you're closing fds automatically on ENOMEM? Ergh. There is little else to do. This isn't malloc or so, but a kernel interface, and usually due to hard limits (not really out of memory). Point being, libev(ent) cannot continue in this condition, there cannot be any progress. Closing the fd and signalling the relevant part of the application is in no way different then a network overload or problem resulting in the same condition. If the app cannot handle that, deep shit. That seems rather undesirable for many applications. Well, its arguably better than libevents behaviour, which is simply returning from event_loop, leaving the app unclear on what has happened and what to do. In any case, you can get the same behaviour as libevent by calling unloop in case of an error, so the interface is strictly more powerful. Thanks a lot for your questions, I hope I could clarify some things and design decisions. Its indeed not easy to get everything right, and I am sure the ev.h API can get improvements. In some cases there have been design trade-offs (no mutexes leading to less automatic management for example). I do think the design is useful in practise, where error handling is rarely done to the utmost extent and sensible behaviour in dead-end situations counts a lot. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Mon, Nov 05, 2007 at 02:42:16AM +0100, Marc Lehmann [EMAIL PROTECTED] wrote: However, libev has an interface (ev_timer_again) that incrementally updates the heap. Also, for repeating timers in general, there is no removal/insertion but only an incremental update. Oh, and sorry for always forgetting stuff... the rationale for supporting this operation is that I think its pretty important to support timers that get reset all the time without usually firing (e.g. idle read timeouts on a normally busy tcp connection). The other rationale is that its trivial to implement with a heap, because you already have all the code to do it: /* incremental timer update in ev_timer_again: */ w-at = mn_now + w-repeat; downheap (timers, timercnt, w-active - 1); /* compare with timer removal: */ timers [w-active - 1] = timers [--timercnt]; downheap (timers, timercnt, w-active - 1); In such a case (updating a timer) the event will simply wander down from current place in the heap to its new place. I am not sure wether this can be done with an rb-tree (likely), but I am sure that I do not want to have to maintain the code that does that :) (In any case, see the timer benchmark for a good comparison of heap vs. red-black-tree). -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
On Sun, Nov 04, 2007 at 06:00:56PM -0800, Christopher Layne [EMAIL PROTECTED] wrote: My point was that an event_del() on an event which has been called before it's timer has expired *or* an event_add() with a new timer will require reheapifying atleast part of the timer heap. Hmm, I do not see why this would ever be the case. Removing a timer that hasn't expired yet might actually be much cheaper than removing the one at the top element because it isn't at the root, so the n in the O(log n) is potentially much smaller. Having an intrinsically sorted collection of elements and then altering a value within the middle of said collection before it has sifted to the top will require a reheap from that point on. Not sure what you mean with re-heap, but the opertaion you describe is the same O(log n) operation as for removing elements elsewhere in the heap. And given that you take the latets timer and insert it at that point, removing a timer that hasn't expired is usually faster. Which isn't really that big a deal as similar time is spent in the present RB implementation as it is. No, I still maintain that the RB tree is slower because its rebalancing operations are frequent and very complex. Heap code is trivial. Yes, they have the same asymptotic growth behaviour, but the practical cases are all very far away from infinity, and the hidden C in O(log n) is quite important. (Again, please refer to the benchmark at http://libev.schmorp.de/bench.html which directly contrasts behaviour of libevent and libev with timers and no timers, especially look at the difference in runtime when timers are being used). I'm all for a binary-heap rather than a RB-tree personally. I think the performance will benefit primarily for heap_min() (which is done before every re-entry into the event backend to reset the max-wait timer for epoll/poll/select, etc). I thought so, too, until recently but in fact the event loop is run pretty rarely (except in benchmarks), and if you handle hundreds of clients within each run (very typical of busy servers), then you can have hundreds of timer updates, and these do show up in timing measurements. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] test/*.c files use installed header files not the ones from the package
during integration of libev into libevent, I found that the test/* files include the *installed* header files instead of the ones from the package. this doesn't work if there are any structure changes (obviously the structure change in this case was going to libev). fixing the #include event.h statements to use fixed this. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?
While debugging a problem of http.c with libev, I found this code in evhttp_write_buffer: if (event_pending(evcon-ev, EV_WRITE|EV_TIMEOUT, NULL)) event_del(evcon-ev); event_set(evcon-ev, evcon-fd, EV_WRITE, evhttp_write, evcon); evhttp_add_event(evcon-ev, evcon-timeout, HTTP_WRITE_TIMEOUT); now, event_set initialises a struct event. the problem with the above code is that it calls event_set on a struct event that is currently in use and expects it to work. this doesn't directly contradict the sparse documentation, but is rather weird. looking at event_set closely, I see that it doesn't initialise the struct event fully (but it does initialiseev_flags). I also saw that some parts of the testsuite are indeed expecting that libevent treats struct events that have never been initialised by an event_set properly (= ignoring them). I therefore conclude that there is no function in libevent that actually initialises struct events. This strikes me as a rather weird design, especially as it requires the user of libevent to clear all his struct events before passing it to event_set. It also is not threadsafe, as event_set overwrites ev_base with current_base which might not be the correct one. Note also that some tets programs do not properly clear the event structure they allocate (test-time for example). This makes me wonder about these questions: 1. could anybody confirm wether a user must clear the struct event before passing it to event_set? 2. shouldn't this rather fundamental requirement be documented *somewhere*? 3. is there some guarantee that one can call event_add/del on cleared memory arrays and this will not crash? 4. will the code fragment above not cause an event to be added twice to some lists, as event_set clears the flags that event_add uses to detect wether an event is already on the list? In any case, I can manage in the libev emulation layer by also relying on some has been initialiased before flag, but this is of course no guarantee that code like the above will ever work (and I suspect it will not). In general, this whole design strikes me as rather messy. (in libev, you always have to call one of the watcher initialiser macros that do not depend on earlier contents). Thanks a lot for any insights. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?
On Sat, Nov 03, 2007 at 05:40:50PM -0700, Niels Provos [EMAIL PROTECTED] wrote: I think the documentation is very clear on this: I read that part, but the documentation doesn't mention anything about wether one can pass in uninitalised memory (as I explained in my mail). Did you read it? The function event_set() prepares the event structure ev to be used in future calls to event_add() and event_del(). The event will be prepared to call the function specified by the fn argument with an int argument indicating the file descriptor, a short argument indicating the type of event, and a void * argument given in the arg argument. The fd indicates the file descriptor that should be monitored for events. The events can be either EV_READ, EV_WRITE, or both, indicating that an application can read or write from the file descriptor respectively without blocking. Ok, where does it say one can event_add, event_set, and then event_add again? This is what http.c actually does. And where does it say that one can call event_del on a struct event without calling event_set before? This is what other code actually does. Perhaps you might like to create a libev mailing list and discuss further development of libev there? It would be off-topic there, as the libev API doesn't suffer from these problems, and its testsuite isn't as buggy. I find your somewhat discourteous insinuation of bugs distracting. 1. I reported a number of bugs in libevent so far. I am not insinuating, but reporting bugs, to improve libevent. 2. I rewrote the libevent core part to be faster, much more scalable, with less artificial limitations and worked hard to contribute this back to libevent. I don't think reporting bugs (or talking about possible bugs) is discourteous, nor do I think I was in any way discourteous. If you don't want to hear about bugs or improvements for libevent, you can say so, but I think you are treating me rather unfairly, given the amount of work I did. Even if libev never gets integrated into libevent, fixing the bugs I found while trying to get it to run should be of interest to you. Or is it the fact that something came up that is faster and more featureful (and smaller, too) than libevent that distracts you? Fear not, as I still intend to contribute, but I realyl don'T want to be insulted for improving your library. :( Thank you, Niels. omg, a top-poster and full-quoter, too. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?
On Sun, Nov 04, 2007 at 01:57:31AM +0100, Marc Lehmann [EMAIL PROTECTED] wrote: I find your somewhat discourteous insinuation of bugs distracting. 1. I reported a number of bugs in libevent so far. I am not insinuating, but reporting bugs, to improve libevent. I wanted to mention that I also delivered the fixes to those bugs while reporting them. -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] bug in evhttp_write_buffer/weird event_set semantics?
On Sun, Nov 04, 2007 at 02:00:15AM +0100, Marc Lehmann [EMAIL PROTECTED] wrote: On Sun, Nov 04, 2007 at 01:57:31AM +0100, Marc Lehmann [EMAIL PROTECTED] wrote: I find your somewhat discourteous insinuation of bugs distracting. 1. I reported a number of bugs in libevent so far. I am not insinuating, but reporting bugs, to improve libevent. I wanted to mention that I also delivered the fixes to those bugs while reporting them. And so that this isn't just idle words, here is a list: - debian bug #448165 against libevent (the TAILQ_ENTRY problem just reported) - debian bug #448173 crash bug in evdns_resolve_reverse_ipv6 (with obvious fix) - the header file problems, causing crashes or worse in the testsuite in ABI changes. - the http.c reported and (by now) verified (no fix, but the function should probably just call event_del). Thats not insinuating, these are clear bugs and clear reports. I think you need a serious attitude adjustment if you treat actively contributing people like this :( Doh :( -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent
way to support both, so I will provide both. Thanks a lot for your input! -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] test/*.c files use installed header files not the ones from the package
On Sat, Nov 03, 2007 at 04:18:19PM -0700, Niels Provos [EMAIL PROTECTED] wrote: I appreciate your insights, but your message has nothing to do with libevent. The make files in libevent use the -I option to provide the path to the header files. Well, since you thre documentation at me before, let me just say its high time to actually read up on what those -I switches do to various compilers But then, I see you are not really interested in fixing bugs: I also noticed that you seem to have found several bugs in libevent. And I keep finding more the more I look at it. It would be nice if you could send patches for them. I did that for all bugs where I could identify the right solution. Most bugs are actually fixed by the event core replacement code I wrote, for the remaining ones I send in detailed instructions how to patch the file. I am a little bit dubious about some of the claims such as not being able to remove an event while in a callback. Yes, that seems to be simply wrong, sorry for that. If you keep hitting and finding bugs (or undocumented features, such as not beign able to have multiple event watchers for most events, which is probably not a bug, but certainly something outside of my expectation for an event loop), sometimes one makes a mistake. I am, however, not that thrilled anymore about contributing to libevent after the treatment you give to me, ignoring my patches, and insulting me :( -- The choice of a Deliantra, the free code+content MORPG -==- _GNU_ http://www.deliantra.net ==-- _ generation ---==---(_)__ __ __ Marc Lehmann --==---/ / _ \/ // /\ \/ / [EMAIL PROTECTED] -=/_/_//_/\_,_/ /_/\_\ ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
