Re: [Libevent-users] libevent test failure: FAIL test/regress_http.c:3092: assert(req)
On Fri, Mar 8, 2013 at 6:47 PM, Dan Kegel d...@kegel.com wrote: On Fri, Mar 8, 2013 at 1:05 PM, Dan Kegel d...@kegel.com wrote: On Fri, Mar 8, 2013 at 12:44 PM, Dan Kegel d...@kegel.com wrote: libevent master pulled as of Jan 11, fails tests for me. System details: OS: Ubuntu 12.04.2 64 bits Here's one failure log: regress: http/connection_retry: FAIL test/regress_http.c:3092: assert(req) FAIL test/regress_http.c:3192: assert(test_ok == 1) [connection_retry FAILED] 1/209 TESTS FAILED. (8 skipped) That failure doesn't seem to happen on a similar box running Ubuntu 10.04. Scratch that. It happens sometimes (1 in 4 tries so far) on 10.04, too. I'll try to narrow it down before I post again. Ouch. This sounds like the kind of thing that's probably going to be timing-dependent. Anything you can do to try to track down what's going on here would be great. :/ (Also, the official libevent list is now libevent-us...@freehaven.net . Subscription information is at http://archives.seul.org/libevent/users/ ) ___ Libevent-users mailing list Libevent-users@monkey.org http://lists.monkey.org:8080/listinfo/libevent-users
Re: [Libevent-users] SONAME in 2.0.12
On Fri, Jun 17, 2011 at 12:11 PM, Leo 'costela' Antunes
cost...@debian.org wrote:
On 17/06/11 17:42, Nick Mathewson wrote:
* AGE: With how many previous versions of the ABI is Current
version backward-compatible? This increments whenever the ABI changes
in a backward compatible way, and
h, that's the catch I merrily overlooked!
According to the libtool source, different shared-library systems
build their version numbers in different ways. Linux does
{Current-age}.age.revision; freebsd does current.revision; Irix
gets downright weird, and so on.
So yeah; nothing to worry about here. :)
Indeed, makes all the sense in the world now.
But since I got your attention, I'll plug a follow-up question about the
build-system: what's the reason for setting BUILD_WITH_NO_UNDEFINED only
for cygwin and win32?
I'm currently patching the packages to always use it (reasons outlined
here[0]) and haven't run into any problems, but maybe I'm overlooking
something obvious again.
The main reason for doing it in windows only is that it is
(supposedly? apparently?) necessary to get any kind of working DLLs
out of gcc on windows. (I'm not sure that whether it's *absolutely*
needed there, or whether it's just only needed so long as you don't
explicitly list all of your DLL imports and exports.) Until your
mail, I wasn't actually aware that the option did anything useful on
non-windows platforms. :)
If not, would you accept a patch to turn it into a configure-time
option? (besides keeping it on for cygwin and win32, of course)
I'm not sure I understand the article you linked to. What's the
impact here, and what are the potential advantages/disadvantages?
Either way, I'd be a lot more comfortable about a patch on the 2.1
series than on 2.0.
yrs,
--
Nick
___
Libevent-users mailing list
Libevent-users@monkey.org
http://lists.monkey.org:8080/listinfo/libevent-users
Re: [Libevent-users] SONAME in 2.0.12
On Fri, Jun 17, 2011 at 6:25 AM, Leo 'costela' Antunes
cost...@debian.org wrote:
Hi,
In the process of building the Debian package for 2.0.12, I noticed
Makefile.am contains VERSION_INFO=6:1:1, but the generated libs have all
SONAME=*.so.5
I'm still parsing through the build-system (I'm the new guy helping with
this package) trying to figure out what's going on, but decided to ask
around before assuming it was a mistake. So, was this intentional? Am I
missing something obvious?
No, but you're missing something that will seem obvious in retrospect:
Libtool VERSION_INFO triples don't correspond directly to SONAMEs.
The three elements of the VERSION_INFO triple are:
* CURRENT: The current version of the ABI. This increments whenever
a binary interface changes, or a new binary interface is added.
* REVISION: Which implementation of the Current ABI is this? This
is set to 0 whenever the ABI hits a new version, and increments by one
whenever
* AGE: With how many previous versions of the ABI is Current
version backward-compatible? This increments whenever the ABI changes
in a backward compatible way, and
According to the libtool source, different shared-library systems
build their version numbers in different ways. Linux does
{Current-age}.age.revision; freebsd does current.revision; Irix
gets downright weird, and so on.
So yeah; nothing to worry about here. :)
cheers,
--
Nick
___
Libevent-users mailing list
Libevent-users@monkey.org
http://lists.monkey.org:8080/listinfo/libevent-users
Re: [Libevent-users] Bufferevent throughput experiments
On Sun, Jan 16, 2011 at 4:39 PM, Marcel Roelofs marcel.roel...@gmail.com wrote: Hi, Marcel! Thanks for the I'd like to see a lot more optimization work in Libevent 2.1. One thing that this really needs IMO is more work on benchmarking and profiling. Chris Davis started work on a benchmarking tool last year; his site seems to be down, so I've uploaded a copy to git://github.com/nmathewson/libevent-bench.git . I'd encourage everybody else who'd like to work on Libevent performance to look into using some reproducible and shareable benchmark mechanism (ideally, a small program) so that we can try them out and see the impact of different changes together. This way, instead of just fixing bottlenecks one by one, we can track performance over time and make sure that we don't introduce performance regressions between releases. , [...] So, I decided to do a little bit of profiling on my Windows box, and noticed a lot of time spent in grabbing the locks in read_complete and write_complete in bufferevent_async.c. After some further digging I noticed a couple of fixed buffer sizes limited to 16K and 4K in bufferevent_async.c bufferevent_ratelim.c buffer.c Weird that this would have to do with buffer sizes; I would have thought that the fixed maximum amount send/received per call to wsasend/wsarecv would have hurt more. (Actually, is that what you mean? The 16K in bufferevent_async is not a buffer size; it's a read limit.) Another possibility to look into is whether we can better tune the rest of our application by looking into algorithms with less lock contention, or twiddling the SPIN_COUNT of the locks on windows, or what. If possible, could you upload your profiles somewhere? It would be cool to see where else you found bottlenecks. [...] Now I'not saying that 64K in internal buffers would be a good price to pay for every type of connection, but I can imagine that when throughput is important it would be nice if one could change these fixed buffer sizes through an API. If we have to do that, so be it. But personally, I'd prefer to check out if there isn't some way we can get the Libevent code to do the right thing here more automatically. After all, nearly all users would rather just leave the buffer sizes alone, so if we can find some way to get better performance without forcing developers to mess with magic numbers, that would make for an even easier-to-use library. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://lists.monkey.org:8080/listinfo/libevent-users
Re: [Libevent-users] structure redefinition in _libevent_time.h and winsock2.h
On Fri, Feb 5, 2010 at 10:30 AM, Shiqing Fan f...@hlrs.de wrote: Hi, When using libevent 1.4.13 in another project on Windows, we found some structure are redefined in _libevent_time.h and winsock2.h, for example 'struct timeval'. I made a patch to solve this problem, but I'm not sure if it's correct to exclude _libevent_time.h on Windows. Something fishy is going on here. For one thing, most of the files you changed should never actually get built for windows: devpoll.c, poll.c, kqueue.c, and epoll.c all refer to backend mechanisms that don't exist on windows. Other than that, I'm trying to reproduce the bug, but I'm not having much success. For me, 1.4.13 builds fine on Windows under mingw, and the current 1.4.x development version in git version builds fine on Windows under mingw, or with VC++ and nmake. Since you patched configure, I'm guessing you're on some version of mingw or maybe cygwin? If so, which version? ___ Libevent-users mailing list Libevent-users@monkey.org http://lists.monkey.org:8080/listinfo/libevent-users
[Libevent-users] Moving the libevent-users mailing list to freehaven.net
Hi, all. As promised, we're moving the libevent-users mailing list due to monkeymail's general flakiness. The new address is libevent-us...@freehaven.net. All current members of libevent-users@monkey.org have been subscribed to the new mailing list. We'll be shutting down the @monkey.org address soon, or making it forward to libevent-us...@freehaven.net, or whatever turns out to be right. Subscription information and archives of the new list will be at: http://archives.seul.org/libevent/users/ [We'll try to add the old archives if we can dig out the mbox files from wherever they have wound up.] Niels, could you please update the website to point to the new list, and either disable the old one or make it forward? Thanks for your patience, everyone! yrs, -- Nick Mathewson ___ Libevent-users mailing list Libevent-users@monkey.org http://lists.monkey.org:8080/listinfo/libevent-users
Re: [Libevent-users] API compatibility (was Re: patch to use different return value for event_base_loop())
On Tue, Sep 29, 2009 at 11:39:59AM -0700, Ka-Hing Cheung wrote: On Sun, 2009-08-16 at 10:39 -0700, Nick Mathewson wrote: but I am willing to submit another patch (the one with accessor to base-event_gotterm/event_break) if you think that's the better way to do it. Probably. Sorry for taking so long. Attached is the promised patch which adds accessors to base-event_gotterm/event_break, as well as modifying when they are reset in event_base_loop(). Applied; thanks! -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://lists.monkey.org:8080/listinfo/libevent-users
Re: [Libevent-users] Problems when compiling asciidoc of libevent reference manual book
On Sun, Sep 20, 2009 at 07:45:08PM -0700, Raine Fan wrote: Hi, yesterday I 'git cloned' the reference manual book from git://git.torproject.org/~nickm/git/lebook and I'm having problems when compiling it. Here is the output: ERROR: 01_intro.txt: line 270: missing listing block closing delimiter I'm using asciidoc version 8.2.7 on Ubuntu Jaunty. Any help? Try a newer asciidoc. I don't know how new it needs to be exactly, but the makefile says: # asciidoc 8.4.3 is known to work. asciidoc 8.2.5 is known not to work. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Infinite loop in event_process_active()
On Thu, Sep 17, 2009 at 02:56:59PM +0200, nicolas dumont wrote: Hi, I'm using events in 1.4.12-stable. [...] Finnaly, to prevent to go in the infinite loop , do I have to run event_del(ev) each time before using event_set(ev) ? This is exactly so. You can't call event_set or event_assign on an event while it is added. Unfortunately, because event_set works on uninitialized memory, there is no way for it to check whether it's looking at an added event, or a random chunk of memory that happens to have bits set in the right place. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Must delete events before closing the socket handle
On Fri, Sep 18, 2009 at 10:57:25AM +0800, Adrian Chadd wrote: Well, the order of which you can do this depends on what the underlying OS will do for you. I'd suggest sticking to delete event, close socket usage. Anything else isn't portable. Nick, is this actually documented anywhere? Not that I recall, and it really ought to be. Anybody want to write the documentation patch? -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Must delete events before closing the socket handle
On Fri, Sep 18, 2009 at 11:09:52AM +0800, Adrian Chadd wrote: On Thu, Sep 17, 2009, Nick Mathewson wrote: Nick, is this actually documented anywhere? Not that I recall, and it really ought to be. Anybody want to write the documentation patch? I've been threatening to do this for ages. I may as well start now. Is there a project wiki or something somewhere where I can start braindumping clue into? Right now most of the clue is in the reference book thing I've started writing. The source for it is here, in Git: git://git.torproject.org/~nickm/git/lebook.git I'm glad to apply patches or pull contributions from other people's repositories. There's no wiki; if somebody knows a good wiki host that we should use, I'd be glad to set one up for non-book things, though I'd like to concentrate as much clue in the book as possible. Also, as a heads-up: We'll probably be moving the mailing list to some other host soon, due to monkeymail's general unreliability. Currently sourceforge seems likeliest. The likeliest time for this, assuming we don't flake out, is next week once Niels is back home. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] API compatibility (was Re: patch to use different return value for event_base_loop())
On Thu, Aug 13, 2009 at 11:04:52AM -0700, Ka-Hing Cheung wrote: On Thu, 2009-08-13 at 09:16 -0700, Nick Mathewson wrote: On Wed, Aug 12, 2009 at 11:42:47AM -0700, Ka-Hing Cheung wrote: [...] I do realize that this breaks existing assumptions about return values. What if instead of using a different return value, we don't reset base-event_gotterm and base-event_break when triggered? We can reset them at the beginning of event_base_loop() instead. How would you inspect them? The event_base structure isn't exposed in any public headers. I suppose we could add a new accessor function. Right. I like the idea of the original patch, but I'm also concerned about return values. If anyone has written code like if (event_base_loop(base,0)0) or if (event_base_loop(base,0)==-1), they'll get unexpected results. Yes I like that too. I don't know how strict the api compatibility requirement is, The foremost API requirement is: we don't want to break source compatibility for existing correct programs. Breaking existing programs annoys developers, and prevents distributors from upgrading to new Libevent versions[*]. Since the return values for event_base_loop() and friends have been documented and relied upon for a while, I don't think that we can change them safely. but I am willing to submit another patch (the one with accessor to base-event_gotterm/event_break) if you think that's the better way to do it. Probably. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] patch to use different return value for event_base_loop()
On Wed, Aug 12, 2009 at 11:42:47AM -0700, Ka-Hing Cheung wrote: [...] I do realize that this breaks existing assumptions about return values. What if instead of using a different return value, we don't reset base-event_gotterm and base-event_break when triggered? We can reset them at the beginning of event_base_loop() instead. How would you inspect them? The event_base structure isn't exposed in any public headers. I suppose we could add a new accessor function. I like the idea of the original patch, but I'm also concerned about return values. If anyone has written code like if (event_base_loop(base,0)0) or if (event_base_loop(base,0)==-1), they'll get unexpected results. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] ANN: Libevent 2.0.2-alpha is released
Good morning, evening, night, or afternoon! The first alpha release in the long-promised Libevent 2.0 series is finally out. You can download Libevent 2.0.1-alpha from http://monkey.org/~provos/libevent-2.0.2-alpha.tar.gz This is an alpha release. Libevent 2.0 is not finished. There will be bugs, and we make no promises about the stability of any APIs introduced in the 2.0.x-alpha releases. When you find bugs, please let us know. Libevent 2.0 is intended to be backward compatible with the Libevent 1.4 APIs[*]. Any program that worked with Libevent 1.4 should still work with Libevent 2.0, unless we screwed up. Please test your programs when you have a chance, so that if we _did_ screw up, we can notice soon. [*] Unless you were messing around with the internals of internal structures. This release adds many new features to the previous alpha release, and fixes many bugs. See the ChangeLog for full details. Highlights include: - evdns is now threadsafe, with locking support - There's an evconnlistener type that you can use to abstract cross-platform differences in accepting connections. - The evbuffer interface (and therefore bufferevents) now supports zero-copy much better. - About a zillion fixes for tricky bugs in the new Libevent 2.0.1-alpha code. Special thanks to everybody who helped find bugs and improve the code, especially James Mansion, Zack Weinberg, and Joachim Bauch. yrs, -- Nick Mathewson ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent cmake scripts
On Sat, Jul 25, 2009 at 01:07:40PM +0900, Brodie Thiesfield wrote: On Fri, Jul 24, 2009 at 9:42 PM, q6Yr7e0o nIJDVMjCu9oqc...@googlemail.com wrote: Would it be possible to create cmake scripts for 2.x, too since they will much likely be included into the release I'll look into it, but at the moment I'm using 1.4.* releases. Thanks! That would be much appreciated. I don't know if it will become our primary build tool or not (we'll have to see how it works out), but it would sure be nice to make the MSVC people happy. Incidently, the 1.4.12 release works with the current make scripts unchanged. Nice. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] How to schedule a plain event
On Mon, Jul 13, 2009 at 02:29:21PM +0200, q6Yr7e0o nIJDVMjC wrote: Hi, how is it possible to schedule a callback in the next eventloop iteration? Of course i could add a timer event with timeout 0 but this seems like a lot of overhead. I just want a function to be called in the next event iteration (i can't call it directly in another libevent callback because the first callback is called inside the buffer_chain_free method and the method itself wants to write to the buffer). The easiest thing to do here is to use the event_active() call to make the event active now. But it doesn't do quite what you asked for: the callback will be run later in the current iteration of the loop, not in the next iteration. If it needs to be in the next iteration, I can't offhand think of a better way than adding an event with a 0 callback. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] HAVE_CONFIG_H usage in buffer.c
On Thu, Jul 02, 2009 at 07:00:46PM -0700, Ka-Hing Cheung wrote: In buffer.c:28 an include event-config.h is wrapped by HAVE_CONFIG_H, but on line 85 there's a bare include event-config.h. Should the 2nd include and the HAVE_CONFIG_H check removed? IMO, pretty much all of the '#ifdef HAVE_CONFIG_H' stuff should go; we no longer support building without a config.h, if we ever did. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] How libevent works
On Fri, Jun 05, 2009 at 03:17:00PM -0400, Alex wrote: Hey there. When working with bufferevents, in what order does libevent execute callbacks? Does it fill all the buffers first and then call the CBs, or does it fill a buffer and call its corresponding CB, one at a time? From what I understand, it is more efficient to recv and send on all available sockets all at once rather than recv, process, recv, process, etc. By default, the bufferevent tries to read as much data as it can, and it invoke the read callback after it has done so. You can change this behavior in a few ways, including: - If you set a low-watermark for reading on that bufferevent, it doesn't invoke the callback until a given number of bytes are available. - If you set the bufferevent's DEFER_CALLBACKS flag, it doesn't invoke any of the bufferevent's callbacks until it is done handling IO for all the other active bufferevents. (So if you set the flag on all your bufferevents, it does IO on all of them, then it runs all the appropriate callbacks.) The second feature is only available in Libevent 2. I hadn't heard about a performance boost from clustering the reads on _unrelated_ sockets, but apparently today is a good day for me learning new things. In my case I have to wait until n bytes are recv'd before I can begin processing. Therefore it might not be worth it to use bufferevents, since I will have to use buffers which are not drained. It sounds like you want to look at the watermark feature on bufferevents. You can set a read low-water mark on a bufferevent, and you won't get a callback until at least that number of bytes have been read. Shameless plug: I'm still making progress in trying to document all this stuff! You can see the latest draft at http://www.wangafu.net/~nickm/libevent-book/ It now covers bufferevents. Please send me corrections where it's wrong. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] How libevent works
On Fri, Jun 05, 2009 at 06:31:35PM -0400, Alex wrote: [...] Hey, great book! I checked it out of git about an hour ago and have been reading it ever since. Since I am decrypting data from the TCP stream I might also want to look in to filtering with the BEV_NEED_MORE flag as well as the aforementioned watermark. Okay! You should know that the filtering is a little experimental, and so you might run into bugs or API warts on it. If you can't make it do what you want, don't assume that you are the one at fault: let us know. Also, if you're using SSL/TLS, be aware that you can't actually write a filter for it using the bufferevent_filter API, since sometimes a TLS read operation can require underlying reads _and_ writes, and the bufferevent_filter API only works for filters that work as a pure transform on an input or output stream. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Libevent 2.0.1-alpha is released
On Fri, May 22, 2009 at 07:44:36PM +0100, James Mansion wrote: Nick Mathewson wrote: [...] So reading that message again, I guess I wasn't as explicit as I should have been. Short version: yes, I think the next release will build okay with visual C. But somebody besides me test out the code in the subversion repository to make sure there aren't any surprises. yrs, (Using: Visual Studio Express and Windows 7 SDK Beta, Vista Home Premium 32 bit) As of r1298. Thanks! I've fixed some of the stuff below, I hope. See comments at the bottom for some more stuff we should do that I will probably need help with. - Build fails on ssize_t in args to evbuffer_reserve_space. If I search for ssize_t then I can see: #define _EVENT_ssize_t use of ssize_t use of SSIZE_T use of ev_ssize_t I added: #define ssize_t _EVENT_ssize_t #define ev_ssize_t _EVENT_ssize_t I've changed the code to use ev_ssize_t thoughout, which should be defined in include/event2/util.h. [...] time-test.c does not compile because event2\event_struct.h uses struct timeval.h and WinSock.h has not been included. I added: #ifdef WIN32 #include winsock2.h #include windows.h #include io.h #endif to time-test.c, but the reality is that any file including event_struct.h will need to do this. I suggest using a private type that is equivalent of including WinSock2.h into a base libevent header. This isn't likely to increase the dependencies of a Windows application that uses libevent. A private type? What kind of thing do you mean here? - signal-test.c and time-test.c use the C99 __func__. I added: #define __func__ __FUNCTION__ in a Win32-specific block - I'm lazy, its compiler-specific really. Fixed in svn. - Execution fails: C:\src\libevent\WIN32-Prjtime_test\debug\time_test [err] evsig_init: socketpair: Successful WSAStartup not yet performed [WSANOTINITIALISED ] C:\src\libevent\WIN32-Prjsignal_test\debug\signal_test.exe [err] evsig_init: socketpair: Successful WSAStartup not yet performed [WSANOTINITIALISED ] Fixed these two too. event_test fails the same way though I don't get an error message when running from the command line (I do under the debugger, which is odd). It looks like event_test is just plain broken on Windows. Whoever wrote it seems to have been under the impression that CreateFile would give you something you can pass to select(), which isn't so. (Yeah, we totally need new example code. If anybody wants to write some, that would be wonderful.) OTHER QUESTIONS: Q1: It sounds like you're winding up with project files that are better than the ones we have. Is that so? Should we ship yours instead? (I'd try to reproduce your changes, but I always screw up Visual Studio stuff whenever I touch it, and wind up making changes only to the debug build, or adding absolute paths by mistake, or stuff like that.) Q2: Were you able to compile regress.exe in test? That's the one with the actual serious regression tests that let us know whether Libevent is working. I don't know if there's a project for it. Its sources on win32 should be: regress.c regress.gen.c regress.gen.h regress.h regress_buffer.c regress_bufferevent.c regress_dns.c regress_et.c regress_http.c regress_iocp.c regress_main.c regress_minheap.c [NOT INCLUDED ON WIN32: regress_pthread.c] regress_rpc.c regress_util.c [NOT INCLUDED ON WIN32: regress_zlib.c] tinytest.c tinytest.h tinytest_macros.h Q3: There are a bunch of other new files in Libevent 2.0 that may not be in the old project files. On win32, the libevent_core library should contain: WIN32-Code/win32.c buffer.c buffer_iocp.c bufferevent.c bufferevent_async.c bufferevent_filter.c bufferevent_pair.c bufferevent_sock.c event.c event_iocp.c evmap.c evthread_win32.c evutil.c listener.c log.c strlcpy.c and the libevent_extra library should contain: event_tagging.c http.c evdns.c evrpc.c (The libevent library is for backward compatibility, and should contain all of the above.) If any of these sources are missing from the appropriate project files, they should be added in. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent boehm gc
On Tue, May 19, 2009 at 10:20:08PM +1000, Peter Ross wrote: Hi, I've been tracking down a bug with boehm gc prematurely collecting the bufferevent callback arg value. This occurs when one is the middle of processing another bufferevent in the read callback. I've tried to grok the code to understand how one would get from the eventbase to the other bufferevent, so I can check where the boehm gc is losing the pointer, but having difficulty understanding the code. Could someone explain how I get from an event base to all the bufferevents associated with that base? You don't. Only when a bufferevent is actively listening for read or write events does it have any references from the event base. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Libevent 2.0.1-alpha is released
On Tue, May 19, 2009 at 10:46:03PM +0100, James Mansion wrote: Nick Mathewson wrote: Good morning, evening, night, or afternoon! The first alpha release in the long-promised Libevent 2.0 series is finally out. You can download Libevent 2.0.1-alpha from http://monkey.org/~provos/libevent-2.0.1-alpha.tar.gz Is there any ambition that this can be built on Win32 at the moment? Yes. With MinGW, it has built all along. With respect to building it with Visual C, please see my message to this very list from last Thursday: http://monkeymail.org/archives/libevent-users/2009-May/001650.html You may want to read the rest of the thread too, for context. There should be a 2.0.2-alpha sometime in the next week or two that should build better for you. If anybody wants to try it out before then to make sure it builds for you, you can as always check it out from svn with svn checkout https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent libevent hth, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Simple Echo server with libevent 2.0
On Mon, May 18, 2009 at 07:10:47PM +0200, q6Yr7e0o nIJDVMjC wrote: Hi For one example of how to use the 2.0.1-alpha API, you could have a look at chapter 1 of ? ? ?http://www.wangafu.net/~nickm/libevent-book/ . Would it be possible to link that book on http://monkey.org/~provos/libevent/ ? I've long googled for such reference material but didn't find any Sorry! We'll put it up eventually (probably in the next week or two). It hasn't been called an official thing yet because it is very much a work in progress. (For instance, I haven't tried to compile any of the examples in the reference section yet. Also, some of the stuff that it documents only exists in Subversion.) [...] I hope we can get Libevent 2.0.2-alpha released this week. ?Until then, you can use subversion to fetch the latest code from the repository at ? https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent One think I don't understand about buffer events (I haven't read the source yet): If i call struct evbuffer *input = bufferevent_get_input(bev); and then read the contents of input by calling bufferevent_read, is there a way to not remove the data from the input buffer? First, you probably wouldn't grab the input evbuffer if you were using bufferevent_read(): bufferevent_read(be, b, n) is just an alias for evbuffer_remove(bufferevent_get_input(be), b, n). In 2.0.1-alpha, you can see the N bytes at the start of an evbuffer without removing them by calling evbuffer_pullup(). It isn't very efficient, though: see the documentation. The 2.0.2-alpha release will add an evbuffer_peek function to see the bytes in the evbuffer without copying them out, and without rearranging the internals of the evbuffer. If you're trying to reduce byte copies, it may do what you want. Perhaps I just want to look at it if it's enought to parse and if not wait for another call of my echo_read_cb? Or do I have to read the input buffer into another buffer to preserve it? If I have to read/write the buffers into each other, are the contents copied each time? If you use evbuffer_add_buffer(), the contents aren't copied. If you use evbuffer_remove_buffer(), the contents are only copied to the extent they do not fall cleanly on a chunk boundary. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Libevent 2.0.1-alpha is released
On Tue, May 19, 2009 at 06:03:18PM -0400, Nick Mathewson wrote: [...] With respect to building it with Visual C, please see my message to this very list from last Thursday: http://monkeymail.org/archives/libevent-users/2009-May/001650.html So reading that message again, I guess I wasn't as explicit as I should have been. Short version: yes, I think the next release will build okay with visual C. But somebody besides me test out the code in the subversion repository to make sure there aren't any surprises. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Simple Echo server with libevent 2.0
On Mon, May 18, 2009 at 05:26:04PM +0200, q6Yr7e0o nIJDVMjC wrote:
Hi,
can somebody guide me how to write a simple echo server using libevent 2.0?
I've read into the sources of libevent (especially http.c) because i wanted
to know how to bind to a socket, accept new connections and read/write async
to the network.
But there are so many interal functions (like bind_socket bind_socket_ai)
which i have to copy (?) in order to bind to a socket that i think i got
something wrong.
I wouldn't look to http.c for examples; it was originally written for
a much older version of the API, and though it has been updated since
then, there are still a fair number of new interfaces that it isn't
using yet.
For one example of how to use the 2.0.1-alpha API, you could have a
look at chapter 1 of
http://www.wangafu.net/~nickm/libevent-book/ .
That's still a pretty ungainly way to do a server, though. In
2.0.2-alpha, there will be a new evconnlistener API to wrap most of
the busywork of listening for new connections. I've attacked a quick
echo server written with it to this email. [It works for me on Linux;
haven't tested it anywhere else.]
I hope we can get Libevent 2.0.2-alpha released this week. Until
then, you can use subversion to fetch the latest code from the
repository at
https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent
yrs,
--
Nick
#include event2/listener.h
#include event2/bufferevent.h
#include event2/buffer.h
#include arpa/inet.h
#include string.h
#include stdlib.h
#include stdio.h
static void
echo_read_cb(struct bufferevent *bev, void *ctx)
{
/* This callback is invoked when there is data to read on bev. */
struct evbuffer *input = bufferevent_get_input(bev);
struct evbuffer *output = bufferevent_get_output(bev);
/* Copy all the data from the input buffer to the output buffer. */
evbuffer_add_buffer(output, input);
}
static void
echo_event_cb(struct bufferevent *bev, short events, void *ctx)
{
if (events BEV_EVENT_ERROR)
perror(Error from bufferevent);
if (events (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
bufferevent_free(bev);
}
}
static void
accept_conn_cb(struct evconnlistener *listener,
evutil_socket_t fd, struct sockaddr *address, int socklen,
void *ctx)
{
/* We got a new connection! Set up a bufferevent for it. */
struct event_base *base = evconnlistener_get_base(listener);
struct bufferevent *bev = bufferevent_socket_new(
base, fd, BEV_OPT_CLOSE_ON_FREE);
bufferevent_setcb(bev, echo_read_cb, NULL, echo_event_cb, NULL);
bufferevent_enable(bev, EV_READ|EV_WRITE);
}
int
main(int argc, char **argv)
{
struct event_base *base;
struct evconnlistener *listener;
struct sockaddr_in sin;
int port = 9876;
if (argc 1) {
port = atoi(argv[1]);
}
if (port=0 || port65535) {
puts(Invalid port);
return 1;
}
base = event_base_new();
if (!base) {
puts(Couldn't open event base);
return 1;
}
/* Clear the sockaddr before using it, in case there are extra
* platform-specific fields that can mess us up. */
memset(sin, 0, sizeof(sin));
/* This is an INET address */
sin.sin_family = AF_INET;
/* Listen on 0.0.0.0 */
sin.sin_addr.s_addr = htonl(0);
/* Listen on the given port. */
sin.sin_port = htons(port);
listener = evconnlistener_new_bind(base, accept_conn_cb, NULL,
LEV_OPT_CLOSE_ON_FREE|LEV_OPT_REUSEABLE, -1,
(struct sockaddr*)sin, sizeof(sin));
if (!listener) {
perror(Couldn't create listener);
return 1;
}
event_base_dispatch(base);
return 0;
}
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Building with Visual Studio
On Thu, May 14, 2009 at 04:58:02PM -0700, Rush Manbert wrote: Hi, I have done some searching and have read a couple of posts on this subject, but nothing that quite answers my questions. I am writing a Windows port of the Thrift RPC library (http://incubator.apache.org/thrift/ ) that will build in Visual Studio with no Cygwin or other *nix/Posix emulation required. Thrift uses libevent, so I tried to build the same sort of version of libevent. I was surprised to find that libevent would not build out of the box on Windows using Visual Studio. However, once I found an open source version of stdint.h for use in Visual Studio, I was able to get the 1.4.10 stable version building fairly easily. Once the library built, I was able to build and successfully execute the regression test using the same technique. The other tests weren't so successful due to missing vcproj files. I just opened up a WinXP VM, fired up Visual C++ 2008 Express, checked out the 1.4 branch, and tried to build from the project file in Svn. It converted mostly okay. I ran into a few warnings when I built it, but no errors. It seems to have worked fine for me. (I fixed the warnings and checked the modified code back in.) Then I tried it with the distributed version. Ouch! Apparently, we have not been including the event-config.h file that is supposed to be in WIN32-Code for VC libevent builds... but we _have_ been including the generated event-config.h that shouldn't actually be distributed; only installed. I've cleaned up the code that builds the distribution. (It wasn't an absence of stdint.h; Libevent works around the fact that 10 years out, MS still hasn't seen fit to implement C99. But using the wrong event-config.h made libevent act as if it had stdint.h when it didn't, which was the problem.) I have two questions: 1) Why doesn't libevent as released just build in Visual Studio if this was so easy to accomplish for me? Or have I missed something? Apparently, nobody who knew how Libevent was _supposed_ to build with Visual Studio had tried building with Visual Studio using a source distribution for a long time. We suck at paying attention to Visual Studio. [How it's _supposed_ to work is this: on everywhere that has a shell that can handle autoconf, the autoconf script detects the system settings and generates a config.h file, which gets post-processed into a event-config.h automatically. On anywhere without a shell that can handle autoconf (this is basically only Visual Studio), we use a pre-generated event-config.h file that has the right settings for Windows on Visual Studio's idea of C.] 2) I would be happy to provide a patch that adds this support. Are any of the developers interested in receiving it? Give 1.4.11 a try when it comes out some time in the next 24 hours; let us know if it's any better for you? (Also, once 2.0.2-alpha is out, somebody needs to work on decent project files for the 2.0.x series. The ones we have now are kind of stupid and crufty.) apologies, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Libevent 2.0.1-alpha is released
On Fri, Apr 24, 2009 at 10:32:46AM -0700, Dan Kegel wrote: On Sat, Apr 18, 2009 at 11:24 AM, Nick Mathewson ni...@freehaven.net wrote: Libevent 2.0 is intended to be backward compatible with the Libevent 1.4 APIs[*]. ?Any program that worked with Libevent 1.4 should still work with Libevent 2.0, unless we screwed up. ?Please test your programs when you have a chance, so that if we _did_ screw up, we can notice soon. [*] Unless you were messing around with the internals of internal structures. Say, wouldn't this be a good time to remove the deprecated event_gotsig and event_sigcb interface? Valgrind just complained about a thread race there on me, so I'm going to remove them from my local copy... Good catch; done. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] epoll_wait problems
On Mon, Mar 23, 2009 at 02:23:37PM +0100, Richter, J??rg wrote:
Hi,
Not really libevent specific, but perhaps someone here has seen this before
and can help me with this one.
The problem is that epoll_wait hangs longer than requested.
This is a strace -t -T -e epoll_wait output of one such call:
12:07:08 epoll_wait(4, {{EPOLLIN, {u32=144477200, u64=144477200}}}, 1023,
2347024) = 1 4575.649567
Here you see that the call takes 4575 seconds, but epoll_wait was only
requested to wait 2347 seconds.
epoll_wait would have waited longer, but fd activity made it return. Without
fd activity it would have waited much longer.
This is reproducible.
Is there a maximum timeout epoll_wait can handle?
Perhaps you have any other ideas what this might be?
Sorry for the delay. There is indeed a wart in (most? all) versions
of the Linux epoll implementation where the timeout maxes out at
(LONG_MAX - 999UL) / HZ milliseconds. If your long is 32 bits, and
your HZ is 1000, this tops out at a little over 2147 msec for you.
Libevent versions later than 1.4.4-stable work around this.
--
Nick
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] epoll keyed wakeups - Patch merged on 2.6.30-rc1
On Wed, Apr 08, 2009 at 08:33:58AM -0700, Raine Fan wrote: Hi! I'm just curious if libevent 1.5/2.0 will be carrying this improvements (epoll keyed wakeups - see article on LWN.net: http://lwn.net/Articles/317489/) from epoll patch set that was merged recently on kernel 2.6.30-rc1 this week. Reading over the test code, I can't figure out if there is actually a new API here. If I understand correctly, we may be getting the benefits of this for free on any system with the new kernel code. Am I missing something that we'd need to do for this? -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] bug in min_heap
On Wed, Apr 15, 2009 at 08:00:39PM +0300, Marko Kreen wrote: [...] Well, I attached a draft of it, but its totally untested and the minheap code is not very parseable for me. So somebody with has better understanding of the code should review it. (Maxim?) Basic idea - if you replace an element in the middle of heap with last element, there is chance it needs to be pushed up not down. I've checked this in for now, but please see the comments on Sourceforge at https://sourceforge.net/tracker/index.php?func=detailaid=2778433group_id=50884atid=461324 I think that this might not actually be a bug; I can't seem to write a test case that provokes it. Have I missed something? Do you have a test case? yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] Re: Libevent 2.0.1-alpha is released
On Sat, Apr 18, 2009 at 02:24:15PM -0400, Nick Mathewson wrote: Good morning, evening, night, or afternoon! The first alpha release in the long-promised Libevent 2.0 series is finally out. You can download Libevent 2.0.1-alpha from http://monkey.org/~provos/libevent-2.0.1-alpha.tar.gz But when you do, you'll notice there's a missing header, defer-internal.h :( You can get a copy here for now: http://levent.svn.sourceforge.net/viewvc/levent/trunk/libevent/defer-internal.h?revision=1181 yrs apologetically, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] Libevent 2.0.1-alpha is released
Good morning, evening, night, or afternoon! The first alpha release in the long-promised Libevent 2.0 series is finally out. You can download Libevent 2.0.1-alpha from http://monkey.org/~provos/libevent-2.0.1-alpha.tar.gz To make sure that you're getting the real Libevent source and no other, you can fetch the GPG signature from http://monkey.org/~provos/libevent-2.0.1-alpha.tar.gz.sig This is an alpha release. Libevent 2.0 is not finished. There will be bugs, and we make no promises about the stability of any APIs introduced in this release. When you find bugs, please let us know. Libevent 2.0 is intended to be backward compatible with the Libevent 1.4 APIs[*]. Any program that worked with Libevent 1.4 should still work with Libevent 2.0, unless we screwed up. Please test your programs when you have a chance, so that if we _did_ screw up, we can notice soon. [*] Unless you were messing around with the internals of internal structures. The list of changes in 2.0 is too large to summarize concisely. My next email will describe what's there so far. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] What's new in Libevent 2.0
Here's the current draft What's new in Libevent 2.0 document. What's New In Libevent 2.0 so far: 1. About this document This document describes the key differences between Libevent 1.4 and Libevent 2.0, from a user's point of view. It was most recently updated based on features in subversion trunk as of 16 April 2009. NOTE 1: If any features or fixes get backported from trunk to 1.4, they should get moved from here into whatsnew-14.txt, since they will no longer be differences between 1.4 and this version. NOTE 2: We may have missed some things on this list. Caveat haxxor. 2. New and Improved Event APIs Many APIs are improved, refactored, or deprecated in Libevent 2.0. COMPATIBILITY: Nearly all existing code that worked with should Libevent 1.4 should still work correctly with Libevent 2.0. However, if you are writing new code, or if you want to port old code, we strongly recommend using the new APIs and avoiding deprecated APIs as much as possible. Binaries linked against Libevent 1.4 will need to be recompiled to link against Libevent 2.0. This is nothing new; we have never been good at preserving binary compatibility between releases. We'll try harder in the future, though: see 2.1 below. 2.1. New header layout for improved compatibility Libevent 2.0 has a new header layout to make it easier for programmers to write good, well-supported libevent code. The new headers are divided into three types. There are *regular headers*, like event2/event.h. These headers contain the functions that most programmers will want to use. There are *backward compatibility headers*, like event2/event_compat.h. These headers contain declarations for deprecated functions from older versions of Libevent. Documentation in these headers should suggest what's wrong with the old functions, and what functions you want to start using instead of the old ones. Some of these functions might be removed in a future release. New programs should generally not include these headers. Finally, there are *structure headers*, like event2/event_struct.h. These headers contain definitions of some structures that Libevent has historically exposed. Exposing them caused problems in the past, since programs that were compiled to work with one version of Libevent would often stop working with another version that changed the size or layout of some object. We've moving them into separate headers so that programmers can know that their code is not depending on any unstable aspect of the Libvent ABI. New programs should generally not include these headers unless they really know what they are doing, and are willing to rebuild their software whenever they want to link it against a new version of libevent. Functionality that once was located in event.h is now more subdivided. The core event logic is now in event2/event.h. The evbuffer functions for low-level buffer manipulation are in event2/buffer.h. The bufferevent functions for higher-level buffered IO are in event2/bufferevent.h. COMPATIBILITY: All of the old headers (event.h, evdns.h, evhttp.h, evrpc.h, and evutil.h) will continue to work by including the corresponding new headers. Old code should not be broken by this change. 2.2. New thread-safe, binary-compatibile APIs Some aspects of the historical Libevent API have encouraged non-threadsafe code, or forced code built against one version of Libevent to no longer build with another. The problems with now-deprecated APIs fell into two categories: 1) Dependence on the current event_base. In an application with multiple event_bases, Libevent previously had a notion of the current event_base. New events were linked to this base, and the caller needed to explicitly reattach them to another base. This was horribly error-prone. Functions like event_set that worked with the current event_base are now deprecated but still available (see 2.1). There are new functions like event_assign that take an explicit event_base argument when setting up a structure. Using these functions will help prevent errors in your applications, and to be more threadsafe. 2) Structure dependence. Applications needed to allocate 'struct event' themselves, since there was no function in Libevent to do it for them. But since the size and contents of struct event can change between libevent versions, this created binary-compatibility nightmares. All structures of this kind are now isolated in _struct.h header (see 2.1), and there are new allocate-and- initialize functions you can use instead of the old initialize-only functions. For example, instead of malloc and event_set, you can use event_new(). (For people who do
Re: [Libevent-users] accessing a bufferevent's evbuffer directly
On Wed, Apr 08, 2009 at 11:06:37AM -0700, Ka-Hing Cheung wrote: On Mon, 2009-04-06 at 11:17 -0700, Nick Mathewson wrote: If you want to try 0-copy stuff with bufferevents, you'll need to wait for Libevent 2.0. Libevent 1.4 doesn't really support that so well. Is there some kind of rough schedule on when libevent 2.0 may come out? I'd like to hope we'll get the first numbered beta release (probably called 2.0.1-beta or something) out within the next month or two. But I might be wrong on that one. If you want to see what's there so far, you can just check out the svn trunk. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] accessing a bufferevent's evbuffer directly
On Mon, Apr 06, 2009 at 01:38:44PM -0700, Phoenix Sol wrote: Thanks, Nick. Is zero-copy possible at all with the current libevent? (I already assumed it wouldn't work with bufferevent) If you mean, just using the regular event_base interface, without any bufferevent or evbuffer interfaces, then... sure, so long as your OS supports it, and supports it in a way that works with libevent. None of the functions that manipulate a 'struct event' (such as event_set, event_add, event_del) ever read or write any data to the network stack, so none of _them_ will add any copies to your data's path to and from the network stack. What you do to communicate with your kernel then is up to you. If you use read or write, data will be copied. If you use splice or sendfile, it typically won't. So if you could write a zero-copy program that used kqueue or epoll, you could write one using Libevent. Then again, I could be offbase here. I know libevent pretty well, but I'm not an expert on zero-copy stuff. Adrian -- are you still around? I bet this topic is right up your alley. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] busy waiting on AIX (timercmp bug)
On Mon, Mar 09, 2009 at 01:57:46PM +0100, Richter, J?rg wrote: [...] Note that the Linux definition has the same comment about the compare operator as the AIX one. But the comment seems to predate the Linux definition. I'd guess that the comment is there because, if you're writing a portable program, you need to make sure that you don't use timercmp with =/=, even if it _does_ happen to work on Linux. [...] I found no official documentation if timercmp should support = or =. I would suggest using always the libevent definition of timercmp, without ever using the system one. That's a good idea! We already did it back in 1.4.4, which was released last May. The latest stable libevent release is 1.4.9. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Multiple identical events on same fd in same loop
On Sat, Mar 07, 2009 at 01:45:21PM -0500, Thor Lancelot Simon wrote: [...] I see two ways to handle this. One is with very, very nasty application logic, and the other is to simply set two separate events with the SSL fd, each waiting on EV_WRITE. So when the SSL fd comes ready for write -- I'd hope -- both my pending SSL_read() and my pending SSL_write() would be invoked, each from a separate event's callback. The first way is what you have to do with Libevent 1.4.x and earlier; Libevent only supports one _pending_ event at a time per fd/operation pair. In other words, it's okay to have two events set to watch EV_WRITE on the same FD, but you can only ever add one at a time. The application logic to handle this isn't actually so bad; instead of having two events (one write, and one write in order to read), have just one write for something event, and have your per-fd data structure remember which underlying semantic events you actually meant. The Tor source code does this if you want to check out an open-source example of how it's possible; I'm sure there are other good projects to check out too. On the other hand, the development branch for Libevent 2.0 _does_ support multiple events per fd; if you're willing to use alpha code until Libevent 2.0 stabilizes, that might be one way to go. (I'd also like to get a bufferevent filter that handles openssl into Libevent 2.0, but the implementation for that one isn't done yet.) yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] bug in epoll.c ?
On Fri, Mar 06, 2009 at 10:20:02AM +0100, Alon Noy wrote: When I'm running my libevent based application under valgrind I'm getting messages about memory access violations in epoll.c. when changing line 125 from: nfiles = rl.rlim_cur - 1; to: nfiles = rl.rlim_cur; then the application runs with no complains from valgrind. The problems shows when reaching the limit of allowed open file descriptors. Hm! I suppose there's nothing necessarily wrong with omitting the -1 on non-Solaris platforms, if the goal is to be able to run right up to rlim_cur active fds. Out of curiosity, what was the valgrind error you were seeing? From the remark above this line I understand that the -1 was hacked in because some issue with Solaris but was it ever tested with other platforms? Yes. I develop on Linux and OS X, and IIRC Niels is pretty eclectic about his OS choices. It would be neat to have a regression test for this case, if you have the time to write one. Yrs, -- Nick Mathewson ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] UDP and TCP Communications
On Fri, Mar 06, 2009 at 09:16:26AM -0500, Brian wrote: Thanks Guillaume for the feedback. Unfortunately malloc/calloc method didn't work for me. I think I will just go back to how it was. Does anyone know if there is an inherent flaw, perhaps, with doing the event_init()/event_dispatch() within a posix thread? Guillaume is right. In the sample code you posted, the memory used to hold the struct events on the stack will not continue to be reserved after the function exits, so a call to another function can totally overwrite it. This isn't a pthreads thing. (To answer your threading question: in Libevent 1.4.x, it's okay to do nearly anything from inside a thread, but only one thread can safely use a given event_base at a time. Also, if you're using threads with Libevent 1.4.x, you need to make sure that you always specify event_base objects explicitly, and use event_base_set on each new event. Otherwise, calls to functions will generally use the most recently initialized event_base, leading to race conditions.) -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Please check svn trunk before you write feature patches [was Re: [Libevent-users] [PATCH] Add callback for raw read/write]
On Thu, Feb 19, 2009 at 08:09:18AM -0500, jamal wrote: olla, I dont know what the process for submitting patches here, but this is against version 1.4.9. This makes it easier to integrate libraries that provide their own read/write functions (such as openssl). Hi, Jamal! Thanks for the patch, but the Libevent 2 dev series (currently calling itself 1.4.99) handles evbuffers and bufferevents significantly differently. (Old code still works, but the implementation is more efficient.) We're trying to keep the 1.4.x series as stable as possible, and put new features into 2.0. This reminder goes for everybody writing patches: if you have a feature patch (not a bugfix) that you want to go into the next version of Libevent, please make sure your patch is against the trunk of the subversion repository, not against the latest stable release. (Instructions for getting Libevent from subversion are here: http://sourceforge.net/svn/?group_id=50884 The URL you want to check out is https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent ) (As for the actual patch, it looks a lot more like the kind of problem we'd try to solve in 2.0 with a specialized bufferevent implementation.) yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] About what's coming to libevent 1.5/2.0
On Sat, Jan 17, 2009 at 01:11:17PM -0800, Raine Fan wrote: [...] What make me post here to ask if it's possible to you (libevent staff) to share what it's coming (features and theorical-release-date) on the next release (dunno if will be called 1.5 or 2.0). The easiest way to see what's been done for 2.0 is to check out svn trunk at https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent There's a pretty extensive changelog there already, though there's more stuff planned. I've been reading your mailing lists a long time ago and following all threads about multithreaded questions like having an acceptor thread which delivers fd's to another threads to process request. Basically my project design will be based on a pool of threads waiting to events on fd's 'listened' on one or more acceptor threads. I found this explanation posted on your home page (http://monkeymail.org/archives/libevent-users/2007-January/000450.html) about multithreaded use of libevent, but pipe() is not the way I though in passing messages from one thread to another, even the fd's from acceptors to the thread-pool. Do you think that in the next release 1.5/2.0 this behavior should be well implemented, instead of using libev features any time soon? _Which_ behavior exactly? I can't tell for sure from your write-up what feature it is that you want. Are you talking about the libev - fast intra-thread communication between multiple event loops (with optional fast linux eventfd backend). thing? Looking at the source, it seems that it does use pipe everywhere except on Linux. Is it the _abstraction_ here that you had in mind; the ability to wake up a looping event base from another thread in a safe, generic, and platform-optimized way? Something like that would indeed be cool for 2.0. Heck, I think I'll go hack up the basic implementation now. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Access struct event callback argument
On Fri, Dec 12, 2008 at 08:54:03PM -0200, Rodrigo Campos wrote: On Wed, Dec 3, 2008 at 5:15 PM, Rodrigo Campos rodrig...@gmail.com wrote: On Wed, Dec 3, 2008 at 4:46 PM, Niels Provos pro...@gmail.com wrote: On Wed, Dec 3, 2008 at 10:28 AM, Rodrigo Campos rodrig...@gmail.com wrote: I wanted to know how public is the struct event defined in event.h. I mean, is it part of the interface/api ? We are moving away from making structures accessible. If you want to change the call back argument, you could delete the event and re set it. I dont want to change it. I want to free it :) Sorry to ask again, but if you are moving away from making structures accessible, do you plan to provide some funtion to get them ? (I'm particulary intrested in the ev_arg) So, here's the plan. In libevent 2.0, we're splitting most headers into three separate headers, and keeping the old header names for backward compatibility. For example, if you #include event.h, you'll get all the functions and structures that you get now, and most of your code should still work. If you include event2/event.h, you'll only get the non-deprecated functions. If you include event2/event_compat.h, you get the deprecated functions for backward compatibility. If you include event2/event_struct.h, you also get the structure declarations. If you do this, we explicitly disclaim promises of forward-compatibility: structure layout will not change gratuitously, but it is likely to change between versions. If there's something that you need to get at, or free, or whatever, and you need to be forward-compatible about it, the right solution is probably to participate in libevent development, and make sure that there is a function declared in event2/event.h (for example) to access whatever the field is. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] unable to read from stdin on os x leopard
On Tue, Dec 09, 2008 at 11:43:32AM +0100, Rapsey wrote: This program works normally on ubuntu, but on os x leopard it immediately exits with errno: Operation not supported. Why does it not work on os x? Short version: You're using stdin, and it looks like the backend libevent uses on leopard doesn't support non-socket fds. Libevent uses underlying nonblocking IO mechanisms the platform gives it. Some of these work well with non-socket file descriptors; some don't. By default, libevent uses the fastest (best-scaling) backend that it knows about for your platform, even if that backend doesn't support all kinds of fd. In the current svn trunk (which will eventually become libevent 2.0), there's a feature to let you specify that you want a backend that works with file descriptors, even if it doesn't scale well. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] files and libevent
On Thu, Jun 12, 2008 at 07:59:45AM -0700, Teunis Peters wrote: I've not found any code to work with - but is there any reason that libevent would not work with standard files? I keep getting permission denied more or less: fd = open(filename, O_RDONLY) event_set(ev, EV_READ | EV_PERSIST, rd_callback, rd_data); event_add(ev, NULL) - EPERM Or does libevent only work with network connections? Libevent uses underlying nonblocking IO mechanisms the platform gives it. Some of these work well with non-socket file descriptors; some don't. By default, libevent uses the fastest (best-scaling) backend that it knows about for your platform, even if that backend doesn't support all fds. In the current svn trunk (which will eventually become libevent 2.0), there's a feature to let you specify that you want a backend that works with file descriptors, even if it doesn't scale well. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [err] event_queue_insert: 0xb6a4517c(fd 17) already on queue 2
On Fri, Jun 13, 2008 at 10:01:33AM +0200, Ron Arts wrote:
Hello,
I get this error on libevent 1.3e on CentOS5. Upgrading to
a higher version is not easy to push through, to say the least,
so that's why I am asking the list first.
The error message looks like an event that's already been added to a
queue is getting added again, and that's not supposed to happen.
queue 2 is EVLIST_INSERTED, the list of all events that have been
event_add()ed for IO events and not event_del()d.
The double insert is pretty weird, though, since event_add() is pretty
good about not adding an event that's already been added. The code
where it puts an event into the EVLIST_INSERTED queue is (in 1.3-svn):
if ((ev-ev_events (EV_READ|EV_WRITE))
!(ev-ev_flags (EVLIST_INSERTED|EVLIST_ACTIVE))) {
event_queue_insert(base, ev, EVLIST_INSERTED);
return (evsel-add(evbase, ev));
}
so unless I can't read C this morning, it should only add the event to
the queue if it's already not-inserted and not-active.
In other words, something fishy is afoot here. Having a stack trace
and the contents of the offending struct event might help track it down.
yrs,
--
Nick
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent on windows: select() fails
On Tue, Jun 03, 2008 at 07:08:57PM +0200, Shiqing Fan wrote: Hi all, I'm testing Libevent 1.3 with VS2005 on Windows. The same problem below there happens on my machine, select() always returns -1. Does anyone know the reason? Sorry for polling this old mail up, I just found someone has asked the same question that I got, but no answer. Thanks a lot. Sometimes this means that you haven't called WSAStartup ? Winsock wants to be initialized. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Integrate Windows I/O Completion Port into Libevent
On Sun, Jun 01, 2008 at 01:51:53PM +0800, liusifan wrote: [...] Integrate Windows I/O Completion Port into Libevent IOCP is true async i/o model, and libevent is event-driven i/o model. 1. How to emulate event-driven recv/send ? It is possible to use IOCP to emulate event-driven mode. When you make read or write calls, pass in a zero-byte buffer (ie, you can manage your i/o buffers using event driven semantics). So if you issue a read passing in a zero-byte length buffer, your will get notified when there is data to read. This seems pretty promising, and could wind up being a better short-term idea than our older plan to get IOCP support by making it a new backend for bufferevents. How well tested is this? Can you get the unit tests to pass, at least, to the extent that the unit tests in 1.4.4 currently pass on win32? I'd like people who use libevent on windows to try this code out; if it works well, we should integrate it. A useful but not-immediately-necessary followup benchmark would be to see how well this does against using IOCP in the normal (not event-driven) way. If there's no big difference, we can defer bufferevents IOCP work for a while; but if there's still a significant amount of overhead here, we should try to apply this patch *and* get IOCP working with bufferevents. [...] 6. Source code http://spserver.googlecode.com/files/libevent-1.4.4-iocp.zip diff file : libevent-1.4.4-iocp\libevent-iocp\diff.txt (For future reference, please use unified diffs (generated with diff -u) rather than old-style diffs. They're shorter and (usually) easier to read. This diff is small enough that there's not much worry about this one, though.) add file : libevent-1.4.4-iocp\WIN32-Code\win32iocp.cpp You mean .c? Hm. This might be better applied against trunk (which will eventually become libevent 2.0) than against 1.4.4; the 1.4 series is supposed to be stable, and Big New Features are usually out there... ...but now that I look at the code, it looks like the only big change is to add a new backend file, which (as far as I can tell) doesn't require any big changes elsewhere in the code. I'd be curious to see what Niels thinks here, but I wouldn't be opposed to checking it into the stable branch. BTW: I haven't read the code fully yet, but it looks relatively clean and straightforward. I'd appreciate it if people who know Windows networking better than I do would check it out and comment. Finally: Thanks, Stephen! This looks like great work to me so far. peace, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Support for Edge-Triggered behaviour
On Fri, May 30, 2008 at 10:18:24AM +0200, Valery Kholodkov wrote: Below is the new version of file test_et.c. The resulting executable should return 1 whenever ET works, 0 otherwise. Applied to trunk; thanks! -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Support for Edge-Triggered behaviour
On Thu, May 29, 2008 at 05:40:16PM +0200, Valery Kholodkov wrote: Greetings! Since discovering libevent for myself I've been wondering why where is still no support for Edge-Triggered behaviour, which from my point of view could be easily implemented. Basically, because nobody had written it yet. Thanks for writing it! :) [...] The patch itself is submitted into SF's patch tracking system: http://sourceforge.net/tracker/index.php?func=detailaid=1968284group_id=50884atid=461324 Thanks for reminding me; I've just added a few questions there. This patch introduces EV_ET flag. Whenever you specify EV_ET in event_set call a specific module will try enforce Edge-Triggered behaviour. Well, not exactly. The way the patch is written now, when EV_ET is set, *and* you're using the epoll backend or the kqueue backend, you'll get edge-triggered behavior. Otherwise, you get regular behavior. I'm not sure whether this is the right API or not; it might make more sense for event_add to give an error when you try to add an EV_ET event to a backend that doesn't support EV_ET. Alternatively, we could define the semantics of EV_ET so that instead of meaning make this event edge triggered it means make this event edge triggered if possible. (This is one of the comments on the sourceforge patch; I'm hoping we can discuss this more there. Niels: what do you think?) [...] I hope that maintainers will agree to integrate the patch in future versions of libevent. I'd definitely like to merge functionality like this into Libevent 2.0. The code looks clear enough, and the functionality certainly seems valuable enough. -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Support for Edge-Triggered behaviour
On Thu, May 29, 2008 at 08:02:18PM +0200, Valery Kholodkov wrote: For the convenience I'll answer to Nick's questions from SF's patch tracker in this list. A few initial questions: - How exactly does the test_et.c file test the edge-triggered behavior? As near as I can tell, the test ought to pass whether EV_ET works or not. What am I missing? I hardly find a direct and automatic way to do it... Try following patch for test_et.c [...] You should get following now: $ ./test_et read_cb: etread 1 ^C *HANGING!!!* $ EVENT_NOKQUEUE=1 EVENT_NOEPOLL=1 ./test_et read_cb: read 1 read_cb: read 1 Ah. In the edge-triggered case, the handler only gets called once, but in the level-triggered case, the handler is called so long as read() would still succeed. [...] I must think a bit how to automate this. Please give me time. Maybe you could use the EVLOOP_NONBLOCK flag to event_loop(), and call it twice, and verify that in the edge-triggered case the event is only activated on the first call, but in the level-triggered case the event is activated on the second call too? yours, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Success compile libevent 1.4.3 with VC6
On Wed, Apr 23, 2008 at 07:37:32PM -0400, Nick Mathewson wrote: On Wed, Apr 23, 2008 at 05:34:04PM +0400, Eugene 'HMage' Bujak wrote: [...] * Winsock library needs to be initialized and freed explicitly on win32. This is true, but it's not libevent's job to do it. Actually, I think I was wrong here. As James Mansion pointed out, it's okay to call WSAStartup more than once. And given that I just spent 30 minutes trying to track down a bug that was really just forgetting to call WSAStartup() from inside a test in a configure script, I've become pretty sure that it is a useful thing to do. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] patch for realtime signal events in libevent-1.4.3-stable
On Mon, May 12, 2008 at 05:39:51PM -0400, Phil Budne wrote: I needed to do apples-to-apples comparison between rtsignals and epoll for a client, so I fixed rtsig.c from 1.3e (see an earlier post) to compile, then fixed it to work, and ported that to 1.4.3-stable. NOTE WELL! Used only for benchmarking: the rtsig_dealloc routine may not free everything it needs to! Out of curiousity, how did the benchmarks turn out? Any reason to try to get rtsig working again in the mainline distribution? -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Patch: useful sendfile for HTTP servers (1.4.3-stable)
On Wed, May 07, 2008 at 11:47:09PM -0700, Niels Provos wrote: Hi Manual, this is a good suggestion. Nick and I are currently working on how buffers and http work in libevent 2.0. You might want to check out trunk to see some of the progress there. In any case, it seems that your sendfile changes would be a good fit there. BTW, sendfile is available on a large number of platforms now. In fact, this fits pretty well into the newer evbuffer implementation. Whereas the old implementation used a big chunk of memory for the entire buffer, the new code uses a linked list of small chunks. (This removes the need for big memmov operations, and generally makes writing big files more efficient.) All we'd need to do to support sendfile is add another chunk type whose contents were a file rather than a chunk of ram. We'd probably want at least two ways to add a file at this point in the stream: one taking a filename and one taking an fd. Of course, we'd need to make sure to fall back on mmap() for systems lacking sendfile(), and maybe on a series of read() operations on systems lacking mmap(). yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] FYI: libevent/libev benchmark updated for libevent-1.4.3
On Thu, May 08, 2008 at 05:05:20PM -0500, Dan wrote: Its to bad when people write emails like this, clearly no social skills. Dan, please take this off-list. This kind of flamewar does not help libevent, libev, or anybody. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Success compile libevent 1.4.3 with VC6
On Sun, Apr 27, 2008 at 02:01:44PM +0100, James Mansion wrote: Also, I think this change will double-initialize winsock on all programs that use multiple event bases, and double-shutdown winsock whenever the bases are closed on those programs. From the MSDN docs: An application can call *WSAStartup* more than once if it needs to obtain the *WSADATA* structure information more than once. On each such call, the application can specify any version number supported by the Winsock DLL. An application must call the *WSACleanup* function for every successful time the *WSAStartup* function is called. Ah! Never mind that part of my message then. If this is so, I see no reason not to add a WSAStartup call. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Success compile libevent 1.4.3 with VC6
On Wed, Apr 23, 2008 at 05:34:04PM +0400, Eugene 'HMage' Bujak wrote:
Hi,
I want to submit a more complete patch for VC6.
This one makes all elements of the code compilable and this code is
being used on a production-level project at work.
The patch is in attachment. Compiles and works under VC6.
* signal_test works.
* timer_test works.
* event_test fails due to the reason that win32 select() can't work on
anything but network sockets.
Thanks for the patch; it's good stuff! I'd like to apply it, but
there are a few small issues or things that could be done better in
order to work well on *non*-vc6 projects.
* INPUT and OUTPUT are defined in platform SDK, so I had to redefine
them as EVRPC_INPUT and EVRPC_OUTPUT.
We can't do this change in mainline libevent; it'll break all existing
evprc users that use the old names. Maybe there's some way we we can
make the old names available on non-vc code, but deprecated in favor
of the EVRPC names? something like this:
enum EVRPC_HOOK_TYPE {
EVRPC_INPUT,
EVPRC_OUTPUT,
};
#ifndef _WIN32
#define INPUT EVRPC_INPUT
#define OUTPUT EVRPC_OUTPUT
#endif
[...]
* Winsock library needs to be initialized and freed explicitly on win32.
This is true, but it's not libevent's job to do it. If we make this
change so that libevent starts initializing winsock, we'll break all
existing programs that use libevent and know to initialize winsock
themselves. It's quite reasonable to call some sockets code before
calling event_init, but this part of the patch makes that result in
double-initializing winsock.
Also, I think this change will double-initialize winsock on all
programs that use multiple event bases, and double-shutdown winsock
whenever the bases are closed on those programs.
[...]
-/* Define to appropriate substitue if compiler doesnt have __func__ */
-#if defined(_MSC_VER) _MSC_VER 1300
+/* Define to appropriate substitute if compiler doesn't have __func__ */
+#ifdef _MSC_VER
#define __func__ ??
I thought some versions of VC *did* have an acceptable version of
__func__ or a replacement for it. If that's true, this will break
those other VCs. What is _MSC_VER defined to on VC6? Should the test
be something other than _MSC_VER 1300?
diff -ur libevent-1.4.3-stable/event-config.h
libevent-1.4.3-stable-win32/event-config.h
--- libevent-1.4.3-stable/event-config.h Wed Apr 23 17:31:14 2008
+++ libevent-1.4.3-stable-win32/event-config.hWed Apr 23 17:31:01 2008
@@ -51,7 +51,9 @@
#define _EVENT_HAVE_INET_NTOP 1
/* Define to 1 if you have the inttypes.h header file. */
+#ifndef _MSC_VER // MSVC doesn't have that
#define _EVENT_HAVE_INTTYPES_H 1
+#endif
This approach is the wrong way to make event-config.h work. On all
toolsets besides VC, the file is automatically generated from
config.h, which itself is automatically generated by the autoconf
script. The right approach for VC is just to have a separate config.h
and event-config.h script that work, ideally in the WIN32-Code
directory, and set up the compiler's search path so those get used
instead of any other ones that might be kicking around.
[...]
+#ifdef _WIN32 // MSVC doesn't have that
FWIW, many older C compilers don't allow C++-style comments: they
weren't added to the C standard till the C99 standard. We need
libevent to build on older compilers, so we can't add C++-style
comments to libevent.
Otherwise, though, this looks like solid work, and I'm quite glad that
somebody has a copy of VC6 and the willingness to do it! If you can
send in a revised patch, that'd be great. Otherwise, I'll try to
hand-apply it with modifications as noted above, but since my x86 box
is down at the moment, I can't fire up windows to test it out now, and
things might go badly.
many thanks,
--
Nick Mathewson
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent and autotools
On Tue, Apr 22, 2008 at 11:00:29PM -0300, Daniel Morales wrote: Hi all, Since libevent doesn't have a *.pc file to use with pkg-config, i need to add some manual check to a configure.in. Hm! pkg-config! Yes, that's a good idea; we should support that in libevent 2.0 and later. Would you like to write a patch to libevent to add pkg-config support? Any example for libevent to share? :) You could just search for a header called event.h that defines event_init, and flags like -levent-core that defines event_init. If you want to abuse autoconf and look in a bunch of likely places for the headers and libraries, you could try to see what Tor does, but I'm afraid it's a little baroque. Look in https://www.torproject.org/svn/trunk/configure.in for mention of libevent, and in https://www.torproject.org/svn/trunk/acinclude.m4 for a definition of the TOR_SEARCH_LIBRARY macro. (Yes, this is an ugly and underdocumented hack. Sorry. All of it seemed important at the time.) ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [patch] replace linear search in evdns with hashed list and add support of listening on unix sockets for evhttp
On Mon, Feb 18, 2008 at 06:58:32PM +0300, Vsevolod Stakhov wrote:
[...]
I've made some modifications to patch to attract your wishes, thought
I've not tested it in production yet.
Sorry, attached reversed diff.
Hi, and sorry about the delay! Now that libevent 1.4 is stable, I'm
able to spend more time getting more patches into the next series. I
did a match to make evdns.c threadsafe, but after I did so your patch
no longer applied cleanly. I've done an updated version of your patch
and attached it here; could you test it out and let me know if it
works for you and does what you want under load?
You'll need to apply it to svn trunk; it won't work with 1.4.
I'm still curious whether evdns_transmit() shows up on your profiles:
it's the only remaining common case where we walk the list of all
inflight requests. As you said before, your patch shouldn't make it
any slower, but I'm curious whether it appears slow for you at all.
If so, there should be another list of requests for which transmit_me
is set, so we don't need to walk the whole list of inflight requests
just in order to retry those.
yrs,
--
Nick
=== evdns.c
==
--- evdns.c (revision 15216)
+++ evdns.c (local)
@@ -293,8 +293,16 @@
};
struct evdns_base {
- struct request *req_head, *req_waiting_head;
+ /* An array of n_req_heads circular lists for inflight requests.
+ * Each inflight request req is in req_heads[req-trans_id % n_req_heads].
+ */
+ struct request **req_heads;
+ /* A circular list of requests that we're waiting to send, but haven't
+ * sent yet because there are too many requests inflight */
+ struct request *req_waiting_head;
+ /* A circular list of nameservers. */
struct nameserver *server_head;
+ int n_req_heads;
struct event_base *event_base;
@@ -330,6 +338,8 @@
((struct server_request*) \
(((char*)(base_ptr) - OFFSET_OF(struct server_request, base
+#define REQ_HEAD(base, id) ((base)-req_heads[id % (base)-n_req_heads])
+
/* These are the timeout values for nameservers. If we find a nameserver is down */
/* we try to probe it at intervals as given below. Values are in seconds. */
static const struct timeval global_nameserver_timeouts[] = {{10, 0}, {60, 0}, {300, 0}, {900, 0}, {3600, 0}};
@@ -453,7 +463,8 @@
/* failure */
static struct request *
request_find_from_trans_id(struct evdns_base *base, u16 trans_id) {
- struct request *req = base-req_head, *const started_at = base-req_head;
+ struct request *req = REQ_HEAD(base, trans_id);
+ struct request *const started_at = req;
if (req) {
do {
@@ -511,6 +522,7 @@
nameserver_failed(struct nameserver *const ns, const char *msg) {
struct request *req, *started_at;
struct evdns_base *base = ns-base;
+ int i;
/* if this nameserver has already been marked as failed */
/* then don't do anything */
if (!ns-state) return;
@@ -544,16 +556,18 @@
/* trying to reassign requests to one */
if (!base-global_good_nameservers) return;
- req = started_at = base-req_head;
- if (req) {
- do {
- if (req-tx_count == 0 req-ns == ns) {
-/* still waiting to go out, can be moved */
-/* to another server */
-req-ns = nameserver_pick(base);
- }
- req = req-next;
- } while (req != started_at);
+ for (i = 0; i base-n_req_heads; ++i) {
+ req = started_at = base-req_heads[i];
+ if (req) {
+ do {
+if (req-tx_count == 0 req-ns == ns) {
+ /* still waiting to go out, can be moved */
+ /* to another server */
+ req-ns = nameserver_pick(base);
+}
+req = req-next;
+ } while (req != started_at);
+ }
}
}
@@ -665,7 +679,7 @@
req-ns = nameserver_pick(base);
request_trans_id_set(req, transaction_id_pick(base));
- evdns_request_insert(req, base-req_head);
+ evdns_request_insert(req, REQ_HEAD(base, req-trans_id));
evdns_request_transmit(req);
evdns_transmit(base);
}
@@ -757,19 +771,19 @@
/* a new request was issued so this request is finished and */
/* the user callback will be made when that request (or a */
/* child of it) finishes. */
-request_finished(req, req-base-req_head);
+request_finished(req, REQ_HEAD(req-base, req-trans_id));
return;
}
}
/* all else failed. Pass the failure up */
reply_callback(req, 0, error, NULL);
- request_finished(req, req-base-req_head);
+ request_finished(req, REQ_HEAD(req-base, req-trans_id));
} else {
/* all ok, tell the user */
reply_callback(req, ttl, 0, reply);
nameserver_up(req-ns);
- request_finished(req, req-base-req_head);
+ request_finished(req, REQ_HEAD(req-base, req-trans_id));
}
}
@@ -1106,20 +1120,12 @@
static u16
transaction_id_pick(struct evdns_base *base) {
for (;;) {
- const struct request *req, *started_at;
u16 trans_id = trans_id_function();
if (trans_id == 0x) continue;
/* now check to see if that id is already inflight */
- req = started_at = base-req_head;
- if (req) {
- do
[Libevent-users] ANN: Libevent 1.4.3-stable released
Hello, everyone! It's finally time to release a stable libevent again. Libevent 1.4.3-stable is now tagged and ready to go. You can get the source here: http://monkey.org/~provos/libevent-1.4.3-stable.tar.gz There are a few bug fixes since 1.4.2-rc. Here's a summary of what's changed since the last rc in this series: - Kqueue fixes - Make event_reinit() work properly with kqueue. - HTTP fixes: - We now include Content-Length when replying to HTTP/1.0 requests with keep-alive. - More functions in http.c are now threadsafe. - DNS fixes: - Stop deleting uninitialized timeout events. See the changelog for full details. This is only what's new since the last release candidate, 1.4.2-rc. The 1.4.x series as a whole has lots of new features since the 1.3 series. I'll send a copy of the What's New document to the list in my next message. Many thanks to everybody who sent in code, including Christopher Layne, Scott Lamb, and many others. Key bug reporters include Charles Kerr, Ilya Martynov, and others on sourceforge. To report a bug, make a feature request, or submit code, you can use our sourceforge interface here: https://sourceforge.net/projects/levent many thanks, -- Nick Mathewson ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
[Libevent-users] What's New in Libevent 1.4
What's New In Libevent 1.4: 0. About this document This document describes the key differences between Libevent 1.3 and Libevent 1.4, from a user's point of view. It was most recently updated based on features from libevent 1.4.2-rc. 1. Packaging Issues. 1.1. The great library division. The libevent source now builds two libraries: libevent_core and libevent_extra. The libevent_core library includes event loops, timers, buffer code, and various small compatibility functions. The libevent_extra library includes code for HTTP, DNS, RPC, and so on. Thus, if you're writing software that only uses libevent's event loop, you should link against only the libevent_core library, whereas if you're writing software that uses libevent's protocol support as well, you need to link libevent_extra as well. For backward compatibility, libevent also builds a library called libevent that includes everything. 1.2. The event-config.h header Libevent configure script now builds two headers from its configure script: config.h (which it uses internally) and event-config.h (which it installs as a header file). All of the macros in event-config.h are modified so that they're safe to include in other projects. This allows libevent's header files (like event.h and evutil.h) information about platform configuration. What does this mean for you? As of 1.4.x, it should never be necessary to include extra files or define extra types before you include event.h (or any other libevent header); event.h can now look at the information in event-config.h and figure out what it needs to include. 1.3. Documentation Libevent now includes better doxygen documentation. It's not perfect or complete, though; if you find a mistake, please let us know. 1.4. Libtool usage We now use libtool's library versioning support correctly. If we don't mess this up, it means that binaries linked against old version of libevent should continue working when we make changes to libevent that don't break backward compatibility. 1.5. Portability Libevent now builds with MSVC again. We've only tested it with MSVC 2005, and the project files might not be right. Please let us know if you run into any issues. Libevent now builds on platforms where /bin/sh is not bash. Libevent's regression test no longer requires Python to be installed. 2. New and Improved APIs: (This list includes functions that are new, functions whose behavior has changed, and functions that were included in previous releases but which never actually worked before.) 2.1. Utility functions are defined in evutil.h Libevent now exposes a small set of functions for cross-platform network programming in evutil.h, on the theory that they've been useful enough to us that other people may likely want to use them too. These are mainly workarounds for Windows issues for now: they include evutil_socketpair (to fake socketpair on platforms that don't have it) and evutil_make_socket_nonblocking (to make a socket nonblocking in a cross-platform way. See the header for more information. 2.2. In the libevent core. The event_base_free() function now works. Previously, it would crash with an assertion failure if there were events pending on a base. Now, it simply deletes all the pending events and frees the base. Be careful -- this might leak fds and memory associated with the old events. To avoid leaks, you should still remove all the events and free their resources before you delete the base. Libevent should now work properly with fork(). Just call event_reinit() on your event base after the fork call, and it should work okay. Please let us know about any bugs you find. There's a new event_base_new() function that acts just like event_init(), but does not replace the default base. If you are using multiple event bases in your code, you should just use event_base_new() instead of event_init(), to avoid accidental bugs. There's new event_loopbreak() function to make a current event loop stop exiting and return. Unlike event_loopexit, it stops subsequent pending events from getting executed. This behavior is useful for scripting languages to implement exceptions from inside callbacks. There's a new event_base_get_method() function, for use in place of event_get_method() in multi-base applications. 2.3. New in HTTP. There's an evhttp_connection_set_local_address() function you can use to set the local address of an HTTP connection. HTTP/1.1 chunking now correctly ends chunks with '\r\n'. 2.4. New in DNS Instead of picking your method for generating DNS transaction IDs at startup, you can use evdns_set_transaction_id() to provide a transaction ID function at runtime. The class field in evdns_server_request is now renamed to dns_question_class, so that it won't break compilation under C++. This uses some preprocessor hacks
Re: [Libevent-users] dns server availability check / google.com
On Wed, Feb 27, 2008 at 07:17:50AM +0100, Florian Lohoff wrote: Hi, while looking into an async dns solution i found the code to recheck dns servers in libevent. I dont think checking for google.com is a very good choice. In the past even ccTLDs have disappeared or start to disappear (.su, .dd, .yu) despite companys renaming, disappearing or having hicups. Actually, looking for google.com isn't a problem here, if I understand correctly. Look at the code in nameserver_probe_callback: it accepts either DNS_ERR_NONE _or_ DNS_ERR_NOTEXIST as an answer meaning that the nameserver is running. In other words, a nameserver that says there is no google is considered just as up as a nameserver that reports what we'd currently expect to see. Does that seem like a sensible argument, or am I missing something? yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] thread-safety bug report with patch
On Tue, Feb 26, 2008 at 10:12:08AM -0800, Tani Hosokawa wrote: These are four broken functions that didn't make it into the latest release -- I noticed that one of them is even marked as not being thread-safe. Thanks! I've checked your patch into trunk and into the 1.4 branch. ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent (svn revision 651) segfaults on linux 2.6.12, ARM little endian architecture
On Fri, Feb 22, 2008 at 10:54:57AM +0100, Frank Schaeckermann wrote: I have installed transmission-daemon (a Bittorrent client) svn revision 5051 on my Maxtor Shared Storage II (ARM little endian) with Linux 2.6.12. Very soon after starting the daemon it segfaults. Hi, Frank! Here are some things to try: Does this also happen when you set the EVENT_NOEPOLL environment variable to disable the epoll method? Can you post a backtrace from gdb for this? Does this also happen with libevent 1.3e, or with the libevent 1.4 branch? Unfortunately, I don't have much way to tell whether this is a libevent bug or a transmission-daemon bug right now, but it would be great to help get it fixed either way. yrs, -- Nick ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] LibEvent 1.3e - Compiling for Windows using VS2005
On Sun, Dec 30, 2007 at 04:37:49PM +0200, Eran Sandler wrote: Nick Mathewson wrote: It should all be checked in to the subversion repository. I added a .sln file and a .vcproj file; is there anything else? As noted, I'm not much of a windows person, so it's likely there are mistakes; please let me know what they are. :) Thanks Nick. I hope to get around to checking it soon. There are some differences from the modified libevent code that was used in the win32 version of memcached and the current libevent 1.4 code and I'm not sure which is better (yet). If there are fixes in the modified code, could you please send me a pointer to code, and let me know which version of libevent it's based on? I'd like to check for any fixes we missed so we can merge them into the primary codebase. yrs, -- Nick pgpoUodX0GH0a.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] LibEvent 1.3e - Compiling for Windows using VS2005
On Mon, Dec 17, 2007 at 08:40:28AM +0200, Eran Sandler wrote: [...] Any chance you can simply check in your VS2005 project files. I think its about time we can start using VS2005 and leave MSVC (unless there is anything specific in VS2005 that is preventing us from doing so). It should all be checked in to the subversion repository. I added a .sln file and a .vcproj file; is there anything else? As noted, I'm not much of a windows person, so it's likely there are mistakes; please let me know what they are. :) yrs, -- Nick pgpmsojoAnCfm.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] evhttp supporting virtual hosts
On Thu, Nov 22, 2007 at 08:39:47AM +0100, [EMAIL PROTECTED] wrote: For my python binding to evhttp (temporarily called fapws2 http://www.opensource4you.com/cgi-bin/gitweb.cgi?p=fapws2;a=summary), I would like to implement the concept of virtual host. Is there anyone having expertise with that within libevent ? I'm thinking to add it within the http structure. Indeed, an uri is always coupled with a virtual host. You mean evhttp_request, right? There are a bunch of http-related structures in libevent. Then the evhttp_dispatch_callback need to check the uri AND the virtualhost against the HTTP_X_FORWARDED_FOR environment variable (in case of proxy) or the req-remote_host. What logic did you have in mind for X-Forwarded-For? Do you mean the header, or the environment variable? My understanding is that the environment variable is mainly used by CGI stuff, and I don't want to make libevent CGI-specific by default. To be backward compatible, an empty virtualhost means that the uri is always valid. Sounds reasonable. Unfortunately this is quite invasive within the libevent code. If I'm not wrong the following calls must be adapted: evhttp_free evhttp_set_cb evhttp_del_cb evhttp_set_gencb Changing code is no problem. Changing APIs is not so good, though, if it makes old code stop working. I think the only function whose interface would need to change would be evhttp_set_cb. Instead, it would probably be best to add a new fuction. There's already some discussion of improving the callback-matching interface in the sourceforge feature request tracker at [ 1826562 ] Add wildcard calbacks to http.c http://sourceforge.net/tracker/index.php?func=detailaid=1826562group_id=50884atid=461325 It would be neat to come up with an interface that satisfied both the need of adding wildcards and adding vhost support. Does the core developers will accept to port it in the trunk of libevent ? Sounds reasonable, if the code is clean. It would be even better if it came with regression tests. :) peace, -- Nick pgpEJP0Z4zoyp.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] epoll.c: epoll_dispatch() clarity/simplification/branch reduction cleanups
On Fri, Nov 09, 2007 at 02:27:07AM -0800, Christopher Layne wrote: Regress checks out OK with both -O0 and -Os. Both size are with -O0. $ size .libs/libevent-1.4.so.2 textdata bss dec hex filename 9373510887084 101907 18e13 .libs/libevent-1.4.so.2 $ size .libs/libevent-1.4.so.2 textdata bss dec hex filename 9366310887084 101835 18dcb .libs/libevent-1.4.so.2 Index: libevent/epoll.c === --- libevent/epoll.c (revision 504) +++ libevent/epoll.c (working copy) @@ -209,36 +209,23 @@ The strategy in this patch looks like a clear win. Will apply once back to home computer. Thanks! yrs, -- Nick Mathewson pgp1ybKPQWOO2.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] epoll.c: epoll_dispatch() clarity/simplification/branch reduction cleanups
On Fri, Nov 09, 2007 at 02:27:07AM -0800, Christopher Layne wrote: Regress checks out OK with both -O0 and -Os. Both size are with -O0. Applied; thanks! yrs, -- Nick Mathewson pgpGNOLeYC7MB.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] sample/Makefile.am: additional include
On Fri, Nov 09, 2007 at 11:12:49PM -0800, Christopher Layne wrote: Separate build dir issue. Applied to trunk. yrs, -- Nick Mathewson pgppVzvy8jLzH.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] remove useless argument in bufferevent_read_pressure_cb
On Thu, Nov 08, 2007 at 11:05:27AM +0100, Charles Longeau wrote: Hello, In bufferevent_read_pressure_cb() function, the second argument (which is size_t old) is not used. Here's a patch which removes it. Hi, Charles! bufferevent_read_pressure_cb() may not use the second argument, but other users of the evbuffer API can and likely do. (The evbuffer API is, after all, exported in event.h.) I don't want to break existing code if I can help it. yrs, -- Nick Mathewson pgpd6GCXYCiDt.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] remove useless argument in bufferevent_read_pressure_cb
On Thu, Nov 08, 2007 at 09:15:18PM +0100, Charles Longeau wrote: Hi, bufferevent_read_pressure_cb() may not use the second argument, but other users of the evbuffer API can and likely do. (The evbuffer API is, after all, exported in event.h.) I don't want to break existing code if I can help it. I understand that breaking existing API is not a good solution. However, we can still avoid passing unneeded arguments to bufferevent_read_pressure_cb. Here's an updated patch which removes unneeded arguments without breaking the evbuffer API. I'm afraid that will still break existing code. Any applications that are using evbuffer will expect to get the old buffer size passed to their callback. For example, they may want to know how much data was just added to the buffer. If they get 0 instead of the original buffer length, they will compute an incorrect answer for how much data was just added. yrs, -- Nick Mathewson pgpXTmdhoZ8Tt.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Building on MSVC [was Re: [Libevent-users] libevent win32 issues]
On Tue, Nov 06, 2007 at 12:02:27PM -0500, Nick Mathewson wrote: On Tue, Nov 06, 2007 at 04:40:30PM -, Toby Douglass wrote: Nick Mathewson wrote: [...] I'd like to have everything work on win32. Trunk compiles on mingw fine. I would like it also to compile under MSVC, but I don't have a copy of MSVC. That's why diffs would be nice. :) MS offer a free command line compiler and SDK. Given they're free and compatable with the commercial build environments, would they be a viable choice of Win32 build environment? I seem to recall being told about licensing issues, but the person who told me may have been an alarmist. I'll try it out some time in the next few weeks if I can. Or possibly the next few days. :) I downloaded MSVC++ Express 2005 last night, along with the platform SDK, and gave them a try. The license issues were not as bad as I thought: they put some tricky restrictions on redistribution of binaries, but so long as I'm just using it for development, it should be good enough. Thanks for the tip, Toby! The upshot now is that now there's a .vcproj file for libevent, and now all of the libevent core sources seem to build correctly on MSVC++ Express 2005. Here's what remains to be done for libevent on MSVC: - Right now, the .vcproj project file is set up to build a single libevent library. Instead, it should build the same 3 libraries as trunk does (libevent, libevent-core, libevent-extras). - Once that's done, somebody should update the older project files to include the right code, and see whether there are remaining build issues there. - There are some warnings left to fix. - There should be a Makefile for MSVC, so people can have controlled and repeatable builds. - The unit tests should be made to build under MSVC... and maybe even pass. :) But yeah -- if any brave MSVC users out there want to try out the code in subversion trunk and see if it works for you, that would be grand! cheers, -- Nick Mathewson pgprvURMXAgqL.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent win32 issues
On Tue, Nov 06, 2007 at 04:40:30PM -, Toby Douglass wrote: Nick Mathewson wrote: On Tue, Nov 06, 2007 at 05:09:07PM +0100, Marc Lehmann wrote: There are two factors that keep the select() implementation on win32 from using the same strategy/code as the one : 1) win32's select() doesn't use bitfields; it uses an array of sockets. This is because... 2) win32 sockets, unlike unix fds, are not consecutively numbered integers starting at 0. Thus, it is _NOT_ a good idea to use an array to map fds to events like select.c does; your array would be enormous and sparse. A balanced tree implementation might an improvement here. It would be nice if somebody would step up and write one. select() under win32 only works on sockets, so it's only purpose is socket selection. Given that, select() is a poor choice - I/O completion ports are much better; for one thing, they scale. libiocp exists, although I don't know at all what would be involved in the integration with libevent. Please see all the past discussions on this. :) IOCP is a good match for the bufferevents interface and there has indeed been some work to try to get an IOCP-based bufferevents interface working. See http://sourceforge.net/tracker/index.php?func=detailaid=1826615group_id=50884atid=461325 IOCP doesn't fit very cleanly into the interface supported by the rest of libevent, however: the basic event types expect that the backend will notify them when a blocked operation can be reattempted. IOCP, on the other hand, notifies the application when pending operation is finished. Then again, it may be possible to shoehorn IOCP's behavior into the select/poll/etc style interface of the rest of libevent. I'd love to be proven wrong about this. :) There are other ways to do fast IO on win32. WSAWaitForMultipleEvents is supposedly pretty fast, but it doesn't accept more than 64 sockets, so you need to get fancy with threads to make it work. In other words, Yes, we can do better, but there will be programming involved. Somebody will have to do that programming before we do better. ;) Looking around with google it seems that indeed, evdns et. al does not build on windows (except when using e.g. cygwin, but thats trivial). So I wonder if libevent as a whole is supported under windows at all in current versions? I'd like to have everything work on win32. Trunk compiles on mingw fine. I would like it also to compile under MSVC, but I don't have a copy of MSVC. That's why diffs would be nice. :) MS offer a free command line compiler and SDK. Given they're free and compatable with the commercial build environments, would they be a viable choice of Win32 build environment? I seem to recall being told about licensing issues, but the person who told me may have been an alarmist. I'll try it out some time in the next few weeks if I can. In the mean time, feel free to give it a try; see what happens; see what you can fix; send in a diff? yrs, -- Nick Mathewson pgp0xExUKDNys.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] evdns C++ incompatibility
On Tue, Oct 02, 2007 at 02:49:28AM -0400, Roger Clark wrote:
At least on my system, gcc complains about 'class' being used in these
places in evdns.h:
Hi, Roger! I've checked in a possible fix for this as subversion
revision r486. The fix will appear in the first 1.4 release, assuming
that it doesn't break anything else too badly.
For backward compatibility with existing C code, evdns_server_question
is now defined as:
struct evdns_server_question {
int type;
#ifdef __cplusplus
int dns_question_class;
#else
/* You should refer to this field as dns_question_class. The
* name class works in C for backward compatibility, and will be
* removed in a future version. (1.5 or later). */
int class;
#define dns_question_class class
#endif
char name[1];
};
This should resolve bug 1826516 on the tracker. Please let us know if
there are any remaining C++ issues with the header, or if you run into
anything else like this?
many thanks,
--
Nick Mathewson
pgp6lTrpfiTim.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Two questions...
On Mon, Nov 05, 2007 at 03:49:40PM +0100, Marco Bambini wrote: Hi guys, I just examined the libevent library and I found it very interesting. I have two questions - what about its license? can I use it in a commercial (not open source) project? The license is the so-called 3-clause BSD or modified BSD license. You can find a copy at the head of every file. It does not prohibit you from using libevent in commercial projects. All you need to do is follow the terms of the license: basically, this means that you need to include the license and it its accompanying disclaimer as a part of all source and binary distributions, and all documentation accompanying binary distributions. (I am not a lawyer; the summary above is not a substitute for reading the terms of the license.) yrs, -- Nick Mathewson pgpUde8Z376p4.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Heap-based timer implementation (was Re: [Libevent-users] announcing libev, towards a faster and more featureful libevent)
On Sun, Nov 04, 2007 at 05:19:36PM -0800, Christopher Layne wrote: On Sun, Nov 04, 2007 at 05:04:25PM -0800, Scott Lamb wrote: * timers are managed by a priority queue (O(1) for important operations as opposed to O(log n) in libevent, also resulting in much simpler code). In terms of concrete data types, you appear to have used a binary heap? So by important operations you mean removal, correct? Insertion is still O(log n)? The asymptotic behavior is no different, then, as insertion happens at least as often as removal. Not to put on my O-face, but binary heap insert is *average* O(1). There should be a performance win for libevent, when it comes to timer checking, as using a heap will also be O(1) for heap_min() - which will benefit pending timer calculation code. However, early delete w/ pending timer will need some rigging or tombstone games. I wouldn't be surprised that, in a case where one is consistently resetting timers (think read/write before x time passes) and re-adding said event, that in the end it will be the same amount of CPU time. In case anybody here isn't watching the commits list [1], Niels applied a patch from Maxim Yegorushkin to make the timeout implementation based on a heap, rather than a RB-tree. Thanks, Maxim! This will probably need some testing; if anybody wants to play around with it, just check out the SVN trunk [2] and send in any bugs you run into, or any improvements that we should make. [1] The list is [EMAIL PROTECTED] ; go to https://lists.sourceforge.net/lists/listinfo/levent-commits if you want to subscribe. [2] The subversion repository is at https://levent.svn.sourceforge.net/svnroot/levent To check out trunk, make sure you have subversion installed, and type svn co https://levent.svn.sourceforge.net/svnroot/levent/trunk libevent-trunk yrs, -- Nick Mathewson pgph6MPLjfKJi.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Using the sourceforge.net bugtracker.
Hi, all! I've noticed that we're getting a lot more bug reports and patches than can easily be handled by the mailbox method of organization. Clearly, we need to do a better job. Fortunately, we have a sourceforge.net project page for Libevent: http://sourceforge.net/projects/levent This page has trackers for bugs, patches, and feature requests. Let's use it! What does this mean in practice? - If you have a bug report, feature request, or patch, and you enter it into the appropriate tracker, that'd be great! It will help make sure that nothing gets forgotten. - If you prefer not to use the sourceforge interface, you don't have to; if you send stuff to me or to the mailing list, I'll try to get it added to the tracker there as soon as possible. - With any luck, this should make libevent development more organized. Some time today or tomorrow, I'm going to go through my backlog in libevent-users, my personal notes, and my to-do mailbox, and enter in all the pending libevent bugs, patches, and feature requests that I can find. Please send me an email soon if: - You'd rather enter your bugs/patches/feature ideas yourself. - You don't want me to enter your bugs/patches/feature ideas. I'll send another email when I'm done. I'm also creating a mailing list to get announcements of new tracker entries; I'll include that info in my mail too. many thanks, -- Nick Mathewson pgpr7gY6B7hdJ.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] [UPDATE] Using the sourceforge.net bugtracker.
On Mon, Nov 05, 2007 at 02:58:27PM -0500, Nick Mathewson wrote: Hi, all! I've noticed that we're getting a lot more bug reports and patches than can easily be handled by the mailbox method of organization. Clearly, we need to do a better job. Fortunately, we have a sourceforge.net project page for Libevent: http://sourceforge.net/projects/levent This page has trackers for bugs, patches, and feature requests. Let's use it! Update: I have added 2 patches, 10 feature requests, and 12 bug reports to the tracker. I re-read or skimmed about 500 emails to do this, so it is likely I missed something, or added a bug that has already been fixed. My apologies in advance. I. What to do with the stuff on the tracker list If you have a fix for a bug or an implementation for a feature, add a new patch for it. Maybe discuss your approach a little first, if it's tricky. Niels -- I haven't done any triage on the lists yet. Some of the things listed should get fixed in 1.3; some are probably 1.4 showstoppers; and some can probably wait till 1.5 or later. II. Adding new stuff to the list. Every bug or feature request or patch that I know about is on the tracker now. If you have a bug or feature request or patch that *isn't* there, then please do the following: - Double-check that I didn't add it to one of the lists. - Confirm that the bug still exists. - Verify that the bug is really a bug - Come up with some test code to elicit the buggy behavior, if you possibly can. (If you are sure that the code is wrong, but you can't actually write code to provoke the bad behavior that you're sure is happening, then maybe you haven't really understood the code. Ask on the list maybe?) - Submit it to the tracker. [If this sounds like too much work, just bounce me an email saying hey, you missed this one!] II. How to find out about new tracker entries There's a mailing list at [EMAIL PROTECTED]; you can subscribe here: https://lists.sourceforge.net/lists/listinfo/levent-tracker III. Other stuff you should know, and which should probably go on the webpage. The commits list is here: https://lists.sourceforge.net/lists/listinfo/levent-commits The SVN repository is here: https://levent.svn.sourceforge.net/svnroot/levent Things that you might want to check out of it are: https://levent.svn.sourceforge.net/svnroot/levent/trunk/libevent https://levent.svn.sourceforge.net/svnroot/levent/branches/patches-1.3/libevent Now, it's really time for me to sleep. peace, all. -- Nick Mathewson pgpXoRbZPOWro.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkeymail.org/mailman/listinfo/libevent-users
Re: [Libevent-users] test/*.c files use installed header files not the ones from the package
On Sat, Nov 03, 2007 at 04:18:19PM -0700, Niels Provos wrote: Hi Marc, I appreciate your insights, but your message has nothing to do with libevent. The make files in libevent use the -I option to provide the path to the header files. I just tried to test this out, as follows. I built libevent, and then added #error Foo to the top of the event.h header in the libevent build directory. I confirmed that there existed valid headers (without the #error directive) in /usr/include and /usr/local/include. Then, I entered the test directory and typed make clean; make The compiler gave me: gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I../compat -g -O2 -Wall -c test-init.c In file included from test-init.c:23: ../event.h:2:2: error: #error foo The test/test-init.c file does indeed #include event.h, but it looks like thanks to the -I.. flag passed to gcc, it is finding the copy of event.h in the parent directory. This seems to confirm to me pretty well that, on my OS at least, things work okay. Marc, have you seen this problem you report in the wild? If so, can you give me some help reproducing it? As far as I can tell, Niels is right above. yrs, -- Nick Mathewson pgpTgIjCZtzfF.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] test/*.c files use installed header files not the ones from the package
On Sun, Nov 04, 2007 at 03:02:02AM +0100, Marc Lehmann wrote: On Sat, Nov 03, 2007 at 04:18:19PM -0700, Niels Provos [EMAIL PROTECTED] wrote: I appreciate your insights, but your message has nothing to do with libevent. The make files in libevent use the -I option to provide the path to the header files. Well, since you thre documentation at me before, let me just say its high time to actually read up on what those -I switches do to various compilers But then, I see you are not really interested in fixing bugs: I am indeed interested in fixing bugs. So is Niels. I am going through the emails you have sent me in the order I got them, trying to confirm or disconfirm the bugs, and looking for the fixes you made. I also noticed that you seem to have found several bugs in libevent. And I keep finding more the more I look at it. It would be nice if you could send patches for them. I did that for all bugs where I could identify the right solution. Most bugs are actually fixed by the event core replacement code I wrote, for the remaining ones I send in detailed instructions how to patch the file. I think that by patches, Niels means diffs. I do not see where those are; please forgive me if I have missed them. I am a little bit dubious about some of the claims such as not being able to remove an event while in a callback. Yes, that seems to be simply wrong, sorry for that. If you keep hitting and finding bugs (or undocumented features, such as not beign able to have multiple event watchers for most events, which is probably not a bug, but certainly something outside of my expectation for an event loop), sometimes one makes a mistake. Right. Anybody can make mistakes. That's how bugs happen. But you can make mistakes too: that means that we need to confirm the bugs that you send in (unit tests and diffs would be ideal) and confirm that the fixes fix the bugs and don't break anything else. I'm sorry that this isn't going as fast as possible. As mentioned earlier, I'm at a party in cape cod this weekend, and right now I'm taking time out to try to help get this stuff going better. I am, however, not that thrilled anymore about contributing to libevent after the treatment you give to me, ignoring my patches, and insulting me :( Marc, Niels: There are ways we could all communicate more politely and effectively. Let's take this stuff off-list. It doesn't help make a better libevent to argue about it here. many thanks, and hopes for a better working relationship in the future, -- Nick Mathewson pgpC9hjwadsCf.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] Add autoconf/make functionality for --disable-dns, --disable-http, and --disable-bevents
On Sun, Oct 21, 2007 at 02:00:38PM +0200, Chris Brody-GMail wrote: [...] Thank you guys for the attention to the issue. Small questions: 1. would the evbuffer bufferevents still be part of the libevent-core, or something else? I am working on a C++ interface for this part. I'm pretty sure that evbuffer and friends should stay in event-core, for two reasons: * Unlike HTTP, RPC, and DNS, they're pretty protocol-neutral. * Like event_base, they can be optimized on different platforms in different ways. (Windows, for instance, is begging for an IOCP implementation of bufferevents. I've got one from a student that I want to merge into the next major revision or the one after that, time permitting.) 2. Could we give extras a slightly sexier name such as WWW or web-server? I do find this to be a valuable part as well, and I am planning to work on a C++ interface for the HTTP server part in the future. It's not www only: it's http, dns, and rpc. In the future, it might grow. I'm going to go with the name Niels gave, since it lets me make the changes sooner rather than later. :) yrs, -- Nick Mathewson pgp2u2EZmxbhH.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] Add autoconf/make functionality for --disable-dns, --disable-http, and --disable-bevents
On Sat, Oct 20, 2007 at 10:22:27AM -0700, Niels Provos wrote:
After talking with Nick, I think the solution that makes the most
sense is to build three different libraries:
- libevent-core which contains only the event loop
- libevent-extras which contains DNS, HTTP, etc.
- libevent which contains everything for backwards compatibility
That should make everyone happy.
Okay! I'll do this in trunk in the next couple of days, assuming that
nobody objects.
Another issue: We're apparently using the --revision and --version
arguments to libtool wrong. Our use of --revision means that binaries
built against one version of libevent need to be rebuilt to use the
next. Our current non-use f --version-info means that once we fix the
--revision problem, we'll give linkers the wrong idea about which
versions of libevent are which.
What we need to do in order to use --version-info correctly is
described here:
http://sourceware.org/autobook/autobook/autobook_91.html
Is this something that we're willing to do? If I understand the
documentation correctly, then practice, we'd have to keep 3 numbers:
Current, Revision, and Age. We'd need to note, whenever we do a
new release: whether it adds new interfaces, and whether it breaks
binary compatibility, and then, by cases:
if (the interface changed) {
if (binary compatibility isn't broken)
increment age;
Increment current, set revision to 0.
} else {
increment revision;
}
It's a little more bookkeeping, but I believe it would make us play
better with linkers and applications. For more background, see the
thread Question: Library Naming.
yrs,
--
Nick Mathewson
pgpFaEybGoy1x.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] libevent/event.3: small cleanup and additional notes
On Wed, Sep 26, 2007 at 05:16:23AM -0700, Christopher Layne wrote: It's really difficult to have a diff explain manpage changes due to the nature of ROFF markup, etc. What I like to do in this situation is to compare the roff'd output in order to see what the inputs are. (Patch applied in trunk.) yrs, -- Nick Mathewson pgppm4ehXb07k.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] evdns C++ incompatibility
On Tue, Oct 02, 2007 at 03:18:15PM -0400, Roger Clark wrote:
Most of the header is already surrounded with extern C, but not the
offending part. Just wanted to give someone a heads up.
I just changed it, but it doesn't do any good. My understanding is
that 'extern C' is only for linkage, and it doesn't keep the
compiler from interpreting class as a C++ keyword.
For example, the following C++ is invalid:
extern C {
struct s { int class; };
}
So, it looks like we've got a few options here:
1) Rename the field and break old code that uses it. I think I may
be the only user of this interface, so this wouldn't necessarily
be a tragedy... but if there are others, we might cause them some
grief.
2) Do some malarkey like
struct evdns_server_request {
/* ... */
#ifdef __cplusplus
int dns_question_class;
#else
int class;
#define dns_question_class class
#endif
/* ... */
};
so that we don't break existing C code that uses the field, but
we start building under C++, and everybody can start migrating to
the question_class field name.
3) Do something like
struct evdns_server_request {
/* ... */
#ifdef __cplusplus
int dns_question_class;
#else
int class;
#endif
/* ... */
};
so that we break absolutely no old code, and C++ starts working.
Of course, this would give the field a different name in C than
in C++, and that's not so great.
yrs,
--
Nick Mathewson
pgpVc3QaMJg7s.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Question: Library naming
On Tue, Oct 02, 2007 at 08:14:32PM -0700, Scott Lamb wrote: Trond Norbye wrote: I thought that it was normal to name the libraries like libname.so.major.minor.micro (eg: libevent.so.1.3.4) and create symbolic links so the application may pick up the newest version of the library (with the correct ABI) without re-linking the application. As did I, and I've wondered about this for a while. Is there a good reason for not doing so? Nick, have you given any thought to this? I'm surprised to see your arguments about preserving compatibility considering that each client application needs to be relinked to pick up libevent bugfixes, and many versions of libevent kept around on systems where they haven't been. libevent's APIs (existing functions' signatures + struct event layout) changes infrequently enough that I think most releases could just be a minor version bump. Agreed; I believe the only reason that neither Niels or I has made the change is that neither of us knows libtool very well. It would be good if somebody would submit a patch to do the right thing, and explain which numbers we need to bump in the future when the version changes and/or when we lose backward compatibility. With any luck, it's as simple as messing with libevent_la_LDFLAGS in in Makefile.am. yrs, -- Nick Mathewson pgpzy9irhQOLc.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [patch] 1.3 revision strsep last i post have a mistake
On Sun, Sep 30, 2007 at 10:52:50AM +0800, [EMAIL PROTECTED] wrote: I am pretty sorry that the last patch I posted is not correct. this one will sovlve it. Thanks; I've checked in a version of this patch. yrs, -- Nick Mathewson pgpDRflAW0nYw.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [PATCH] Do not call gettime in timeout_process if the timetree is empty
On Thu, Sep 20, 2007 at 03:14:51PM +0200, Trond Norbye wrote: [...] Why not just return if RB_EMPTY() is true rather than adding another level of indention? Some people (I'm not one of them) dislikes multiple return statements from a function, so I just added the extra level to avoid annoying them ;-) I'm not one of those people, so I'm rewriting your patch to a simple if (RB_EMPTY(base-timetree)) return; and checking it in. ;) yrs, -- Nick Mathewson pgpweF3qLpscB.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Win32 signal handling (and other win32 issues)
So, I'd really like win32 libevent to be a first class libevent
citizen. Right now, trunk doesn't even compile.
Attached is a first cut at trying to make win32 work better: the big
issues are (1) win32.c still assumes that we're still using a global
signalqueue rather than per-base queue as the new code implements.
(The 1.3 branch solves this issue by just disabling the signalqueue
on windows entirely) and (2) signal.c assumes that everybody has a
working sigaction() function (which Windows does not).
Also, I'm pretty sure you can't use read and write for windows
sockets: only send and recv.
This may not be as gloriously threadsafe as the current non-Windows
code: this is where I'd like guidance.
Remaining Win32 issues in trunk:
(1) The win32 socketpair implementation in Win32-Code is all wrong.
It returns a pair of linked win32 pipes. But select() only
accepts sockets, so socketpair() in misc.c can't possibly work
for libevent's needs.
Once the present patch is in, I'll grab the fake-socketpair code
from Tor, which has a compatible license, and which seems to
work on win32 just fine.
Also, it might make sense to expose this as event_socketpair()
or something similar, so that all the libraries using libevent
that want a socketpair can have one without reinventing the
rickety dubious wheel.
(2) Almost none of the files test compile under win32; in fact, they
seem to be disabled. I've make some progress here, but so many
of them require a working socketpair() that I'm going to hold
off until the above issues are solved.
Thoughts?
--
Nick Mathewson
Index: configure.in
===
--- configure.in(revision 440)
+++ configure.in(working copy)
@@ -42,7 +42,7 @@
dnl Checks for header files.
AC_HEADER_STDC
-AC_CHECK_HEADERS(fcntl.h stdarg.h inttypes.h stdint.h poll.h signal.h unistd.h
sys/epoll.h sys/time.h sys/queue.h sys/event.h sys/ioctl.h sys/select.h
sys/devpoll.h port.h netinet/in6.h)
+AC_CHECK_HEADERS(fcntl.h stdarg.h inttypes.h stdint.h poll.h signal.h unistd.h
sys/epoll.h sys/time.h sys/queue.h sys/event.h sys/ioctl.h sys/select.h
sys/devpoll.h port.h netinet/in6.h netdb.h)
if test x$ac_cv_header_sys_queue_h = xyes; then
AC_MSG_CHECKING(for TAILQ_FOREACH in sys/queue.h)
AC_EGREP_CPP(yes,
@@ -135,7 +135,7 @@
AC_HEADER_TIME
dnl Checks for library functions.
-AC_CHECK_FUNCS(gettimeofday vasprintf fcntl clock_gettime strtok_r strsep
getaddrinfo getnameinfo strlcpy inet_ntop)
+AC_CHECK_FUNCS(gettimeofday vasprintf fcntl clock_gettime strtok_r strsep
getaddrinfo getnameinfo strlcpy inet_ntop signal sigaction)
if test x$ac_cv_func_clock_gettime = xyes; then
AC_DEFINE(DNS_USE_CPU_CLOCK_FOR_ID, 1, [Define if clock_gettime is
available in libc])
@@ -333,6 +333,9 @@
AC_LIBOBJ(evport)
needsignal=yes
fi
+if test x$bwin32 = xtrue; then
+ needsignal=yes
+fi
if test x$needsignal = xyes ; then
AC_LIBOBJ(signal)
fi
Index: signal.c
===
--- signal.c(revision 440)
+++ signal.c(working copy)
@@ -30,6 +30,12 @@
#include config.h
#endif
+#ifdef WIN32
+#define WIN32_LEAN_AND_MEAN
+#include windows.h
+#include winsock2.h
+#undef WIN32_LEAN_AND_MEAN
+#endif
#include sys/types.h
#include sys/tree.h
#ifdef HAVE_SYS_TIME_H
@@ -38,7 +44,9 @@
#include sys/_time.h
#endif
#include sys/queue.h
+#ifdef HAVE_SYS_SOCKET_H
#include sys/socket.h
+#endif
#include signal.h
#include stdio.h
#include stdlib.h
@@ -67,7 +75,7 @@
struct event *ev = arg;
ssize_t n;
- n = read(fd, signals, sizeof(signals));
+ n = recv(fd, signals, sizeof(signals), 0);
if (n == -1)
event_err(1, %s: read, __func__);
event_add(ev, NULL);
@@ -98,7 +106,14 @@
base-sig.evsignal_caught = 0;
memset(base-sig.evsigcaught, 0, sizeof(sig_atomic_t)*NSIG);
+#ifdef WIN32
+ {
+ u_long nonblocking = 1;
+ ioctlsocket(base-sig.ev_signal_pair[0], FIONBIO, nonblocking);
+ }
+#else
fcntl(base-sig.ev_signal_pair[0], F_SETFL, O_NONBLOCK);
+#endif
event_set(base-sig.ev_signal, base-sig.ev_signal_pair[1], EV_READ,
evsignal_cb, base-sig.ev_signal);
@@ -110,13 +125,16 @@
evsignal_add(struct event *ev)
{
int evsignal;
+#ifdef HAVE_SIGACTION
struct sigaction sa;
+#endif
struct event_base *base = ev-ev_base;
if (ev-ev_events (EV_READ|EV_WRITE))
event_errx(1, %s: EV_SIGNAL incompatible use, __func__);
evsignal = EVENT_SIGNAL(ev);
+#ifdef HAVE_SIGACTION
memset(sa, 0, sizeof(sa));
sa.sa_handler = evsignal_handler;
sigfillset(sa.sa_mask);
@@ -126,6 +144,11 @@
if (sigaction(evsignal, sa, NULL) == -1)
return (-1);
+#else
Re: [Libevent-users] [PATCH] Fix problems in evport.c
On Wed, Sep 12, 2007 at 10:23:52AM +0200, Trond Norbye wrote: This patch fix two problems with evport.c on Solaris: 1. The grow-function reallocates the ed_fds array, but ed_pending array may contain pointers to the elements in the ed_fds array. These pointers must be relocated to avoid using freed memory. 2. evport_dispatch does not handle a NULL-timval-pinter (no time-based events are active, wait for I/O). Thanks! I've applied this to trunk and to the 1.3 branch. yrs, -- Nick Mathewson pgpfSwvP7FH3h.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] Re: No Date header in HTTP/1.1 response
On Thu, Aug 23, 2007 at 12:14:09PM +0200, Ralph Moritz wrote: On 23/08/07, Jan Kneschke [EMAIL PROTECTED] wrote: This patch will break on non C99 compilers with mixed declaration/code errors. Please move the declarations up to the start of this block. Thanks for spotting that. New patch below. Hopefully this one's kosher. Thanks! I've checked in to trunk and to the 1.3 branch, with the following changes: - Now, it only adds the Date header if there is not a Date header already. - On Windows, it uses gmtime() instead of gmtime_r(). (Windows has no gmtime_r(), but that's okay since its gmtime() uses thread-local storage to achieve thread-safety. Hooray for cross-platform programming, such as it is.) Please try it out and let us know if there are any problems? yrs, -- Nick Mathewson pgpUjulZZdrDe.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] Sigprocmask vs pthread_sigprocmask
Hi, all. There's been a recent exchange on the Tor development list about the intersection of signals with pthreads with libevent. Both posts are (I hope) concise, so I'll link to them here: http://archives.seul.org/or/dev/Feb-2007/msg00028.html http://archives.seul.org/or/dev/Feb-2007/msg00030.html In summary: when pthreads are in use, it's wrong to call sigprocmask(), and right to call its identical cousin, pthread_sigmask(). This isn't just a cosmetic issue; it can cause real bugs with signal delivery. Right now, libevent only calls sigprocmask. It would be good if there were some way to tell libevent, I'm going to use pthreads; use pthread_sigmask() instead of sigprocmask(). I don't know what that interface should be, but the corresponding code should be pretty simple to write. yrs, -- Nick Mathewson pgpjhJl3Tsp4y.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] [Patch] Install autoconf header file as evconfig.h
Hi, all! Now that Libevent 1.3 is out, it's time for Patches Less Obvious. One longstanding problems has been that it's not possible to include event.h unless you first define u_int8_t, u_int32_t, struct timeval, and so on. It wasn't possible for event.h to do so itself, since the info needed to decide whether to define those existed only in the config.h file created by autoconf. Here is a patch that solves this problem by renaming config.h to evconfig.h, making it get included from event.h, and installing it as a header file to $PREFIX/include. This way, people on platforms like mingw and Solaris will be able to build projects that use mingw correctly. This isn't a new or controversial approach, so far as I can tell: many other projects, like mysql, postgresql, glib, expat, firefox, SDL, python, and so on, do the same thing. (For an approximate list, run ls /usr/include/*/*config.h on your favorite _nix machine.) The patch is here: http://www.wangafu.net/~nickm/libevent_evconfig_patch/evconfig.diff In addition to this patch, you'll need to rename WIN32-Code/config.h to WIN32-Code/evconfig.h. You'll need to re-run aclocal, autoheader, autoconf, and automake after applying this patch. I have not tested it on all platforms. yrs, -- Nick Mathewson pgpKRwfU00ruW.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [Patch] Install autoconf header file as evconfig.h
On Fri, Feb 16, 2007 at 06:24:34PM -0500, Nick Mathewson wrote: [...] This way, people on platforms like mingw and Solaris will be able to build projects that use mingw correctly. Oh dear. This is why I should proofread everything _three_ times, I guess. This sentence should read ...to build projects that use event.h correctly. yrs, -- Nick Mathewson pgpzTKrxvnSb4.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] libevent-1.3 released
On Thu, Feb 15, 2007 at 05:30:01PM -0800, Niels Provos wrote: Hi everyone, I just released libevent-1.3. It contains a ton of changes: - DNS server from Nick Mathewson Actually, the original DNS server code is by Adam Langley; I only cleaned it up a little. Adam definitely deserves the credit. yrs, -- Nick Mathewson pgpIlCRF1uyGf.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] [Patch] Install autoconf header file as evconfig.h
On Fri, Feb 16, 2007 at 04:10:03PM -0800, William Ahern wrote: On Fri, Feb 16, 2007 at 06:24:34PM -0500, Nick Mathewson wrote: Hi, all! Now that Libevent 1.3 is out, it's time for Patches Less Obvious. One longstanding problems has been that it's not possible to include event.h unless you first define u_int8_t, u_int32_t, struct timeval, and so on. It wasn't possible for event.h to do so itself, since the info needed to decide whether to define those existed only in the config.h file created by autoconf. Here is a patch that solves this problem by renaming config.h to evconfig.h, making it get included from event.h, and installing it as a header file to $PREFIX/include. This way, people on platforms like mingw and Solaris will be able to build projects that use mingw correctly. Wouldn't it be cleaner to either 1) Use (unsigned char) instead of (u_int8_t), (unsigned int) instead of (u_int32_t), and include sys/time.h or timeb.h, conditional on the WIN32 macro; Actually, you'd want to do unsigned long instead of unsigned int; old C only guarantees that int can hold 16 bits. or, if the above isn't portable enough from a theoretical perspective, 2) Build an event.h using M4 and the autotools framework? The additional header just litters things unneccesarily, especially for something this trivial (not to say the headaches are trivial; I build on all these platforms, too). I think your approach is reasonable too in the medium term; you should probably send a patch to the list too, so yours can win. :) yrs, -- Nick Mathewson pgp6LdQBu9T9V.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] [Patch] Add a regression test for DNS IPv6 records; fix two bugs in the AAAA record code.
So, I saw that Niels had checked in my code to do IPv6 DNS lookups
with the comment:
support for DNS; from Nick Mathewson.
unfortunately, no regression test
Oh dear! I thought. I'd better write a regression test, just to
prove that my code works.
Turns out, it had a couple of bugs. This patch fixes those bugs, and
adds a regression test. It also fixes a small bug in the IPv4 DNS
regression test, which would, if it received N A records, print the
first record N times.
many thanks, and apologies for the bugs,
--
Nick Mathewson
=== configure.in
==
--- configure.in(revision 11547)
+++ configure.in(local)
@@ -132,7 +132,7 @@
AC_HEADER_TIME
dnl Checks for library functions.
-AC_CHECK_FUNCS(gettimeofday vasprintf fcntl clock_gettime strtok_r strsep
getaddrinfo getnameinfo strlcpy)
+AC_CHECK_FUNCS(gettimeofday vasprintf fcntl clock_gettime strtok_r strsep
getaddrinfo getnameinfo strlcpy inet_ntop)
if test x$ac_cv_func_clock_gettime = xyes; then
AC_DEFINE(DNS_USE_CPU_CLOCK_FOR_ID, 1, [Define if clock_gettime is
available in libc])
=== evdns.c
==
--- evdns.c (revision 11547)
+++ evdns.c (local)
@@ -596,6 +596,7 @@
req-user_pointer);
else
req-user_callback(err, 0, 0, 0, NULL,
req-user_pointer);
+return;
}
assert(0);
}
@@ -1783,7 +1784,7 @@
static int
search_request_new(int type, const char *const name, int flags,
evdns_callback_type user_callback, void *user_arg) {
- assert(type == TYPE_A);
+ assert(type == TYPE_A || type == TYPE_);
if ( ((flags DNS_QUERY_NO_SEARCH) == 0)
global_search_state
global_search_state-num_domains) {
=== test/regress_dns.c
==
--- test/regress_dns.c (revision 11547)
+++ test/regress_dns.c (local)
@@ -47,6 +47,9 @@
#include arpa/inet.h
#include unistd.h
#endif
+#ifdef HAVE_NETINET_IN6_H
+#include netinet/in6.h
+#endif
#include netdb.h
#include fcntl.h
#include stdlib.h
@@ -72,6 +75,24 @@
fprintf(stderr, type: %d, count: %d, ttl: %d: , type, count, ttl);
switch (type) {
+ case DNS_IPv6_: {
+#if defined(HAVE_STRUCT_IN6_ADDR) defined(HAVE_INET_NTOP)
+ struct in6_addr *in6_addrs = addresses;
+ char buf[INET6_ADDRSTRLEN+1];
+ int i;
+ /* a resolution that's not valid does not help */
+ if (ttl 0)
+ goto out;
+ for (i = 0; i count; ++i) {
+ const char *b = inet_ntop(AF_INET6, in6_addrs[i],
buf,sizeof(buf));
+ if (b)
+ fprintf(stderr, %s , b);
+ else
+ fprintf(stderr, %s , strerror(errno));
+ }
+#endif
+ break;
+ }
case DNS_IPv4_A: {
struct in_addr *in_addrs = addresses;
int i;
@@ -79,7 +100,7 @@
if (ttl 0)
goto out;
for (i = 0; i count; ++i)
- fprintf(stderr, %s , inet_ntoa(in_addrs[0]));
+ fprintf(stderr, %s , inet_ntoa(in_addrs[i]));
break;
}
case DNS_PTR:
@@ -93,7 +114,7 @@
goto out;
}
- dns_ok = 1;
+ dns_ok = type;
out:
event_loopexit(NULL);
@@ -107,7 +128,7 @@
evdns_resolve_ipv4(www.monkey.org, 0, dns_gethostbyname_cb, NULL);
event_dispatch();
- if (dns_ok) {
+ if (dns_ok == DNS_IPv4_A) {
fprintf(stdout, OK\n);
} else {
fprintf(stdout, FAILED\n);
@@ -116,6 +137,22 @@
}
void
+dns_gethostbyname6()
+{
+ fprintf(stdout, IPv6 DNS resolve: );
+ dns_ok = 0;
+ evdns_resolve_ipv6(www.ietf.org, 0, dns_gethostbyname_cb, NULL);
+ event_dispatch();
+
+ if (dns_ok == DNS_IPv6_) {
+ fprintf(stdout, OK\n);
+ } else {
+ fprintf(stdout, FAILED\n);
+ exit(1);
+ }
+}
+
+void
dns_gethostbyaddr()
{
struct in_addr in;
@@ -125,7 +162,7 @@
evdns_resolve_reverse(in, 0, dns_gethostbyname_cb, NULL);
event_dispatch();
- if (dns_ok) {
+ if (dns_ok == DNS_PTR) {
fprintf(stdout, OK\n);
} else {
fprintf(stdout, FAILED\n);
@@ -138,6 +175,7 @@
{
evdns_init();
dns_gethostbyname();
+ dns_gethostbyname6();
dns_gethostbyaddr();
evdns_shutdown(0);
pgpK6qhX2JnPn.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http
[Libevent-users] [Patch] Avoid conflicts with platform strlcpy
Hi, all! This patch renames the strlcpy defined in strlcpy.c so that
it no longer messes with tests by other programs for the presence of a
strlcpy function. This patch also fixes compilation on mingw, and
fixes a warning in regress_http.c.
cheers,
--
Nick Mathewson
=== Makefile.am
==
--- Makefile.am (revision 11673)
+++ Makefile.am (local)
@@ -40,8 +40,8 @@
endif
libevent_la_SOURCES = event.c buffer.c evbuffer.c log.c event_tagging.c \
- http.c evhttp.h http-internal.h evdns.c evdns.h evrpc.c \
- evrpc.h evrpc-internal.h $(SYS_SRC)
+ http.c evhttp.h http-internal.h evdns.c evdns.h evrpc.c strlcpy.c \
+ evrpc.h evrpc-internal.h strlcpy-internal.h $(SYS_SRC)
libevent_la_LIBADD = @LTLIBOBJS@ $(SYS_LIBS)
libevent_la_LDFLAGS = -release @VERSION@ -version-info 1:3:0
=== configure.in
==
--- configure.in(revision 11673)
+++ configure.in(local)
@@ -132,8 +132,7 @@
AC_HEADER_TIME
dnl Checks for library functions.
-AC_CHECK_FUNCS(gettimeofday vasprintf fcntl clock_gettime strtok_r strsep
getaddrinfo getnameinfo)
-AC_REPLACE_FUNCS(strlcpy)
+AC_CHECK_FUNCS(gettimeofday vasprintf fcntl clock_gettime strtok_r strsep
getaddrinfo getnameinfo strlcpy)
if test x$ac_cv_func_clock_gettime = xyes; then
AC_DEFINE(DNS_USE_CPU_CLOCK_FOR_ID, 1, [Define if clock_gettime is
available in libc])
=== http.c
==
--- http.c (revision 11673)
+++ http.c (local)
@@ -73,6 +73,7 @@
#undef timeout_pending
#undef timeout_initialized
+#include strlcpy-internal.h
#include event.h
#include evhttp.h
#include log.h
=== strlcpy.c
==
--- strlcpy.c (revision 11673)
+++ strlcpy.c (local)
@@ -37,7 +37,8 @@
#include config.h
#endif /* HAVE_CONFIG_H */
-#include string.h
+#ifndef HAVE_STRLCPY
+#include strlcpy-internal.h
/*
* Copy src to string dst of size siz. At most siz-1 characters
@@ -45,7 +46,7 @@
* Returns strlen(src); if retval = siz, truncation occurred.
*/
size_t
-strlcpy(dst, src, siz)
+_event_strlcpy(dst, src, siz)
char *dst;
const char *src;
size_t siz;
@@ -72,3 +73,4 @@
return(s - src - 1);/* count does not include NUL */
}
+#endif
=== test/regress_http.c
==
--- test/regress_http.c (revision 11673)
+++ test/regress_http.c (local)
@@ -128,7 +128,7 @@
event_debug((%s: %s\n, __func__, EVBUFFER_DATA(bev-input)));
- if (evbuffer_find(bev-input, what, strlen(what)) != NULL) {
+ if (evbuffer_find(bev-input, (const unsigned char*) what,
strlen(what)) != NULL) {
struct evhttp_request *req = evhttp_request_new(NULL, NULL);
req-kind = EVHTTP_RESPONSE;
int done = evhttp_parse_lines(req, bev-input);
@@ -445,7 +445,7 @@
http_failure_readcb(struct bufferevent *bev, void *arg)
{
const char *what = 400 Bad Request;
- if (evbuffer_find(bev-input, what, strlen(what)) != NULL) {
+ if (evbuffer_find(bev-input, (const unsigned char*) what,
strlen(what)) != NULL) {
test_ok = 2;
bufferevent_disable(bev, EV_READ);
event_loopexit(NULL);
#ifndef _STRLCPY_INTERNAL_H_
#define _STRLCPY_INTERNAL_H_
#ifdef __cplusplus
extern C {
#endif
#ifdef HAVE_CONFIG_H
#include config.h
#endif /* HAVE_CONFIG_H */
#ifndef HAVE_STRLCPY
#include string.h
size_t _event_strlcpy(char *dst, const char *src, size_t siz);
#define strlcpy _event_strlcpy
#endif
#ifdef __cplusplus
}
#endif
#endif
pgp60WRhtvX37.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users
[Libevent-users] [Patch] Add dns server support, ipv6 support, and misc cleanups to evdns
Hi, all. Hi, Niels! Here's another patch to evdns.c. This patch has
the following effects:
- add functions for forward and reverse IPv6 lookups.
- implement the server side of the DNS protocol: receive a set of
questions, and send an answer back to the client.
- As a subcomponent, implement the compression side of the DNS
name compression scheme.
- macro fix: WIN32 is sufficient; MS_WINDOWS is unnecessary.
- suppress warnings on some compilers related to running off the end
of a function or unused variables or typechecking.
- spelling fix: choaked-choked
The example main() function has been expanded to include demonstration
use of the server functions. The new functions still need
documentation and references in the man page. I've tried to test this
all as well as I could, but there may be bugs left, especially in
retrying writes after they've failed. This patch should not introduce
bugs in code that doesn't use any of the new functions.
yrs,
--
Nick Mathewson
=== configure.in
==
--- configure.in(revision 11547)
+++ configure.in(local)
@@ -341,6 +341,7 @@
AC_CHECK_TYPE(u_int32_t, unsigned int)
AC_CHECK_TYPE(u_int16_t, unsigned short)
AC_CHECK_TYPE(u_int8_t, unsigned char)
+AC_CHECK_TYPES(struct in6_addr, , , [#include netinet/in.h])
AC_MSG_CHECKING([for socklen_t])
AC_TRY_COMPILE([
=== evdns.c
==
--- evdns.c (revision 11547)
+++ evdns.c (local)
@@ -134,12 +134,11 @@
#define MAX_ADDRS 4 // maximum number of addresses from a single packet
// which we bother recording
-#define TYPE_A 1
-#define TYPE_CNAME 5
-#define TYPE_PTR 12
-#define TYPE_ 28
+#define TYPE_A EVDNS_TYPE_A
+#define TYPE_PTR EVDNS_TYPE_PTR
+#define TYPE_ EVDNS_TYPE_
-#define CLASS_INET 1
+#define CLASS_INET EVDNS_CLASS_INET
struct request {
u8 *request; // the dns packet data
@@ -167,6 +166,12 @@
char transmit_me; // needs to be transmitted
};
+#ifndef HAVE_STRUCT_IN6_ADDR
+struct in6_addr {
+ u8 s6_addr[16];
+};
+#endif
+
struct reply {
unsigned int type;
unsigned int have_answer;
@@ -176,6 +181,10 @@
u32 addresses[MAX_ADDRS];
} a;
struct {
+ u32 addrcount;
+ struct in6_addr addresses[MAX_ADDRS];
+ } ;
+ struct {
char name[HOST_NAME_MAX];
} ptr;
} data;
@@ -193,13 +202,79 @@
// when we next probe this server.
// Valid if state == 0
char state; // zero if we think that this server is down
- char choaked; // true if we have an EAGAIN from this server's socket
+ char choked; // true if we have an EAGAIN from this server's socket
char write_waiting; // true if we are waiting for EV_WRITE events
};
static struct request *req_head = NULL, *req_waiting_head = NULL;
static struct nameserver *server_head = NULL;
+// Represents a local port where we're listening for DNS requests. Right now,
+// only UDP is supported.
+struct evdns_server_port {
+ int socket; // socket we use to read queries and write replies.
+ int refcnt; // reference count.
+ char choked; // Are we currently blocked from writing?
+ char closing; // Are we trying to close this port, pending writes?
+ evdns_request_callback_fn_type user_callback; // Fn to handle requests
+ void *user_data; // Opaque pointer passed to user_callback
+ struct event event; // Read/write event
+ // circular list of replies that we want to write.
+ struct server_request *pending_replies;
+};
+
+// Represents part of a reply being built. (That is, a single RR.)
+struct server_reply_item {
+ struct server_reply_item *next; // next item in sequence.
+ char *name; // name part of the RR
+ u16 type : 16; // The RR type
+ u16 class : 16; // The RR class (usually CLASS_INET)
+ u32 ttl; // The RR TTL
+ char is_name; // True iff data is a label
+ u16 datalen; // Length of data; -1 if data is a label
+ void *data; // The contents of the RR
+};
+
+// Represents a request that we've received as a DNS server, and holds
+// the components of the reply as we're constructing it.
+struct server_request {
+ // Pointers to the next and previous entries on the list of replies
+ // that we're waiting to write. Only set if we have tried to respond
+ // and gotten EAGAIN.
+ struct server_request *next_pending;
+ struct server_request *prev_pending;
+
+ u16 trans_id; // Transaction id.
+ struct evdns_server_port *port; // Which port received this request on?
+ struct sockaddr_storage addr
[Libevent-users] [Patch] Fix a bug related to DNS timeout counting
Hi, all! This patch fixes a bug in nameserver timeouts in evdns.c.
Previously, we would count the number of successive requests that had
timed out in a row for each nameserver, but we would never actually
reset the timeout count to zero. This, obviously, would create
problems: see the Tor bugtracker entry at
http://bugs.noreply.org/flyspray/index.php?id=326do=details
This patch resets the successive timeout count to zero when:
- A nameserver comes up
- We receive a reply from a nameserver
- We decide to not use the nameserver for a while because of its
timeout count.
This patch also changes the timeout threshold from 3 to 5 seconds.
Probably, it should be easier to adjust. There may also be other
issues that cause spurious nameserver timeouts: see the final comment
in the bugtracker link above.
yrs,
--
Nick Mathewson
=== evdns.c
==
--- evdns.c (revision 11485)
+++ evdns.c (local)
@@ -212,7 +212,7 @@
static int global_max_requests_inflight = 64;
-static struct timeval global_timeout = {3, 0}; // 3 seconds
+static struct timeval global_timeout = {5, 0}; // 5 seconds
static int global_max_reissues = 1; // a reissue occurs when we get some
errors from the server
static int global_max_retransmits = 3; // number of times we'll retransmit a
request which timed out
// number of timeouts in a row before we consider this server to be down
@@ -447,6 +447,7 @@
evtimer_del(ns-timeout_event);
ns-state = 1;
ns-failed_times = 0;
+ ns-timedout = 0;
global_good_nameservers++;
}
@@ -891,6 +892,7 @@
nameserver_failed(ns, strerror(err));
return;
}
+ ns-timedout = 0;
reply_parse(packet, r);
}
}
@@ -1051,6 +1053,7 @@
req-ns-timedout++;
if (req-ns-timedout global_max_nameserver_timeout) {
+ req-ns-timedout = 0;
nameserver_failed(req-ns, request timed out.);
}
pgpPXZxJJbo7L.pgp
Description: PGP signature
___
Libevent-users mailing list
Libevent-users@monkey.org
http://monkey.org/mailman/listinfo/libevent-users
Re: [Libevent-users] How does libevent deal with more events than a particular syscall can handle?
On Sat, Nov 18, 2006 at 05:53:24PM -0500, Roger Clark wrote: For instance, the maximum number of fds usable by select() on Win32 (or other platforms) is low compared to the number of potential connections needed by a high-throughput HTTP server. Check out the C code; it's all there. :) On most platforms, the number of FDs passed to select() is not actually limited by the kernel; the only limitation is on the size of the fd_set structure. But libevent doesn't use fd_set. On unixlike systems, when select() is used, it builds its own bitfields; on Windows, it builds a counted array of fds. Of course, as noted, select() is very inefficient, and isn't used when libevent can avoid it. Check the list archives for some IOCP discussion. (Generally, it's a good idea to check the archives before posting about anything.) yrs, -- Nick Mathewson pgpq2XTaKTZU8.pgp Description: PGP signature ___ Libevent-users mailing list Libevent-users@monkey.org http://monkey.org/mailman/listinfo/libevent-users
