On Jan 8, 2013, at 12:55 AM, Björn K. wrote: > That piece of code should distribute the incoming connections over the > threads (I want to change that code later to select the thread with > the lowest number of handled connections). If I don't use this code > and use instead > bev = bufferevent_openssl_socket_new(thread_base[0], sock, client_ctx, ... > the problem remains. > > Furthermore I don't think it's a problem of running out of memory. The > segmentation fault even occurs when there are just a few connections > to the server (once the third connection produced the crash). > I tried to get a segmentation fault during a run in valgrind. One time > I succeeded and got this: > > ==310== Invalid read of size 8 > ==310== at 0x41327D: bufferevent_incref_and_lock_ (bufferevent.c:626) > ==310== by 0x41403F: bufferevent_enable (bufferevent.c:448) > ==310== by 0x4044A9: acceptcb (in /home/login/test/simple) > ==310== by 0x424325: listener_read_cb (listener.c:412) > ==310== by 0x41D889: event_process_active_single_queue (event.c:1471) > ==310== by 0x41E0E6: event_base_loop (event.c:1538) > ==310== by 0x4049E0: main (in /home/login/test/simple) > ==310== Address 0x62531a0 is 464 bytes inside a block of size 560 free'd > ==310== at 0x4C240FD: free (vg_replace_malloc.c:366) > ==310== by 0x413E2A: bufferevent_decref_and_unlock_ (bufferevent.c:703) > ==310== by 0x41DAD4: event_process_active_single_queue (event.c:1476) > ==310== by 0x41E0E6: event_base_loop (event.c:1538) > ==310== by 0x404693: worker_function (in /home/login/test/simple) > ==310== by 0x54258C9: start_thread (pthread_create.c:300) > ==310== > > (And many similar messages. What they have in common is that some > memory is freed in the worker thread and the main thread tries to > access it.) > > But I don't know why memory of the bufferevent can be freed in the > worker thread before the buffer event is enabled.
I think the key to solve these issues is change the design - do not use libevent apis from multiple threads except event_active(). The simplest solution would be to use libevhtp <https://github.com/ellzey/libevhtp/> which uses smart event based thread pool and looks very clean. If you want more control, the simplest solution is to get a single process working, then fork few worker processes in the main process, all sharing the same listener file descriptor, bound in the main process. Then let the kernel do the load balancing for you. Successful projects like nginx and unicorn <https://github.com/blog/517-unicorn> work like this. When I try the code you submitted (after some cleanup to make it compile and run), it does not respond when accessing with a browser. How are you stress testing this code? > > 2013/1/7 Mark Ellzey <mtho...@strcpy.net>: >> On Sun, Jan 06, 2013 at 09:39:48PM +0100, Bj?rn K. wrote: >> >> >> at a second glance: >> >> next_thread = (next_thread + 1) % NUM_THREADS; <- are you sure this is >> working as intended? >> *********************************************************************** >> To unsubscribe, send an e-mail to majord...@freehaven.net with >> unsubscribe libevent-users in the body. > *********************************************************************** > To unsubscribe, send an e-mail to majord...@freehaven.net with > unsubscribe libevent-users in the body.
