Hey On Wed, Feb 18, 2009 at 2:18 AM, Kay Röpke <kay.roe...@sun.com> wrote: > Hi! > > On Feb 17, 2009, at 6:57 PM, nick loeve wrote: > >> I reset the funnel branch on launchpad, which is now using the changes >> from Jan's event threads branch. > > Great to hear! > >> ATM there is some memory corruption when backend connections are idle, >> a wait_timeout occurs, mysql kills the conneciton, and the connection >> is removed from the connection pool. Im currently debugging this, >> which seems to propogate as a segfault during libevent >> dispatching/loop-exiting in the event threads themselves. > > Interesting. It sounds like a missing event_del to me. > Looking at the code in network_connection_pool_entry_free() the most obvious > thing > would be to track the free_sock boolean to see what's going on. > I've just gone over the call sites for network_connection_pool_entry_free > and all three > seem to correctly remove the event handler (one is non-obvious at first, at > least to me at this hour). > The next best thing I can come up with is: > going from network_mysqld_con_idle_handle() to > network_connection_pool_remove() to > network_connection_pool_entry_free() will attempt to remove the event > handler. > Maybe the problem lies in there somehow. >
Well I fixed a couple of issues. Im a bit stuck with the following issue: If I set event-threads > 0 then we get memory corruption like the following backtrace. Program received signal SIGABRT, Aborted. [Switching to Thread 0x4294c950 (LWP 20747)] 0x00007ff9321b33c5 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 64 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig); (gdb) bt #0 0x00007ff9321b33c5 in *__GI_raise (sig=<value optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #1 0x00007ff9321b473e in *__GI_abort () at abort.c:88 #2 0x00007ff9321efacf in malloc_printerr (action=2, str=0x7ff93228af45 "malloc(): memory corruption", ptr=0x5108) at malloc.c:5892 #3 0x00007ff9321f1d36 in _int_malloc (av=0x7ff9324bd960, bytes=1025) at malloc.c:4185 #4 0x00007ff9321f3c82 in malloc_check (sz=1024, caller=<value optimized out>) at hooks.c:265 #5 0x00007ff9321f417f in realloc_check (oldmem=0x0, bytes=1024, caller=0x6) at hooks.c:329 #6 0x00007ff93250d3b6 in IA__g_realloc (mem=0x0, n_bytes=1024) at gmem.c:170 #7 0x00007ff93252d337 in g_string_maybe_expand (string=0x65a720, len=607) at gstring.c:359 #8 0x00007ff93252d39b in IA__g_string_sized_new (dfl_size=607) at gstring.c:384 #9 0x00007ff933d07a8b in network_socket_read (sock=0x631d90) at network-socket.c:507 #10 0x00007ff933cfa89f in network_mysqld_read (srv=0x6098f0, con=0x631d90) at network-mysqld.c:499 #11 0x00007ff933cfc3d4 in network_mysqld_con_handle (event_fd=25, events=2, user_data=0x655ce0) at network-mysqld.c:1348 #12 0x00007ff933ad7d9b in event_process_active (base=0x61fc50) at event.c:387 #13 0x00007ff933ad80df in event_base_loop (base=0x61fc50, flags=0) at event.c:536 #14 0x00007ff933ad7e00 in event_base_dispatch (event_base=0x61fc50) at event.c:407 #15 0x00007ff933f19958 in chassis_event_thread_loop (event_thread=0x61fba0) at chassis-event-thread.c:257 #16 0x00007ff932534e71 in g_thread_create_proxy (data=0x62e9c0) at gthread.c:635 #17 0x00007ff9329d7047 in start_thread (arg=<value optimized out>) at pthread_create.c:296 #18 0x00007ff93224328d in clone () from /lib/libc.so.6 #19 0x0000000000000000 in ?? () As each connection should have it's state machine running in exactly one thread at any given time, So I am not how the network_socket queues are becoming an issue. I assume though that although the crash consistently happens while processing the network_socket queues, it may have been caused by my stupidity elsewhere. Running with '--event-threads=0' shows no issue. This has pretty stopped me flat at the moment... any help/pointers would be appreciated. Has anyone had a play with the funnel plugin? Cheers -- Nick Loeve _______________________________________________ Mailing list: https://launchpad.net/~mysql-proxy-discuss Post to : mysql-proxy-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~mysql-proxy-discuss More help : https://help.launchpad.net/ListHelp
