On 02/26/2012 12:16 PM, Steven Dake wrote: > On 02/25/2012 02:25 PM, Daniel Stenberg wrote: >> On Sat, 25 Feb 2012, Steven Dake wrote: >> >>> So it appears the packets are still present on the session list but >>> somehow dropped by the open call and not removed. >>> >>> If you have other suggestions for getting to the root cause please let >>> me know. I am happy to help. >> >> My suspicion is that the call to _libssh2_list_add() within the >> src/packet.c:_libssh2_packet_add() function is what adds the packet to >> the incoming queue of packets, and some packets that arrive are never >> yanked off that list. >> >> If you would add some debug code that would count the length of that >> linked list and logged it occasionally we could see if that is true. The >> list should really only grow temporarily and then go back to almost >> nothing. If that turns out to be true, we then of course need to figure >> out what packets that are lingering there and how to stop them from >> doing it! >> >> (BTW, I'll be away on a week's vacation starting tomorrow so I might not >> respond very quickly until I get back.) >> > > I have found two packets that seem to leak into the session packet list > which are not removed > > The packet id 3 (#define SSH_MSG_UNIMPLEMENTED) seems to leak 1 or 2 times. > > Stack trace: > #0 0x000000366b436285 in raise () from /lib64/libc.so.6 > #1 0x000000366b437b9b in abort () from /lib64/libc.so.6 > #2 0x000000366b42ee9e in __assert_fail_base () from /lib64/libc.so.6 > #3 0x000000366b42ef42 in __assert_fail () from /lib64/libc.so.6 > #4 0x00007ffff7b5684b in _libssh2_packet_add (session=0x62eea0, > data=0x623500 "\024\250x\006\275 \336!\220\070\371#\375uĊ%", > datalen=769, macstate=0) at packet.c:429 > #5 0x00007ffff7b6a96f in fullpacket (session=0x62eea0, encrypted=0) at > transport.c:236 > #6 0x00007ffff7b6b1f3 in _libssh2_transport_read (session=0x62eea0) at > transport.c:556 > #7 0x00007ffff7b57f4a in _libssh2_packet_require (session=0x62eea0, > packet_type=20 '\024', data=0x63bcb8, data_len=0x63bcc0, match_ofs=0, > match_buf=0x0, match_len=0, state= > 0x63bb60) at packet.c:1097 > #8 0x00007ffff7b54167 in _libssh2_kex_exchange (session=0x62eea0, > reexchange=0, key_state=0x63bb58) at kex.c:1717 > #9 0x00007ffff7b5ddf0 in session_startup (session=0x62eea0, sock=12) at > session.c:718 > #10 0x00007ffff7b5e0ba in libssh2_session_handshake (session=0x62eea0, > sock=12) at session.c:796 > #11 0x00007ffff7b5e112 in libssh2_session_startup (session=0x62eea0, > sock=12) at session.c:815 > #12 0x000000000040600c in ssh_assembly_connect (data=0x629c60) at > trans_ssh.c:374 > #13 0x00007ffff7d8b1eb in job_dispatch (item=0x6261b0, p=<optimized > out>) at loop_job.c:39 > #14 0x00007ffff7d89d37 in qb_loop_run_level (level=0x60ff30) at loop.c:45 > #15 qb_loop_run (lp=<optimized out>) at loop.c:206 > #16 0x0000000000403745 in main (argc=<optimized out>, > argv=0x7fffffffde48) at caped.c:171 > > > The packet id of 82 (#define SSH_MSG_REQUEST_FAILURE 82) leaks continuously. > > Two stack traces: > consistent leaks: > #0 0x000000366b436285 in raise () from /lib64/libc.so.6 > #1 0x000000366b437b9b in abort () from /lib64/libc.so.6 > #2 0x000000366b42ee9e in __assert_fail_base () from /lib64/libc.so.6 > #3 0x000000366b42ef42 in __assert_fail () from /lib64/libc.so.6 > #4 0x00007ffff7b56864 in _libssh2_packet_add (session=0x655220, > data=0x66e9b0 "`", datalen=5, macstate=0) at packet.c:430 > #5 0x00007ffff7b6a987 in fullpacket (session=0x655220, encrypted=1) at > transport.c:236 > #6 0x00007ffff7b6b20b in _libssh2_transport_read (session=0x655220) at > transport.c:556 > #7 0x00007ffff7b4d8c2 in channel_wait_eof (channel=0x670c80) at > channel.c:2213 > #8 0x00007ffff7b4d948 in libssh2_channel_wait_eof (channel=0x670c80) at > channel.c:2242 > #9 0x00000000004058ea in assembly_ssh_exec (data=0x670800) at > trans_ssh.c:200 > #10 0x00007ffff7d8b1eb in job_dispatch (item=0x66ffc0, p=<optimized > out>) at loop_job.c:39 > #11 0x00007ffff7d89d37 in qb_loop_run_level (level=0x60ff30) at loop.c:45 > #12 qb_loop_run (lp=<optimized out>) at loop.c:206 > #13 0x0000000000403745 in main (argc=<optimized out>, > argv=0x7fffffffde48) at caped.c:171 > > > a leak during startup on packet 82: > (gdb) where > #0 0x000000366b436285 in raise () from /lib64/libc.so.6 > #1 0x000000366b437b9b in abort () from /lib64/libc.so.6 > #2 0x000000366b42ee9e in __assert_fail_base () from /lib64/libc.so.6 > #3 0x000000366b42ef42 in __assert_fail () from /lib64/libc.so.6 > #4 0x00007ffff7b5684b in _libssh2_packet_add (session=0x62eea0, > data=0x6173a0 "\024\341l\241\236\003\\\277\340\256\357ь\364r\320\060", > datalen=769, macstate=0) at packet.c:429 > #5 0x00007ffff7b6a96f in fullpacket (session=0x62eea0, encrypted=0) at > transport.c:236 > #6 0x00007ffff7b6b1f3 in _libssh2_transport_read (session=0x62eea0) at > transport.c:556 > #7 0x00007ffff7b57f4a in _libssh2_packet_require (session=0x62eea0, > packet_type=20 '\024', data=0x63bcb8, data_len=0x63bcc0, match_ofs=0, > match_buf=0x0, match_len=0, state= > 0x63bb60) at packet.c:1097 > #8 0x00007ffff7b54167 in _libssh2_kex_exchange (session=0x62eea0, > reexchange=0, key_state=0x63bb58) at kex.c:1717 > #9 0x00007ffff7b5ddf0 in session_startup (session=0x62eea0, sock=12) at > session.c:718 > #10 0x00007ffff7b5e0ba in libssh2_session_handshake (session=0x62eea0, > sock=12) at session.c:796 > #11 0x00007ffff7b5e112 in libssh2_session_startup (session=0x62eea0, > sock=12) at session.c:815 > #12 0x000000000040600c in ssh_assembly_connect (data=0x629c60) at > trans_ssh.c:374 > #13 0x00007ffff7d8b1eb in job_dispatch (item=0x6294c0, p=<optimized > out>) at loop_job.c:39 > #14 0x00007ffff7d89d37 in qb_loop_run_level (level=0x60ff30) at loop.c:45 > #15 qb_loop_run (lp=<optimized out>) at loop.c:206 > #16 0x0000000000403745 in main (argc=<optimized out>, > argv=0x7fffffffde48) at caped.c:171 > >
here is the open leak (gdb) where #0 0x000000366b436285 in raise () from /lib64/libc.so.6 #1 0x000000366b437b9b in abort () from /lib64/libc.so.6 #2 0x000000366b42ee9e in __assert_fail_base () from /lib64/libc.so.6 #3 0x000000366b42ef42 in __assert_fail () from /lib64/libc.so.6 #4 0x00007ffff7b56866 in _libssh2_packet_add (session=0x62eea0, data=0x66f6e0 "[", datalen=17, macstate=0) at packet.c:430 #5 0x00007ffff7b6a98b in fullpacket (session=0x62eea0, encrypted=1) at transport.c:236 #6 0x00007ffff7b6b20f in _libssh2_transport_read (session=0x62eea0) at transport.c:556 #7 0x00007ffff7b58288 in _libssh2_packet_requirev (session=0x62eea0, packet_types=0x7ffff7b70e15 "[\\", data=0x63bf80, data_len=0x63bf88, match_ofs=1, match_buf=0x62512c "", match_len=4, state=0x63bf60) at packet.c:1218 #8 0x00007ffff7b49aa9 in _libssh2_channel_open (session=0x62eea0, channel_type=0x408603 "session", channel_type_len=7, window_size=262144, packet_size=32768, message=0x0, message_len=0) at channel.c:230 #9 0x00007ffff7b49eec in libssh2_channel_open_ex (session=0x62eea0, type=0x408603 "session", type_len=7, window_size=262144, packet_size=32768, msg=0x0, msg_len=0) at channel.c:331 #10 0x0000000000405aa8 in assembly_ssh_exec (data=0x62a490) at trans_ssh.c:145 #11 0x00007ffff7d8b1eb in job_dispatch (item=0x654ad0, p=<optimized out>) at loop_job.c:39 #12 0x00007ffff7d89d37 in qb_loop_run_level (level=0x60ff30) at loop.c:45 #13 qb_loop_run (lp=<optimized out>) at loop.c:206 #14 0x0000000000403745 in main (argc=<optimized out>, argv=0x7fffffffde48) at caped.c:171 > It appears something related to how I'm using EOF seems to be causing > the problem. You mentioned in a different thread that call model I am > using may not work. What would you suggest? > > I'm not sure why there isn't a leak in the open path. Maybe valgrind ha > the stack trace recorded incorrectly? Or perhaps I am asserting on the > wrong iteration. I'll play with it more and see if I can find more > information. > > Regards > -steve > _______________________________________________ > libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel _______________________________________________ libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel