Bugs item #3557213, was opened at 2012-08-13 22:38
Message generated for change (Comment added) made by rrb3942
You can respond by visiting:
https://sourceforge.net/tracker/?func=detailatid=1086410aid=3557213group_id=232389
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: modules
Group: 1.8.x
Status: Open
Resolution: None
Priority: 8
Private: No
Submitted By: Ryan Bullock (rrb3942)
Assigned to: Bogdan-Andrei Iancu (bogdan_iancu)
Summary: Stuck Transactions/Memory Leak with B2B top hiding
Initial Comment:
I am experiencing a memory leak when using the B2B modules with the internal
top hiding scenario. It appears to be related to struck/un-freed
transactions, as the opensips mi will report many thousands of
inuse_transactions when the reality is only a handful. An example is we
currently show 15000+ inuse_transactions while only a handful of calls are
actually in progress. These transactions appear to build up over time.
Opensips information:
version: opensips 1.8.0-notls (x86_64/linux)
flags: STATS: Off, USE_IPV6, USE_TCP, DISABLE_NAGLE, USE_MCAST, SHM_MEM,
SHM_MMAP, PKG_MALLOC, DBG_QM_MALLOC, FAST_LOCK-ADAPTIVE_WAIT
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144, MAX_LISTEN 16,
MAX_URI_SIZE 1024, BUF_SIZE 65535
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
svnrevision: 2:9164M
@(#) $Id: main.c 8772 2012-03-08 11:16:13Z bogdan_iancu $
main.c compiled on 12:24:36 Aug 3 2012 with gcc 4.4.6
I currently have the memdump option enabled and am waiting for another buildup
of transactions to gather the memdump information. I can supply access to the
dump and the opensips configuration file via email.
--
Comment By: Ryan Bullock (rrb3942)
Date: 2012-09-24 09:27
Message:
Hey Bogdan,
I tried the patch, but the compiler complained that tm might be
uninitialized and it caused a segfault when I tried to run it. I have the
core dump if you would like a backtrace.
--
Comment By: Bogdan-Andrei Iancu (bogdan_iancu)
Date: 2012-09-19 08:50
Message:
Just uploaded a patch here - it is not a fix, but rather something to help
me with the fix - trying to collect more info.
Please apply this patch, run your b2b and send me logs (debug 3 is ok). I'm
looking for messages on CRITICAL level with Transaction not replied +
any previous messages from the same process.
Thanks and regards,
--
Comment By: Jim OBrien (jimdoesvoip)
Date: 2012-08-20 01:31
Message:
I may be seeing this same behavior as well. I had thought this was related
to TLS / many client sessions all connecting at once as things failed over
from one server to another. Are you both running TCP? I can see that Ryan
is not running TLS from the description above. We use opensips as a proxy;
not b2bua, it is doing far end NAT...
The symptom that I see is that as tm:UAS_transactions increases, so does
tm:inuse_transactions, following it very closely. Eventually when this
happens, the processes run out of memory / things go bad.
--
Comment By: Ryan Bullock (rrb3942)
Date: 2012-08-14 21:44
Message:
Config file was sent to both Vlad and Bogdan via email earlier today and I
just sent login information to obtain the memdump as well. If you did not
receive the emails please let me know and I can try re-sending.
Thanks for the help.
Regards,
Ryan
--
Comment By: Nick Altmann (nikbyte)
Date: 2012-08-14 09:56
Message:
We use the 'b2bl_from_spec_param'.
I use it too.
We are storing the b2b states in a database.
I store it in pgsql too.
We do modify some headers in the local_route.
I also modify headers in local_route
We do sequential routing through the b2b, as in we have a proxy that
hunts and sends its requests through the b2b2.
I'm too.
--
Comment By: Ryan Bullock (rrb3942)
Date: 2012-08-14 09:42
Message:
Only differences I can think of:
We use the 'b2bl_from_spec_param'.
We are storing the b2b states in a database.
We do modify some headers in the local_route.
We do sequential routing through the b2b, as in we have a proxy that hunts
and sends its requests through the b2b2.
I will send you the configuration shortly, it is pretty straight forward.
I will not have the memdump until tonight, when I can force the dump out.
Once I have it I will post it someplace where I can get you access and send
that via email.
Is their anything else I can get that would help with troubleshooting this?
--
Comment By: Nick Altmann
