Re: [OpenSIPS-Devel] Memory leak in DR module, I think

2016-12-16 Thread Liviu Chircu
To apply the patch, simply place yourself in the 2.1 source dir, and run 
this command:


git apply <(base64 -d 

Re: [OpenSIPS-Devel] Memory leak in DR module, I think

2016-12-16 Thread John Quick
Glad I could be of help.
Please could you email me the diff on that file, routing.c, so I can fix it on 
my proxy servers. Thanks.

John Quick


-Original Message-
From: Devel [mailto:devel-boun...@lists.opensips.org] On Behalf Of Liviu Chircu
Sent: 16 December 2016 14:42
To: devel@lists.opensips.org
Subject: Re: [OpenSIPS-Devel] Memory leak in DR module, I think

That's right on the spot, John! It seems this issue had been caught a long time 
ago, but, unfortunately, the fix did not make its way into 2.1 until now, as I 
just backported it.

Thank you for all the help!

Best regards,

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

On 16.12.2016 16:11, John Quick wrote:
> For attention of Liviu Chircu,
>
> Thanks for your help with diagnosing memory leaks. I think I have now 
> found the culprit. The smoking gun!
>
> I kept increasing the syslog rate limiting value until it reached 
> 4 burst. This was still not enough, but I can see the one that is 
> frequent and it fits with the symptoms too.
> Here is a small sample from the end of the dump:
>
> 2016-12-16 13:59:30  13335. N  address=0x7f4be90020d8
> frag=0x7f4be90020a8 size=64 used=1
> 2016-12-16 13:59:30  alloc'd from drouting.c:
> route2_carrier(3046)
> 2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
> c0c0c0c0c0c0c0c0, abcdefedabcdefed
> 2016-12-16 13:59:30  13336. N  address=0x7f4be9002178
> frag=0x7f4be9002148 size=48 used=1
> 2016-12-16 13:59:30  alloc'd from drouting.c:
> route2_carrier(3046)
> 2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
> c0c0c0c0c0c0c0c0, abcdefedabcdefed
> 2016-12-16 13:59:30  13337. N  address=0x7f4be9002208
> frag=0x7f4be90021d8 size=40 used=1
> 2016-12-16 13:59:30  alloc'd from drouting.c:
> route2_carrier(3046)
> 2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
> c0c0c0c0c0c0c0c0, abcdefedabcdefed
>
> I use the DR module to select a route for calls through this proxy. 
> The memory leak is fastest when there are lots of calls during the 
> day, then it is slow at night.
> OpenSIPS Version 2.1.4
>
> John Quick
> Smartvox Limited
> Web: www.smartvox.co.uk
>
>
>
> ___
> Devel mailing list
> Devel@lists.opensips.org
> http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


Re: [OpenSIPS-Devel] Memory leak in DR module, I think

2016-12-16 Thread Liviu Chircu
That's right on the spot, John! It seems this issue had been caught a 
long time ago, but, unfortunately, the fix did not make its way into 2.1 
until now, as I just backported it.


Thank you for all the help!

Best regards,

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

On 16.12.2016 16:11, John Quick wrote:

For attention of Liviu Chircu,

Thanks for your help with diagnosing memory leaks. I think I have now found
the culprit. The smoking gun!

I kept increasing the syslog rate limiting value until it reached 4
burst. This was still not enough, but I can see the one that is frequent and
it fits with the symptoms too.
Here is a small sample from the end of the dump:

2016-12-16 13:59:30  13335. N  address=0x7f4be90020d8
frag=0x7f4be90020a8 size=64 used=1
2016-12-16 13:59:30  alloc'd from drouting.c:
route2_carrier(3046)
2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
2016-12-16 13:59:30  13336. N  address=0x7f4be9002178
frag=0x7f4be9002148 size=48 used=1
2016-12-16 13:59:30  alloc'd from drouting.c:
route2_carrier(3046)
2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
2016-12-16 13:59:30  13337. N  address=0x7f4be9002208
frag=0x7f4be90021d8 size=40 used=1
2016-12-16 13:59:30  alloc'd from drouting.c:
route2_carrier(3046)
2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed

I use the DR module to select a route for calls through this proxy. The
memory leak is fastest when there are lots of calls during the day, then it
is slow at night.
OpenSIPS Version 2.1.4

John Quick
Smartvox Limited
Web: www.smartvox.co.uk



___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel



___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


[OpenSIPS-Devel] [OpenSIPS/opensips] 1cd594: Fix PKG mem leak

2016-12-16 Thread Vlad Paiu
  Branch: refs/heads/2.1
  Home:   https://github.com/OpenSIPS/opensips
  Commit: 1cd594ccfe87f023d79f6a0145172d3096d4b586
  
https://github.com/OpenSIPS/opensips/commit/1cd594ccfe87f023d79f6a0145172d3096d4b586
  Author: Vlad Paiu 
  Date:   2016-12-16 (Fri, 16 Dec 2016)

  Changed paths:
M modules/drouting/drouting.c

  Log Message:
  ---
  Fix PKG mem leak

(cherry picked from commit f276691d05c56748c628fc9aef9e2700aff8a536)


___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


[OpenSIPS-Devel] [OpenSIPS/opensips] 20ebdb: mem/module_info: adapt code to shared memory initi...

2016-12-16 Thread Razvan Crainea
  Branch: refs/heads/master
  Home:   https://github.com/OpenSIPS/opensips
  Commit: 20ebdb0597e958cfa1b998d3d5a98f6f26e13a98
  
https://github.com/OpenSIPS/opensips/commit/20ebdb0597e958cfa1b998d3d5a98f6f26e13a98
  Author: Cerghit Ionel 
  Date:   2016-06-16 (Thu, 16 Jun 2016)

  Changed paths:
M cfg.y
M mem/module_info.c
M mem/module_info.h
M mem/shm_mem.c

  Log Message:
  ---
  mem/module_info: adapt code to shared memory initialisation change

the statistics vector is exteded as more memory groups are declared
if SHM_SHOW_DEFAULT_GROUP is defined allocations done util grups are
defined are added to the default group


  Commit: db310d8b8c3efd62ccd0b7f7de4d142a366dd55b
  
https://github.com/OpenSIPS/opensips/commit/db310d8b8c3efd62ccd0b7f7de4d142a366dd55b
  Author: Cerghit Ionel 
  Date:   2016-06-17 (Fri, 17 Jun 2016)

  Changed paths:
M Makefile.conf.template
M mem/module_info.c

  Log Message:
  ---
  module_info.c: change obsolete flag


  Commit: e9c3b44bfb326441b60c4bc003228c75ce7bf27f
  
https://github.com/OpenSIPS/opensips/commit/e9c3b44bfb326441b60c4bc003228c75ce7bf27f
  Author: Cerghit Ionel 
  Date:   2016-12-09 (Fri, 09 Dec 2016)

  Changed paths:
M mem/module_info.c
M mem/module_info.h
M mem/shm_mem.c

  Log Message:
  ---
  module_info.c: change group reallocating policy


  Commit: 130d3447426d3ae3ce27d099633042c9876018e3
  
https://github.com/OpenSIPS/opensips/commit/130d3447426d3ae3ce27d099633042c9876018e3
  Author: Razvan Crainea 
  Date:   2016-12-16 (Fri, 16 Dec 2016)

  Changed paths:
M Makefile.conf.template
M cfg.y
M mem/module_info.c
M mem/module_info.h
M mem/shm_mem.c

  Log Message:
  ---
  Merge branch 'module-memory-rework' of 
https://github.com/ionel-cerghit/opensips into 
ionel-cerghit-module-memory-rework


  Commit: fe5d9763845aa05972de130a44ce78d56d352e26
  
https://github.com/OpenSIPS/opensips/commit/fe5d9763845aa05972de130a44ce78d56d352e26
  Author: Razvan Crainea 
  Date:   2016-12-16 (Fri, 16 Dec 2016)

  Changed paths:
M Makefile.conf.template
M mem/module_info.c

  Log Message:
  ---
  memgroups: decrease INFO to DEBUG


  Commit: a471509dd5e9fe63e2af59c2427577f0a29cf4a1
  
https://github.com/OpenSIPS/opensips/commit/a471509dd5e9fe63e2af59c2427577f0a29cf4a1
  Author: Razvan Crainea 
  Date:   2016-12-16 (Fri, 16 Dec 2016)

  Changed paths:
M Makefile.conf.template
M cfg.y
M mem/module_info.c
M mem/module_info.h
M mem/shm_mem.c

  Log Message:
  ---
  Merge branch 'ionel-cerghit-module-memory-rework'


Compare: 
https://github.com/OpenSIPS/opensips/compare/c2c496481c50...a471509dd5e9___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


[OpenSIPS-Devel] Memory leak in DR module, I think

2016-12-16 Thread John Quick
For attention of Liviu Chircu,

Thanks for your help with diagnosing memory leaks. I think I have now found
the culprit. The smoking gun!

I kept increasing the syslog rate limiting value until it reached 4
burst. This was still not enough, but I can see the one that is frequent and
it fits with the symptoms too.
Here is a small sample from the end of the dump:

2016-12-16 13:59:30  13335. N  address=0x7f4be90020d8
frag=0x7f4be90020a8 size=64 used=1
2016-12-16 13:59:30  alloc'd from drouting.c:
route2_carrier(3046)
2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
2016-12-16 13:59:30  13336. N  address=0x7f4be9002178
frag=0x7f4be9002148 size=48 used=1
2016-12-16 13:59:30  alloc'd from drouting.c:
route2_carrier(3046)
2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
2016-12-16 13:59:30  13337. N  address=0x7f4be9002208
frag=0x7f4be90021d8 size=40 used=1
2016-12-16 13:59:30  alloc'd from drouting.c:
route2_carrier(3046)
2016-12-16 13:59:30  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed

I use the DR module to select a route for calls through this proxy. The
memory leak is fastest when there are lots of calls during the day, then it
is slow at night.
OpenSIPS Version 2.1.4

John Quick
Smartvox Limited
Web: www.smartvox.co.uk



___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


[OpenSIPS-Devel] [OpenSIPS/opensips]

2016-12-16 Thread liviuchircu
  Branch: refs/heads/mid-registrar
  Home:   https://github.com/OpenSIPS/opensips
___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


[OpenSIPS-Devel] [OpenSIPS/opensips] 8ec91e: usrloc API: Add a loading function

2016-12-16 Thread Liviu Chircu
  Branch: refs/heads/master
  Home:   https://github.com/OpenSIPS/opensips
  Commit: 8ec91e53521119d0de952367a51fdfe4a8037960
  
https://github.com/OpenSIPS/opensips/commit/8ec91e53521119d0de952367a51fdfe4a8037960
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/usrloc/usrloc.h

  Log Message:
  ---
  usrloc API: Add a loading function


  Commit: 21fff6ba5486aebdc3d1c8247786c0c375cdba14
  
https://github.com/OpenSIPS/opensips/commit/21fff6ba5486aebdc3d1c8247786c0c375cdba14
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
A modules/mid_registrar/Makefile
A modules/mid_registrar/mid_registrar.c
A modules/mid_registrar/uac_timer.c
A modules/mid_registrar/uac_timer.h

  Log Message:
  ---
  mid_registrar: initial version (PoC phase)

* basic modparam outline
* basic script exports
* un-sorted dump of code
* tm/usrloc/signaling hooks
*   + registered callbacks
* basic timer routine outline


  Commit: 5117e368ff1f013ac7cace4116baad1b6465c13c
  
https://github.com/OpenSIPS/opensips/commit/5117e368ff1f013ac7cace4116baad1b6465c13c
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/mid_registrar/mid_registrar.c
M modules/mid_registrar/uac_timer.c
M modules/mid_registrar/uac_timer.h

  Log Message:
  ---
  mid_registrar: Fix a series of bugs

* mode = 0 working
* mode = 1 almost working, needs some tweaks


  Commit: ca8b61fe616910a40e213d9afb47bc6b9fc62923
  
https://github.com/OpenSIPS/opensips/commit/ca8b61fe616910a40e213d9afb47bc6b9fc62923
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/usrloc/ucontact.c
M modules/usrloc/ucontact.h

  Log Message:
  ---
  usrloc: Add expires value to contact struct


  Commit: da1a96441f13c5556249890521b41efcc5122a7b
  
https://github.com/OpenSIPS/opensips/commit/da1a96441f13c5556249890521b41efcc5122a7b
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/usrloc/ucontact.c
M modules/usrloc/ucontact.h

  Log Message:
  ---
  usrloc: Add "expires_out" value to contact struct

Alpha phase. Has no restart persistency.


  Commit: aa94f8ba89f3e631ac40e0adf88ce2244134afeb
  
https://github.com/OpenSIPS/opensips/commit/aa94f8ba89f3e631ac40e0adf88ce2244134afeb
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/mid_registrar/mid_registrar.c
M modules/mid_registrar/uac_timer.c

  Log Message:
  ---
  mid_registrar: Working version of contact throttling

REGISTER requests are now properly absorbed by the mid-registrar, which
will internally generate a 200 OK reply when the mid_registrar_save() script
function is called.


  Commit: 5f5d14654948c0254b15d7dc05ce32b462b5e231
  
https://github.com/OpenSIPS/opensips/commit/5f5d14654948c0254b15d7dc05ce32b462b5e231
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/mid_registrar/mid_registrar.c

  Log Message:
  ---
  mid_registrar: Fix 200 OK replies for absorbed registrations

Properly include a 'Contact' header field, listing all bindings


  Commit: 6cb240cca71a48ad049c19ffac0a21a3b7d0ab93
  
https://github.com/OpenSIPS/opensips/commit/6cb240cca71a48ad049c19ffac0a21a3b7d0ab93
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/dispatcher/dispatch.c
M modules/drouting/drouting.c
M modules/tm/dlg.c
M modules/tm/dlg.h

  Log Message:
  ---
  tm API: Add Call-ID parameter when building requests


  Commit: 95a39a0bcf4c053ea09f3f54c39d40a5c7b0103b
  
https://github.com/OpenSIPS/opensips/commit/95a39a0bcf4c053ea09f3f54c39d40a5c7b0103b
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M Makefile.sources
A lib/path.c
A lib/path.h
M modules/path/path.c

  Log Message:
  ---
  path: Code refactoring

Make "Path" append code reusable by moving it into lib/path.c


  Commit: 82b47c0cbb49d603543f49078e1700b43eab6b3c
  
https://github.com/OpenSIPS/opensips/commit/82b47c0cbb49d603543f49078e1700b43eab6b3c
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/mid_registrar/uac_timer.c

  Log Message:
  ---
  mid_registrar: Forward Register requests if not in queue

De-Registrations may lead to a state where contacts are present on
the mid-registrar, but no timer is ticking for the outgoing
registration.

Should we run into this issue, simply forward the registration and
re-initialize the timer.


  Commit: 

[OpenSIPS-Devel] [OpenSIPS/opensips] 70d303: mid_registrar: Avoid redundant del lumps

2016-12-16 Thread Liviu Chircu
  Branch: refs/heads/mid-registrar
  Home:   https://github.com/OpenSIPS/opensips
  Commit: 70d303edb92cad80cd558dd67ff64c90b0f7249e
  
https://github.com/OpenSIPS/opensips/commit/70d303edb92cad80cd558dd67ff64c90b0f7249e
  Author: Liviu Chircu 
  Date:   2016-12-05 (Mon, 05 Dec 2016)

  Changed paths:
M modules/mid_registrar/save.c

  Log Message:
  ---
  mid_registrar: Avoid redundant del lumps


  Commit: e35ab76a4b3ae79f4ace7cc2ad9639a22dcec4ab
  
https://github.com/OpenSIPS/opensips/commit/e35ab76a4b3ae79f4ace7cc2ad9639a22dcec4ab
  Author: Liviu Chircu 
  Date:   2016-12-16 (Fri, 16 Dec 2016)

  Changed paths:
M modules/mid_registrar/lookup.c
M modules/mid_registrar/mid_registrar.c
M modules/mid_registrar/mid_registrar.h
M modules/mid_registrar/save.c

  Log Message:
  ---
  mid_registrar: rename "routing_mode" to "insertion_mode"


  Commit: 9ace8d31115dca623d5f9a60c92e292148565c86
  
https://github.com/OpenSIPS/opensips/commit/9ace8d31115dca623d5f9a60c92e292148565c86
  Author: Liviu Chircu 
  Date:   2016-12-16 (Fri, 16 Dec 2016)

  Changed paths:
M modules/mid_registrar/README
M modules/mid_registrar/doc/mid_registrar_admin.xml

  Log Message:
  ---
  mid_registrar: Update doc


Compare: 
https://github.com/OpenSIPS/opensips/compare/8d5c0f7e9321...9ace8d31115d___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel


Re: [OpenSIPS-Devel] Finding a memory leak

2016-12-16 Thread Liviu Chircu

Hi John,

Memory leak troubleshooting is more difficult on OpenSIPS versions up 
to, and including, 2.1. Starting from version 2.2, we have improved the 
output summary in such a way that leaks are immediately visible, with no 
additional work on behalf of the user!


Now, it's still possible to spot the leak with your current version, we 
only need to work a bit more. Useful steps:


* if needed, make sure any syslog rate-limiting is disabled. These dumps 
can be quite large!


* use "opensipsctl fifo ps" to find the PID of each OpenSIPS process you 
may want to send the "kill -SIGUSR1" command


* "pv_add_extra(4480)" points to an allocation at line 4480, of the 
"pv_add_extra()" function


* if there were a leak at "pv_add_extra(4480)", then periodical dumps of 
a single process will contain more and more lines indicating 
"pv_add_extra(4480)". You may need to do some output text manipulation 
magic in order to sum up all this information, for each dump.


Best regards,

Liviu Chircu
OpenSIPS Developer
http://www.opensips-solutions.com

On 15.12.2016 18:20, John Quick wrote:

Please can someone provide advice on interpreting OpenSIPS memory dumps and
identifying the cause of a memory leak.

A Proxy application I wrote, running v2.1.4 with DBG_QM_MALLOC enabled, is
leaking package memory so badly that I have to restart the service once a
week.

When I use this command:
 opensipsctl fifo get_statistics pkmem: | grep max
...it lists 57 lines of output. Most of these show a similar value from day
to day, but there are three instances near the middle of the list that
increase from one day to the next. For example:
pkmem:0-max_used_size:: 1359072
pkmem:1-max_used_size:: 1427568
pkmem:2-max_used_size:: 1463256
...
pkmem:23-max_used_size:: 3013848
pkmem:24-max_used_size:: 2541240
pkmem:25-max_used_size:: 3799672
...
pkmem:55-max_used_size:: 1420112
pkmem:56-max_used_size:: 1422104
pkmem:57-max_used_size:: 1396384

When I trigger a memory report (dump) using the command
 kill -SIGUSR1 971
...the resulting output in my opensips.log file looks almost the same from
one day top the next.

Do I need to use a different Process ID so I can get a memory dump for the
pkmem:25 child thread?
If so, how do I know which PID to use?  The command "ps ax | grep opensips"
returns a list of 59 process ID's.
Even if I get the right PID, what am I looking for in the memory dump?
Typical memory dump output looks like this:
2016-12-15 15:54:44   41. N  address=0x7f4be8db9048 frag=0x7f4be8db9018
size=48 used=1
2016-12-15 15:54:44  alloc'd from sr_module.c:
register_module(151)
2016-12-15 15:54:44  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
2016-12-15 15:54:44   42. N  address=0x7f4be8db90d8 frag=0x7f4be8db90a8
size=128 used=1
2016-12-15 15:54:44  alloc'd from cfg.lex: addstr(925)
2016-12-15 15:54:44  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
...
2016-12-15 15:54:44   63. N  address=0x7f4be8db9f70 frag=0x7f4be8db9f40
size=80 used=1
2016-12-15 15:54:44  alloc'd from pvar.c: pv_add_extra(4480)
2016-12-15 15:54:44  start check=f0f0f0f0f0f0f0f0, end check=
c0c0c0c0c0c0c0c0, abcdefedabcdefed
2016-12-15 15:54:44   64. N  address=0x7f4be8dba020 frag=0x7f4be8db9ff0
size=80 used=1
2016-12-15 15:54:44  alloc'd from pvar.c: pv_add_extra(4480)

I have not found any cases where the value for "frag size" exceeds 240. The
line that starts "alloc'd from" never seems to have a value in parentheses
exceeding 4480.
I cannot see anything that looks like a smoking gun.  The list never goes
beyond item 64.

John Quick
Smartvox Limited
Tel:   +44-1727-221221



___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel



___
Devel mailing list
Devel@lists.opensips.org
http://lists.opensips.org/cgi-bin/mailman/listinfo/devel