Re: [SR-Users] private memory leak - but where?

[Daniel-Constantin Mierla]

The situation is like declaring:

int a;
int b;
...
int ...

in a C program -- it will run out of memory as well. As said, I plan to
push a solution soon.

Cheers,
Daniel


On 20.09.17 15:35, Sebastian Damm wrote:
> Thanks Daniel,
>
> looks like that is indeed the case. We the hash table key is the
> call-id, but we're not calling $sht(foo=>$ci) but instead passing in
> the call-id as a string.
>
> Ich have now written a test script with those two functions:
>
> function handle_packet_bad()
>   callId = sr.pv.get("$ci")
>   if sr.pv.get("$sht(interceptedSessions=>"..callId..")") ~= nil then
> sr.dbg("Lua found matching packet")
>   end
> end
>
> function handle_packet_good()
>   callId = sr.pv.get("$ci")
>   sr.pv.sets("$var(htableKey)", callId)
>   if sr.pv.get("$sht(interceptedSessions=>$var(htableKey))") ~= nil then
> sr.dbg("Lua found matching packet")
>   end
> end
>
> Then in my Kamailio config, I call one or the other function depending
> on whether the PID is odd or even.
>
> if ($pp & 1) {
> if(!lua_run("handle_packet_bad")) {
> xlog("L_ERR", "SCRIPT: failed to execute lua
> function!\n");
> }
> } else {
> if(!lua_run("handle_packet_good")) {
> xlog("L_ERR", "SCRIPT: failed to execute lua
> function!\n");
> }
> }
>
> Now when I track the "real_mem" used by the UDP listeners, I see the
> difference. While the odd PIDs use more and more memory, the even ones
> stay about at the same level.
>
> I wouldn't have found that by myself. Thanks again.
>
> Best Regards,
> Sebastian
>
> On Wed, Sep 20, 2017 at 10:56 AM, Daniel-Constantin Mierla
>  wrote:
>> Hello,
>>
>> my guess is that you define many cfg variables from lua, which take from
>> private memory (their definition).
>>
>> So $sht(x=>abc) is defined when used first time and kept in memory. If
>> you use also $sht(x=>efg), this is another defined variable.
>>
>> This happens when you do KSR.pv.get("$sht(x=>abc)") or
>> KSR.pv.get("$sht(x=>efg)")
>>
>> To avoid defining a lot of variables, you can do:
>>
>> KSR.pv.sets("$var(n)", "abc");
>> KSR.pv.get("$sht(x=>$var(n)");
>> KSR.pv.sets("$var(n)", "efg");
>> KSR.pv.get("$sht(x=>$var(n)");
>>
>> The practically you define only one cfg variable,
>> KSR.pv.get("$sht(x=>$var(n)")
>>
>> The issue is known, there were other reporting it here on the mailing
>> list. It's in my to-do before 5.1 to fix it in a way or another, by
>> either setting a limit of defined variables and then throw error
>> (easier), or track when a variable is used and start deleting the
>> definition for older ones when the limit is exceeded.
>>
>> Cheers,
>> Daniel
>>
>>
>>
>> On 19.09.17 17:56, Sebastian Damm wrote:
>>> Hi Daniel,
>>>
>>> thanks for the quick response. See my other mail with more details.
>>>
>>> I don't use cfg variable from the lua script, I only access $shv(...)
>>> and $vn(...) or $var(...) and $sht(...) from it. And of course $tU,
>>> $hdr(...) and other builtin variables. Some of the variables I'm
>>> accessing might not be there always, though ($au for example).
>>>
>>> Anything I can help with? I only need about half an hour to reproduce it.
>>>
>>> Best Regards,
>>> Sebastian
>>>
>>> On Tue, Sep 19, 2017 at 5:46 PM, Daniel-Constantin Mierla
>>>  wrote:
 Hello,

 do you use kamailio.cfg variables in the lua script? If yes, can you
 show how you do it?

 If you have troubles with pkg dump, it should be printed when you
 stop/restart. It would be good to do the pkg summary, it is easier to
 see the used chunks as a report. There are global parameters to
 configure it.

 Cheers,
 Daniel

>>> ___
>>> Kamailio (SER) - Users Mailing List
>>> sr-users@lists.kamailio.org
>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>> --
>> Daniel-Constantin Mierla
>> www.twitter.com/miconda -- www.linkedin.com/in/miconda
>> Kamailio Advanced Training - www.asipto.com
>> Kamailio World Conference - www.kamailioworld.com
>>
> ___
> Kamailio (SER) - Users Mailing List
> sr-users@lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com


___
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

Re: [SR-Users] private memory leak - but where?

[Sebastian Damm]

Thanks Daniel,

looks like that is indeed the case. We the hash table key is the
call-id, but we're not calling $sht(foo=>$ci) but instead passing in
the call-id as a string.

Ich have now written a test script with those two functions:

function handle_packet_bad()
  callId = sr.pv.get("$ci")
  if sr.pv.get("$sht(interceptedSessions=>"..callId..")") ~= nil then
sr.dbg("Lua found matching packet")
  end
end

function handle_packet_good()
  callId = sr.pv.get("$ci")
  sr.pv.sets("$var(htableKey)", callId)
  if sr.pv.get("$sht(interceptedSessions=>$var(htableKey))") ~= nil then
sr.dbg("Lua found matching packet")
  end
end

Then in my Kamailio config, I call one or the other function depending
on whether the PID is odd or even.

if ($pp & 1) {
if(!lua_run("handle_packet_bad")) {
xlog("L_ERR", "SCRIPT: failed to execute lua
function!\n");
}
} else {
if(!lua_run("handle_packet_good")) {
xlog("L_ERR", "SCRIPT: failed to execute lua
function!\n");
}
}

Now when I track the "real_mem" used by the UDP listeners, I see the
difference. While the odd PIDs use more and more memory, the even ones
stay about at the same level.

I wouldn't have found that by myself. Thanks again.

Best Regards,
Sebastian

On Wed, Sep 20, 2017 at 10:56 AM, Daniel-Constantin Mierla
 wrote:
> Hello,
>
> my guess is that you define many cfg variables from lua, which take from
> private memory (their definition).
>
> So $sht(x=>abc) is defined when used first time and kept in memory. If
> you use also $sht(x=>efg), this is another defined variable.
>
> This happens when you do KSR.pv.get("$sht(x=>abc)") or
> KSR.pv.get("$sht(x=>efg)")
>
> To avoid defining a lot of variables, you can do:
>
> KSR.pv.sets("$var(n)", "abc");
> KSR.pv.get("$sht(x=>$var(n)");
> KSR.pv.sets("$var(n)", "efg");
> KSR.pv.get("$sht(x=>$var(n)");
>
> The practically you define only one cfg variable,
> KSR.pv.get("$sht(x=>$var(n)")
>
> The issue is known, there were other reporting it here on the mailing
> list. It's in my to-do before 5.1 to fix it in a way or another, by
> either setting a limit of defined variables and then throw error
> (easier), or track when a variable is used and start deleting the
> definition for older ones when the limit is exceeded.
>
> Cheers,
> Daniel
>
>
>
> On 19.09.17 17:56, Sebastian Damm wrote:
>> Hi Daniel,
>>
>> thanks for the quick response. See my other mail with more details.
>>
>> I don't use cfg variable from the lua script, I only access $shv(...)
>> and $vn(...) or $var(...) and $sht(...) from it. And of course $tU,
>> $hdr(...) and other builtin variables. Some of the variables I'm
>> accessing might not be there always, though ($au for example).
>>
>> Anything I can help with? I only need about half an hour to reproduce it.
>>
>> Best Regards,
>> Sebastian
>>
>> On Tue, Sep 19, 2017 at 5:46 PM, Daniel-Constantin Mierla
>>  wrote:
>>> Hello,
>>>
>>> do you use kamailio.cfg variables in the lua script? If yes, can you
>>> show how you do it?
>>>
>>> If you have troubles with pkg dump, it should be printed when you
>>> stop/restart. It would be good to do the pkg summary, it is easier to
>>> see the used chunks as a report. There are global parameters to
>>> configure it.
>>>
>>> Cheers,
>>> Daniel
>>>
>> ___
>> Kamailio (SER) - Users Mailing List
>> sr-users@lists.kamailio.org
>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
> --
> Daniel-Constantin Mierla
> www.twitter.com/miconda -- www.linkedin.com/in/miconda
> Kamailio Advanced Training - www.asipto.com
> Kamailio World Conference - www.kamailioworld.com
>

___
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

Re: [SR-Users] private memory leak - but where?

[Daniel-Constantin Mierla]

Hello,

my guess is that you define many cfg variables from lua, which take from
private memory (their definition).

So $sht(x=>abc) is defined when used first time and kept in memory. If
you use also $sht(x=>efg), this is another defined variable.

This happens when you do KSR.pv.get("$sht(x=>abc)") or
KSR.pv.get("$sht(x=>efg)")

To avoid defining a lot of variables, you can do:

KSR.pv.sets("$var(n)", "abc");
KSR.pv.get("$sht(x=>$var(n)");
KSR.pv.sets("$var(n)", "efg");
KSR.pv.get("$sht(x=>$var(n)");

The practically you define only one cfg variable,
KSR.pv.get("$sht(x=>$var(n)")

The issue is known, there were other reporting it here on the mailing
list. It's in my to-do before 5.1 to fix it in a way or another, by
either setting a limit of defined variables and then throw error
(easier), or track when a variable is used and start deleting the
definition for older ones when the limit is exceeded.

Cheers,
Daniel



On 19.09.17 17:56, Sebastian Damm wrote:
> Hi Daniel,
>
> thanks for the quick response. See my other mail with more details.
>
> I don't use cfg variable from the lua script, I only access $shv(...)
> and $vn(...) or $var(...) and $sht(...) from it. And of course $tU,
> $hdr(...) and other builtin variables. Some of the variables I'm
> accessing might not be there always, though ($au for example).
>
> Anything I can help with? I only need about half an hour to reproduce it.
>
> Best Regards,
> Sebastian
>
> On Tue, Sep 19, 2017 at 5:46 PM, Daniel-Constantin Mierla
>  wrote:
>> Hello,
>>
>> do you use kamailio.cfg variables in the lua script? If yes, can you
>> show how you do it?
>>
>> If you have troubles with pkg dump, it should be printed when you
>> stop/restart. It would be good to do the pkg summary, it is easier to
>> see the used chunks as a report. There are global parameters to
>> configure it.
>>
>> Cheers,
>> Daniel
>>
> ___
> Kamailio (SER) - Users Mailing List
> sr-users@lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

-- 
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - www.kamailioworld.com


___
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users

Re: [SR-Users] private memory leak - but where?

[Sebastian Damm]

Hi Daniel,

thanks for the quick response. See my other mail with more details.

I don't use cfg variable from the lua script, I only access $shv(...)
and $vn(...) or $var(...) and $sht(...) from it. And of course $tU,
$hdr(...) and other builtin variables. Some of the variables I'm
accessing might not be there always, though ($au for example).

Anything I can help with? I only need about half an hour to reproduce it.

Best Regards,
Sebastian

On Tue, Sep 19, 2017 at 5:46 PM, Daniel-Constantin Mierla
 wrote:
> Hello,
>
> do you use kamailio.cfg variables in the lua script? If yes, can you
> show how you do it?
>
> If you have troubles with pkg dump, it should be printed when you
> stop/restart. It would be good to do the pkg summary, it is easier to
> see the used chunks as a report. There are global parameters to
> configure it.
>
> Cheers,
> Daniel
>

___
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users