Re: [SR-Users] Dialog Module Hash Size vs Number of Dialogs
On Mon, Dec 04, 2017 at 03:28:08PM -0500, Mark Blackford wrote: > So, it sounds like from what you describe, I will set my hash size to > be a few times larger than my forecasted profile size to keep matching > operations to a minimal. And since dialogs timeout anyway if they are > not cleared by a proper BYE, a memory leak of uncleared dialogs is > highly unlikely. Yep. Just follow the same sizing practices that you would with any hash table generically in software work. -- Alex Balashov | Principal | Evariste Systems LLC Tel: +1-706-510-6800 / +1-800-250-5920 (toll-free) Web: http://www.evaristesys.com/, http://www.csrpswitch.com/ ___ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Dialog Module Hash Size vs Number of Dialogs
Hi Daniel,
Thanks for your responses. This discussion has been very useful.
Our code is along the lines that you suggest in which we are using some
database ops prior to the dialog calls to gather if the user is call
limited and if so, the provisioned users "max calls allowed" per billing
party. Then, we check their current active profile size:
get_profile_size("prepaid", "$avp(s:billing_party)", "$avp(profile_size)");
If they have available calls, we are using a dialog profile called
"prepaid" with a limit set per user (aka billing_party):
set_dlg_profile("prepaid", "$avp(s:billing_party)");
So, it sounds like from what you describe, I will set my hash size to be a
few times larger than my forecasted profile size to keep matching
operations to a minimal. And since dialogs timeout anyway if they are not
cleared by a proper BYE, a memory leak of uncleared dialogs is highly
unlikely.
Thank you for your assistance. Hopefully we will meet again soon, and I
can share how the solution turns out.
Mark Blackford
Digium Cloud Services
On Mon, Dec 4, 2017 at 2:56 PM, Daniel-Constantin Mierla
wrote:
> The hash_size has no relevance on how many dialogs can be handled. It is
> only related to the lookup performance. So even with hash_size=1 you can
> have same number of dialogs as with hash_size=4096 or higher.
>
> The difference is that with hash_size=1, all the dialogs are in a single
> linked list, so if you have 2000 active dialog, you may have the worse case
> that it requires 2000 matching operations to lookup a specific dialog.
>
> If the hash_size is 2048, the at 2000 active calls and a fair
> distribution, then looking up a specific dialog will be done in a single
> matching operation. Fair distribution cannot be ensure, but it is quite
> good, so typically the difference between the list loaded slot and most
> loaded slot is not that high. I expect that worse case situation won't
> require more that 5 matching operations on a hash table with 2048 slots
> when having 2000 active calls.
> To put limit on active calls, the simplest solution is to do a test on
> $stat(active_dialogs)>=limit then reject the initial invite:
>
> - https://www.kamailio.org/docs/modules/stable/modules/dialog.
> html#idp34022668
>
> You may need statistics module to get the $stat(...) var:
>
> - https://www.kamailio.org/docs/modules/stable/modules/statistics.html
>
> The alternative would be to set all active calls in a single dialog
> profile.
>
> - https://www.kamailio.org/docs/modules/stable/modules/dialog.
> html#dialog.f.set_dlg_profile
>
> Then get the size of the profile and test it against the limit.
>
> This is more suitable if you want to set limits per user, a.s.o., because
> you can group on profiles with a specific key (e.g., username).
>
> Cheers,
> Daniel
>
>
> On 04.12.17 19:28, Mark Blackford wrote:
>
> Thank you for your response!
>
> I am currently replacing the call_control module used with CDRTool Prepaid
> with the dialog module using hash tables. The speed and stability are
> tremendously better!
>
> My concern is two-fold:
> 1) I do not want calls to be blocked because I set the hash table size too
> small.
> 2) I do not want any memory usage problems that could cause kamailio
> stability issues.
>
> My understanding, or at least how I want to use the "hash_size" modparam,
> is that hash_size gives dialog a set memory space (aka slots and buckets)
> to store the dialogs being tracked. This is great since I "think" I will
> have constant memory consumption that will be a fixed amount and never
> "grow". However, I wanted to get a rough idea of how many dialogs
> (customer calls) I could track per kamailio instance to properly choose the
> hash size.
>
> I thought it would be easy to set the value to something very small in the
> lab and just run up the number of calls until kamailio hits an upper limit
> and fails to create any more dialogs.
> modparam("dialog", "hash_size", 32)
>
> However, I am able to create almost 2000 dialogs no matter what size I
> choose. After, reading the information in the link about hash tables and
> your response, I see how a larger table with more "hashing" increases
> lookup performance, but I still do not see how the total number of dialogs
> that the module tracks can be set so that the kamailio memory usage is
> protected. I still look at the hash_size as the key here and that there
> should be a relationship, perhaps rough at best, to the maximum number of
> dialogs supported in the table.
>
> Thanks so much for your time,
> Mark Blackord
> Digium Cloud Services
>
> On Sat, Dec 2, 2017 at 2:50 AM, Daniel-Constantin Mierla <
> mico...@gmail.com> wrote:
>
>> Hello,
>>
>> hash size does not set any limitation to the number of dialogs (active
>> calls), it has impact on searching the dialogs, so if you have a lot of
>> active calls, increasing the hash size might improve performances.
>> Effectively the hash size is used to compute the
Re: [SR-Users] Dialog Module Hash Size vs Number of Dialogs
The hash_size has no relevance on how many dialogs can be handled. It is
only related to the lookup performance. So even with hash_size=1 you can
have same number of dialogs as with hash_size=4096 or higher.
The difference is that with hash_size=1, all the dialogs are in a single
linked list, so if you have 2000 active dialog, you may have the worse
case that it requires 2000 matching operations to lookup a specific dialog.
If the hash_size is 2048, the at 2000 active calls and a fair
distribution, then looking up a specific dialog will be done in a single
matching operation. Fair distribution cannot be ensure, but it is quite
good, so typically the difference between the list loaded slot and most
loaded slot is not that high. I expect that worse case situation won't
require more that 5 matching operations on a hash table with 2048 slots
when having 2000 active calls.
To put limit on active calls, the simplest solution is to do a test on
$stat(active_dialogs)>=limit then reject the initial invite:
-
https://www.kamailio.org/docs/modules/stable/modules/dialog.html#idp34022668
You may need statistics module to get the $stat(...) var:
- https://www.kamailio.org/docs/modules/stable/modules/statistics.html
The alternative would be to set all active calls in a single dialog profile.
-
https://www.kamailio.org/docs/modules/stable/modules/dialog.html#dialog.f.set_dlg_profile
Then get the size of the profile and test it against the limit.
This is more suitable if you want to set limits per user, a.s.o.,
because you can group on profiles with a specific key (e.g., username).
Cheers,
Daniel
On 04.12.17 19:28, Mark Blackford wrote:
> Thank you for your response!
>
> I am currently replacing the call_control module used with CDRTool
> Prepaid with the dialog module using hash tables. The speed and
> stability are tremendously better!
>
> My concern is two-fold:
> 1) I do not want calls to be blocked because I set the hash table size
> too small.
> 2) I do not want any memory usage problems that could cause kamailio
> stability issues.
>
> My understanding, or at least how I want to use the "hash_size"
> modparam, is that hash_size gives dialog a set memory space (aka slots
> and buckets) to store the dialogs being tracked. This is great since
> I "think" I will have constant memory consumption that will be a fixed
> amount and never "grow". However, I wanted to get a rough idea of how
> many dialogs (customer calls) I could track per kamailio instance to
> properly choose the hash size.
>
> I thought it would be easy to set the value to something very small in
> the lab and just run up the number of calls until kamailio hits an
> upper limit and fails to create any more dialogs.
> modparam("dialog", "hash_size", 32)
>
> However, I am able to create almost 2000 dialogs no matter what size I
> choose. After, reading the information in the link about hash tables
> and your response, I see how a larger table with more "hashing"
> increases lookup performance, but I still do not see how the total
> number of dialogs that the module tracks can be set so that the
> kamailio memory usage is protected. I still look at the hash_size as
> the key here and that there should be a relationship, perhaps rough at
> best, to the maximum number of dialogs supported in the table.
>
> Thanks so much for your time,
> Mark Blackord
> Digium Cloud Services
>
> On Sat, Dec 2, 2017 at 2:50 AM, Daniel-Constantin Mierla
> > wrote:
>
> Hello,
>
> hash size does not set any limitation to the number of dialogs
> (active calls), it has impact on searching the dialogs, so if you
> have a lot of active calls, increasing the hash size might improve
> performances. Effectively the hash size is used to compute the
> number of slots (aka buckets) the hash table is going to have, see
> more technical details about hash tables at:
>
> - https://en.wikipedia.org/wiki/Hash_table
>
>
> Can you elaborate on your statement:
>
> > but even setting the hash size to a very tiny number does stop me from
> creating hundreds of dialogs
>
> Do you mean you are not able to create as many dialogs as you want?
>
> Cheers,
> Daniel
>
>
> On 01.12.17 19:54, Mark Blackford wrote:
>> Hello,
>>
>> I am trying to properly size the use of the Dialog Module hash
>> for our implementation using:
>>
>> modparam("dialog", "hash_size", )
>>
>> However, in my testing, I have been unable to figure out the
>> relationship between the hash size and a number of dialogs I need
>> to support. I think the hash size is specifying a memory block
>> in kB, but even setting the hash size to a very tiny number does
>> stop me from creating hundreds of dialogs.
>>
>> Is there a way to determine a relationship between the hash size
>> and a rough number of dialogs that would be
Re: [SR-Users] Dialog Module Hash Size vs Number of Dialogs
Thank you for your response!
I am currently replacing the call_control module used with CDRTool Prepaid
with the dialog module using hash tables. The speed and stability are
tremendously better!
My concern is two-fold:
1) I do not want calls to be blocked because I set the hash table size too
small.
2) I do not want any memory usage problems that could cause kamailio
stability issues.
My understanding, or at least how I want to use the "hash_size" modparam,
is that hash_size gives dialog a set memory space (aka slots and buckets)
to store the dialogs being tracked. This is great since I "think" I will
have constant memory consumption that will be a fixed amount and never
"grow". However, I wanted to get a rough idea of how many dialogs
(customer calls) I could track per kamailio instance to properly choose the
hash size.
I thought it would be easy to set the value to something very small in the
lab and just run up the number of calls until kamailio hits an upper limit
and fails to create any more dialogs.
modparam("dialog", "hash_size", 32)
However, I am able to create almost 2000 dialogs no matter what size I
choose. After, reading the information in the link about hash tables and
your response, I see how a larger table with more "hashing" increases
lookup performance, but I still do not see how the total number of dialogs
that the module tracks can be set so that the kamailio memory usage is
protected. I still look at the hash_size as the key here and that there
should be a relationship, perhaps rough at best, to the maximum number of
dialogs supported in the table.
Thanks so much for your time,
Mark Blackord
Digium Cloud Services
On Sat, Dec 2, 2017 at 2:50 AM, Daniel-Constantin Mierla
wrote:
> Hello,
>
> hash size does not set any limitation to the number of dialogs (active
> calls), it has impact on searching the dialogs, so if you have a lot of
> active calls, increasing the hash size might improve performances.
> Effectively the hash size is used to compute the number of slots (aka
> buckets) the hash table is going to have, see more technical details about
> hash tables at:
>
> - https://en.wikipedia.org/wiki/Hash_table
>
> Can you elaborate on your statement:
>
> > but even setting the hash size to a very tiny number does stop me from
> creating hundreds of dialogs
>
> Do you mean you are not able to create as many dialogs as you want?
>
> Cheers,
> Daniel
>
> On 01.12.17 19:54, Mark Blackford wrote:
>
> Hello,
>
> I am trying to properly size the use of the Dialog Module hash for our
> implementation using:
>
> modparam("dialog", "hash_size", )
>
> However, in my testing, I have been unable to figure out the relationship
> between the hash size and a number of dialogs I need to support. I think
> the hash size is specifying a memory block in kB, but even setting the hash
> size to a very tiny number does stop me from creating hundreds of dialogs.
>
> Is there a way to determine a relationship between the hash size and a
> rough number of dialogs that would be expected?
>
> An example of a a dialog looks like this from kamctl:
>
> [root@kamailio01 ~]# kamctl dialog show
> dialog memory records
> dialog:: hash=22:70
> state:: 4
> ref_count:: 2
> timestart:: 1512151205
> timeout:: 36083666
> callid:: 0gQAAC8WAAACBAAALxYAAClws2wyL8GE+CSgRY7HIhmg9ZUIISZad46ntOPng3i
> PIcLaxzLFaytRTI7M0Bzz0g--@10.155.8.40
> from_uri:: sip:b53667d44239457fbc94fc2f4c4e2...@sip.dcs-staging.net
> from_tag:: 10.155.8.40+1+689d7e5e+8fcf481a
> caller_contact:: sip:43f0ae1480846185e8803f21e9f2b7
> 21@10.155.8.40:5060;transport=udp
> caller_cseq:: 24115
> caller_route_set::
> caller_bind_addr:: udp:10.155.8.11:5060
> callee_bind_addr:: udp:10.155.8.11:5060
> to_uri:: sip:2052773...@sip.dcs-staging.net
> to_tag:: sip+1+bdcd0004+2038f37c
> callee_contact:: sip:ca2013e84f10348a1cc825c12562bd
> e7@10.155.8.40:5060;transport=udp
> callee_cseq:: 0
> callee_route_set::
>
>
>
> Thanks!
> --
> Mark Blackford
> Digium Cloud Services
> 678.230.8769
>
>
>
> ___
> Kamailio (SER) - Users Mailing
> Listsr-users@lists.kamailio.orghttps://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
>
>
> --
> Daniel-Constantin Mierlawww.twitter.com/miconda -- www.linkedin.com/in/miconda
> Kamailio Advanced Training - www.asipto.com
> Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com
>
>
--
Mark Blackford
Digium Cloud Services
678.230.8769
___
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
Re: [SR-Users] Dialog Module Hash Size vs Number of Dialogs
Hello,
hash size does not set any limitation to the number of dialogs (active
calls), it has impact on searching the dialogs, so if you have a lot of
active calls, increasing the hash size might improve performances.
Effectively the hash size is used to compute the number of slots (aka
buckets) the hash table is going to have, see more technical details
about hash tables at:
- https://en.wikipedia.org/wiki/Hash_table
Can you elaborate on your statement:
> but even setting the hash size to a very tiny number does stop me from
creating hundreds of dialogs
Do you mean you are not able to create as many dialogs as you want?
Cheers,
Daniel
On 01.12.17 19:54, Mark Blackford wrote:
> Hello,
>
> I am trying to properly size the use of the Dialog Module hash for our
> implementation using:
>
> modparam("dialog", "hash_size", )
>
> However, in my testing, I have been unable to figure out the
> relationship between the hash size and a number of dialogs I need to
> support. I think the hash size is specifying a memory block in kB,
> but even setting the hash size to a very tiny number does stop me from
> creating hundreds of dialogs.
>
> Is there a way to determine a relationship between the hash size and a
> rough number of dialogs that would be expected?
>
> An example of a a dialog looks like this from kamctl:
>
> [root@kamailio01 ~]# kamctl dialog show
> dialog memory records
> dialog:: hash=22:70
> state:: 4
> ref_count:: 2
> timestart:: 1512151205
> timeout:: 36083666
> callid::
> 0gQAAC8WAAACBAAALxYAAClws2wyL8GE+CSgRY7HIhmg9ZUIISZad46ntOPng3iPIcLaxzLFaytRTI7M0Bzz0g--@10.155.8.40
>
> from_uri:: sip:b53667d44239457fbc94fc2f4c4e2...@sip.dcs-staging.net
>
> from_tag:: 10.155.8.40+1+689d7e5e+8fcf481a
> caller_contact::
> sip:43f0ae1480846185e8803f21e9f2b721@10.155.8.40:5060;transport=udp
> caller_cseq:: 24115
> caller_route_set::
> caller_bind_addr:: udp:10.155.8.11:5060
> callee_bind_addr:: udp:10.155.8.11:5060
> to_uri:: sip:2052773...@sip.dcs-staging.net
>
> to_tag:: sip+1+bdcd0004+2038f37c
> callee_contact::
> sip:ca2013e84f10348a1cc825c12562bde7@10.155.8.40:5060;transport=udp
> callee_cseq:: 0
> callee_route_set::
>
>
>
> Thanks!
> --
> Mark Blackford
> Digium Cloud Services
> 678.230.8769
>
>
>
> ___
> Kamailio (SER) - Users Mailing List
> sr-users@lists.kamailio.org
> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
--
Daniel-Constantin Mierla
www.twitter.com/miconda -- www.linkedin.com/in/miconda
Kamailio Advanced Training - www.asipto.com
Kamailio World Conference - May 14-16, 2018 - www.kamailioworld.com
___
Kamailio (SER) - Users Mailing List
sr-users@lists.kamailio.org
https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users
