Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#910)

2017-01-04 Thread Julien Chavanton
Closed #910. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/910#event-911343468___ sr-dev mailing list sr-dev@lists.sip-router.org

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#911)

2017-01-04 Thread Julien Chavanton
jchavanton commented on this pull request. > @@ -72,6 +70,7 @@ static int add_contact(str aor, ucontact_info_t* ci) // Search by ruid, if possible res = dmq_ul.get_urecord_by_ruid(_d, dmq_ul.get_aorhash(), >ruid, , ); `get_urecord_by_ruid` was also acquiring

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#911)

2017-01-04 Thread Julien Chavanton
While running the CI tests Kamctl / MI fifo was hanging, after further investigation it seems it was trying to acquire a lock while the DMK module was stuck using it. Processes ``` Process:: ID=0 PID=6 Type=main process - attendant Process:: ID=1 PID=168 Type=udp receiver child=0

[sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#911)

2017-01-04 Thread Julien Chavanton
caused by acquiring the same lock twice using get_urecord_by_ruid and lock_udomain You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/911 -- Commit Summary -- * dmq_usrloc: fix deadlock -- File Changes -- M

[sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#910)

2017-01-04 Thread Julien Chavanton
caused by acquiring the same lock twice using get_urecord_by_ruid and lock_udomain You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/910 -- Commit Summary -- * dmq_usrloc: fix deadlock caused by acquiring -- File Changes -- M

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#911)

2017-01-05 Thread Julien Chavanton
jchavanton commented on this pull request. > @@ -72,6 +70,7 @@ static int add_contact(str aor, ucontact_info_t* ci) // Search by ruid, if possible res = dmq_ul.get_urecord_by_ruid(_d, dmq_ul.get_aorhash(), >ruid, , ); if (res == 0) { +

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#911)

2017-01-05 Thread Julien Chavanton
@jchavanton pushed 1 commit. 309873e dmq_usrloc: get_urecord_by_ruid returns -- You are receiving this because you are subscribed to this thread. View it on GitHub:

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: fix deadlock (#911)

2017-01-04 Thread Julien Chavanton
The same process is requesting the lock a second time before unlocking. `futex_get()` is never able to acquire the lock and does not return, I tested this in a very minimalist program using `pthread_mutex_lock()` and it does get stuck. -- You are receiving this because you are subscribed to

Re: [sr-dev] deadlock in dmq_usrloc/usrloc_sync.c

2016-12-27 Thread Julien Chavanton
Sorry first patch was missing a lock that was required On Tue, Dec 27, 2016 at 11:19 AM, Julien Chavanton <jchavan...@gmail.com> wrote: > Hi Folks, > > We faced a deadlock caused by acquiring the same lock twice in > src/modules/dmq_usrloc/usrloc_sync.c > >

[sr-dev] deadlock in dmq_usrloc/usrloc_sync.c

2016-12-27 Thread Julien Chavanton
Hi Folks, We faced a deadlock caused by acquiring the same lock twice in src/modules/dmq_usrloc/usrloc_sync.c dmq_ul.lock_udomain(_d, ); >> lock_get(_d->table[sl].lock); get_urecord_by_ruid >> lock_ulslot(_d, sl); lock_get(_d->table[i].lock); The attached fix will not lock until

[sr-dev] [kamailio/kamailio] db_postgres insert_update and usrloc params (#1044)

2017-03-27 Thread Julien Chavanton
This pull request is including two commits 1: adding `db_postgres` insert_update support 2: adding new parameters to `usrloc` that are not changing the default behaviour. I added both to the same PR to highlight that the usage of insert_update will be immediately available for `usrloc` with

Re: [sr-dev] [kamailio/kamailio] db_postgres insert_update and usrloc params (#1044)

2017-03-27 Thread Julien Chavanton
`db_load` is unrelated, it is useful when using `dmq_usrloc` sync because some contacts are duplicated in memory so in such case loading from the DB is not useful. `db_load` by default will not change anything but if set to 0 will not load registers from the DB on start up. I did not want to

[sr-dev] git:master:5d14f5ca: usrloc: adding params

2017-03-30 Thread Julien Chavanton
Module: kamailio Branch: master Commit: 5d14f5cada8d4bc43bfa1865ca95b0fd2213f8f3 URL: https://github.com/kamailio/kamailio/commit/5d14f5cada8d4bc43bfa1865ca95b0fd2213f8f3 Author: Julien Chavanton <jchavan...@gmail.com> Committer: Julien Chavanton <jchavan...@gmail.com> Date: 2017-03

[sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO UPDATE (#1047)

2017-03-29 Thread Julien Chavanton
You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/1047 -- Commit Summary -- * db_postgress: insert_update() with DO UPDATE -- File Changes -- M src/modules/db_postgres/db_postgres.c (1) M

Re: [sr-dev] [kamailio/kamailio] db_postgres insert_update and usrloc params (#1044)

2017-03-29 Thread Julien Chavanton
Closed #1044. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/1044#event-1020622613___ sr-dev mailing list sr-dev@lists.sip-router.org

Re: [sr-dev] [kamailio/kamailio] db_postgres insert_update and usrloc params (#1044)

2017-03-29 Thread Julien Chavanton
ok I will create two and propose a solution for db_mode=4 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/1044#issuecomment-290121145___

Re: [sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO NOTHING (#1039)

2017-03-22 Thread Julien Chavanton
I am working on OPTION2, where not modification is required outside of db_postgres, if you think this is not an option or not valuable enough let me know ? If you do not want to merge DO NOTHING because it can be confusing / misleading then we can clause this PR. -- You are receiving this

Re: [sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO NOTHING (#1039)

2017-03-23 Thread Julien Chavanton
Closed #1039. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/1039#event-1012500837___ sr-dev mailing list sr-dev@lists.sip-router.org

[sr-dev] [kamailio/kamailio] usrloc: adding params (#1048)

2017-03-29 Thread Julien Chavanton
db_load: enable/disable loading from the database on mod_init (except when in db_mode 4 /READ_ONLY) db_insert_update: insert into table, update on duplicate key You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/1048 -- Commit Summary --

[sr-dev] git:master:88ba02fd: db_postgress: insert_update() with DO UPDATE

2017-03-29 Thread Julien Chavanton
Module: kamailio Branch: master Commit: 88ba02fd42403844afcdde82034e64a460dc1082 URL: https://github.com/kamailio/kamailio/commit/88ba02fd42403844afcdde82034e64a460dc1082 Author: Julien Chavanton <jchavan...@gmail.com> Committer: Julien Chavanton <jchavan...@gmail.com> Date: 2017-03

[sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO NOTHING (#1039)

2017-03-18 Thread Julien Chavanton
Why is insert_update doing nothing in Kamailio db insert_update ? As explained in the following article the design of "UPSERT" in PostgreSQL requires to be explicit about the constraint on which we accept to do update modification to Kamailio database framework/API would be required to expose

Re: [sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO NOTHING (#1039)

2017-03-20 Thread Julien Chavanton
The most strati forward solution would probably be changing the function prototype : ```/** * \brief Insert a row into specified table, update on duplicate key. * * The function implements the INSERT ON DUPLICATE KEY

Re: [sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO NOTHING (#1039)

2017-03-20 Thread Julien Chavanton
There is another option to do it directly in db_postgres constraints "auto detection" https://www.postgresql.org/docs/9.1/static/catalog-pg-constraint.html We would search the catalog : ``` select conntype from pg_constraint where conrelid = (select oid from pg_class where relname like

Re: [sr-dev] [kamailio/kamailio] db_postgress: insert_update() with DO NOTHING (#1039)

2017-03-20 Thread Julien Chavanton
This would require modifications to ``` ~/git/kamailio/src$ grep -R --include="*.c" insert_update * the DB base framework and 4 implementations : db_mysql, db_cassandra, db_mysql, db_cluster lib/srdb1/db.c: if (dbf->insert_update) { lib/srdb1/db.c: dbf.insert_update =

[sr-dev] git:4.4:d782f0d0: dmq_usrloc: Use ruid for saving/updating contacts

2017-03-14 Thread Julien Chavanton
Module: kamailio Branch: 4.4 Commit: d782f0d052f7092576b4935987b9ee25beb0f771 URL: https://github.com/kamailio/kamailio/commit/d782f0d052f7092576b4935987b9ee25beb0f771 Author: Carsten Bock <cars...@ng-voice.com> Committer: Julien Chavanton <jchavan...@gmail.com> Date: 2017-03-10T13

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-04 Thread Julien Chavanton
This is now deployed on cluster running on AWS, fragmentation is taking place and everything is performing much better, no more transactions storm. OK to update the readme, changing the example to 50 contacts and adding a comment about considering 1024 Bytes / contact in order to stay bellow

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-04 Thread Julien Chavanton
I think, I should add a check for maximum packet length. to make sure we will never try to send a datagram larger than 60K. Next step, we could add compression :) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-04 Thread Julien Chavanton
@jchavanton pushed 1 commit. 1226c0f dmq_usrloc: readme batch_msg_contacts -- You are receiving this because you are subscribed to this thread. View it on GitHub:

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-04 Thread Julien Chavanton
The overhead per contact is 188 chars, this is quite a lot, we may select an alternate format later. `1,:{"action":,"aor":"","ruid":"","c":"","received":"","path":"","callid":"","user_agent":"","instance":"","expires":,"cseq":,"flags":,"cflags":,"q":,"last_modified":,"methods":,"reg_id":},` I

[sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-03 Thread Julien Chavanton
This was tested with 100K and 4 servers, when restarting a server we get a full sync from the 3 nodes in a few seconds : ```show registrations: first x "usrloc:location-contacts = 10", show registrations: second x "usrloc:location-contacts = 10", show registrations: third x

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-05 Thread Julien Chavanton
Hi Charles, I am implementing the size check, I think it would be nightmarish no to know if sometimes contacts could be missing because of oversized messages, this will simply things for anyone using this feature. I am verifying that the calculation matches and I will test and commit later

[sr-dev] [kamailio/kamailio] 4.4 dmqfix backport (#1029)

2017-03-10 Thread Julien Chavanton
backport the synchronisation problem fix You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/1029 -- Commit Summary -- * dmq_usrloc: Use ruid for saving/updating contacts * dmq_usrloc: adjust locking sequence -- File Changes -- M

[sr-dev] [kamailio/kamailio] 4.4 dmqfix backport (#1028)

2017-03-10 Thread Julien Chavanton
Test in my lab You can view, comment on, or merge this pull request online at: https://github.com/kamailio/kamailio/pull/1028 -- Commit Summary -- * dmq_usrloc: Use ruid for saving/updating contacts * dmq_usrloc: adjust locking sequence to avoid dead-lock -- File Changes -- M

Re: [sr-dev] [kamailio/kamailio] 4.4 dmqfix backport (#1028)

2017-03-10 Thread Julien Chavanton
Closed #1028. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/1028#event-995462021___ sr-dev mailing list sr-dev@lists.sip-router.org

Re: [sr-dev] [kamailio/kamailio] 4.4 dmqfix backport (#1028)

2017-03-10 Thread Julien Chavanton
missing other modifications -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/kamailio/kamailio/pull/1028#issuecomment-285768417___ sr-dev mailing list

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-05 Thread Julien Chavanton
I added `dmq_usrloc_batch_msg_size` to make sure we are never trying to send messages too large and to simplify the configuration. I did run a bunch of tests again. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [sr-dev] [kamailio/kamailio] dmq_usrloc: sync with multi contacts per message (#1054)

2017-04-07 Thread Julien Chavanton
One thing that could help would be to increase the retransmission time-out only for the sync traffic, I did not see any obvious way to do this. I guess this is off topic and I was planning to discuss this in the mailing list. -- You are receiving this because you are subscribed to this thread.