Re: [SR-Users] dmq issues

[Julien Chavanton]

Hi Charles, I took the time to find my notes and reproduce this in a lab

https://github.com/kamailio/kamailio/issues/1349

I guess we can follow up from there in the github issue

Cheers !

On Fri, Dec 1, 2017 at 4:06 AM, Charles Chance <
charles.cha...@sipcentric.com> wrote:

> Hi Julien,
>
> Please do and I'll gladly take a closer look.
>
> Best,
>
> Charles
>
> On 1 December 2017 at 11:59, Julien Chavanton 
> wrote:
>
>> Hi Charles, great trick the FQDN + multi_notify
>>
>> I am guessing that with the multi_notify setting, the problem I was
>> refering too may be solved.
>>
>> I will document it in a github issue.
>>
>>
>>
>> On Fri, Dec 1, 2017 at 3:26 AM, Charles Chance <
>> charles.cha...@sipcentric.com> wrote:
>>
>>> Hello,
>>>
>>> It seems you have already solved your issue, but just to add, it's often
>>> better to provide an FQDN as notification_address which resolves to
>>> multiple IPs (and set "multi_notify" parameter).
>>>
>>> There should be no issues when shutting down multiple nodes at once, as
>>> long as the remaining nodes are still in contact with each other - at
>>> least, that is my experience across several long-established (and highly
>>> dynamic) clusters. When the disabled nodes are re-enabled, providing they
>>> have the address of at least one active node (or better, an FQDN resolving
>>> to multiple) they'll rejoin seamlessly.
>>>
>>> Of course, if others have a different experience, it would be good to
>>> learn more about it.
>>>
>>> Cheers,
>>>
>>> Charles
>>>
>>>
>>> On 1 December 2017 at 11:00, Julien Chavanton 
>>> wrote:
>>>
 Forgot to mention

 192.168.150.225 must have another node as notitifcation peer, not
 himself else he will not replicate data to other nodes

 On Fri, Dec 1, 2017 at 2:33 AM, Julien Chavanton 
 wrote:

> Hi Aydar,
>
> You can have only one notification_address in this case the last one
> will be used "192.168.150.225" this will be you DMQ master node
>
> I know the bus can break if you shutdown multiple nodes at the same
> time and one of them is the master node and then you restart them. (not
> sure if this is your case)
>
> If this happens the only option I can see is to restart all nodes one
> by one.
>
> I think this case could be handled better maybe by adding a dmq resync
> command that can recreate the bus .
>
>
> On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov <
> aidar.kama...@gmail.com> wrote:
>
>> Hello. I have 4  kamailio nodes with same config:
>>
>> modparam("dmq", "server_address", "sip:LOCALIP:5050")
>> modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
>> modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
>> modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
>> modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
>> modparam("dmq", "num_workers", 6)
>> modparam("dmq_usrloc", "enable", 1)
>> modparam("dmq_usrloc", "sync", 1)
>> modparam("dmq_usrloc", "batch_size", 2000)
>> modparam("dmq_usrloc", "batch_usleep", 1000)
>>
>>
>> regularly (don't know why) I've got issues - replication is broken,
>> for example now at all nodes I have:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.8.213
>> port: 5050
>> resolved_ip: 192.168.8.213
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.10.54
>> port: 5050
>> resolved_ip: 192.168.10.54
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 8
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.107.195
>> port: 5050
>> resolved_ip: 192.168.107.195
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> but at node 192.168.150.225:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> nodes restart doesn't help. but if i exec
>> systemctl stop kamailio && sleep 15 && systemctl start kamailio
>> it is partially solve my issue - dmq.list_nodes show all nodes, but
>> at 192.168.150.225 i see not all user locations: they are added gradually
>>
>> There is no errors in logs
>> Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

Re: [SR-Users] dmq issues

[Aidar Kamalov]

Now I added all my nodes, except self, on all nodes. It is ok now.

1 дек. 2017 г. 15:07 пользователь "Charles Chance" <
charles.cha...@sipcentric.com> написал:

> Hi Julien,
>
> Please do and I'll gladly take a closer look.
>
> Best,
>
> Charles
>
> On 1 December 2017 at 11:59, Julien Chavanton 
> wrote:
>
>> Hi Charles, great trick the FQDN + multi_notify
>>
>> I am guessing that with the multi_notify setting, the problem I was
>> refering too may be solved.
>>
>> I will document it in a github issue.
>>
>>
>>
>> On Fri, Dec 1, 2017 at 3:26 AM, Charles Chance <
>> charles.cha...@sipcentric.com> wrote:
>>
>>> Hello,
>>>
>>> It seems you have already solved your issue, but just to add, it's often
>>> better to provide an FQDN as notification_address which resolves to
>>> multiple IPs (and set "multi_notify" parameter).
>>>
>>> There should be no issues when shutting down multiple nodes at once, as
>>> long as the remaining nodes are still in contact with each other - at
>>> least, that is my experience across several long-established (and highly
>>> dynamic) clusters. When the disabled nodes are re-enabled, providing they
>>> have the address of at least one active node (or better, an FQDN resolving
>>> to multiple) they'll rejoin seamlessly.
>>>
>>> Of course, if others have a different experience, it would be good to
>>> learn more about it.
>>>
>>> Cheers,
>>>
>>> Charles
>>>
>>>
>>> On 1 December 2017 at 11:00, Julien Chavanton 
>>> wrote:
>>>
 Forgot to mention

 192.168.150.225 must have another node as notitifcation peer, not
 himself else he will not replicate data to other nodes

 On Fri, Dec 1, 2017 at 2:33 AM, Julien Chavanton 
 wrote:

> Hi Aydar,
>
> You can have only one notification_address in this case the last one
> will be used "192.168.150.225" this will be you DMQ master node
>
> I know the bus can break if you shutdown multiple nodes at the same
> time and one of them is the master node and then you restart them. (not
> sure if this is your case)
>
> If this happens the only option I can see is to restart all nodes one
> by one.
>
> I think this case could be handled better maybe by adding a dmq resync
> command that can recreate the bus .
>
>
> On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov <
> aidar.kama...@gmail.com> wrote:
>
>> Hello. I have 4  kamailio nodes with same config:
>>
>> modparam("dmq", "server_address", "sip:LOCALIP:5050")
>> modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
>> modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
>> modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
>> modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
>> modparam("dmq", "num_workers", 6)
>> modparam("dmq_usrloc", "enable", 1)
>> modparam("dmq_usrloc", "sync", 1)
>> modparam("dmq_usrloc", "batch_size", 2000)
>> modparam("dmq_usrloc", "batch_usleep", 1000)
>>
>>
>> regularly (don't know why) I've got issues - replication is broken,
>> for example now at all nodes I have:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.8.213
>> port: 5050
>> resolved_ip: 192.168.8.213
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.10.54
>> port: 5050
>> resolved_ip: 192.168.10.54
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 8
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.107.195
>> port: 5050
>> resolved_ip: 192.168.107.195
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> but at node 192.168.150.225:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> nodes restart doesn't help. but if i exec
>> systemctl stop kamailio && sleep 15 && systemctl start kamailio
>> it is partially solve my issue - dmq.list_nodes show all nodes, but
>> at 192.168.150.225 i see not all user locations: they are added gradually
>>
>> There is no errors in logs
>> Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

Re: [SR-Users] dmq issues

[Julien Chavanton]

Hi Charles, great trick the FQDN + multi_notify

I am guessing that with the multi_notify setting, the problem I was
refering too may be solved.

I will document it in a github issue.



On Fri, Dec 1, 2017 at 3:26 AM, Charles Chance <
charles.cha...@sipcentric.com> wrote:

> Hello,
>
> It seems you have already solved your issue, but just to add, it's often
> better to provide an FQDN as notification_address which resolves to
> multiple IPs (and set "multi_notify" parameter).
>
> There should be no issues when shutting down multiple nodes at once, as
> long as the remaining nodes are still in contact with each other - at
> least, that is my experience across several long-established (and highly
> dynamic) clusters. When the disabled nodes are re-enabled, providing they
> have the address of at least one active node (or better, an FQDN resolving
> to multiple) they'll rejoin seamlessly.
>
> Of course, if others have a different experience, it would be good to
> learn more about it.
>
> Cheers,
>
> Charles
>
>
> On 1 December 2017 at 11:00, Julien Chavanton 
> wrote:
>
>> Forgot to mention
>>
>> 192.168.150.225 must have another node as notitifcation peer, not himself
>> else he will not replicate data to other nodes
>>
>> On Fri, Dec 1, 2017 at 2:33 AM, Julien Chavanton 
>> wrote:
>>
>>> Hi Aydar,
>>>
>>> You can have only one notification_address in this case the last one
>>> will be used "192.168.150.225" this will be you DMQ master node
>>>
>>> I know the bus can break if you shutdown multiple nodes at the same time
>>> and one of them is the master node and then you restart them. (not sure if
>>> this is your case)
>>>
>>> If this happens the only option I can see is to restart all nodes one by
>>> one.
>>>
>>> I think this case could be handled better maybe by adding a dmq resync
>>> command that can recreate the bus .
>>>
>>>
>>> On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov 
>>> wrote:
>>>
 Hello. I have 4  kamailio nodes with same config:

 modparam("dmq", "server_address", "sip:LOCALIP:5050")
 modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
 modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
 modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
 modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
 modparam("dmq", "num_workers", 6)
 modparam("dmq_usrloc", "enable", 1)
 modparam("dmq_usrloc", "sync", 1)
 modparam("dmq_usrloc", "batch_size", 2000)
 modparam("dmq_usrloc", "batch_usleep", 1000)


 regularly (don't know why) I've got issues - replication is broken, for
 example now at all nodes I have:
 # kamcmd dmq.list_nodes
 {
 host: 192.168.8.213
 port: 5050
 resolved_ip: 192.168.8.213
 status: 2
 last_notification: 0
 local: 0
 }
 {
 host: 192.168.10.54
 port: 5050
 resolved_ip: 192.168.10.54
 status: 2
 last_notification: 0
 local: 0
 }
 {
 host: 192.168.150.225
 port: 5050
 resolved_ip: 192.168.150.225
 status: 8
 last_notification: 0
 local: 0
 }
 {
 host: 192.168.107.195
 port: 5050
 resolved_ip: 192.168.107.195
 status: 2
 last_notification: 0
 local: 1
 }

 but at node 192.168.150.225:
 # kamcmd dmq.list_nodes
 {
 host: 192.168.150.225
 port: 5050
 resolved_ip: 192.168.150.225
 status: 2
 last_notification: 0
 local: 0
 }
 {
 host: 192.168.150.225
 port: 5050
 resolved_ip: 192.168.150.225
 status: 2
 last_notification: 0
 local: 1
 }

 nodes restart doesn't help. but if i exec
 systemctl stop kamailio && sleep 15 && systemctl start kamailio
 it is partially solve my issue - dmq.list_nodes show all nodes, but at
 192.168.150.225 i see not all user locations: they are added gradually

 There is no errors in logs
 Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

Re: [SR-Users] dmq issues

[Charles Chance]

Hello,

It seems you have already solved your issue, but just to add, it's often
better to provide an FQDN as notification_address which resolves to
multiple IPs (and set "multi_notify" parameter).

There should be no issues when shutting down multiple nodes at once, as
long as the remaining nodes are still in contact with each other - at
least, that is my experience across several long-established (and highly
dynamic) clusters. When the disabled nodes are re-enabled, providing they
have the address of at least one active node (or better, an FQDN resolving
to multiple) they'll rejoin seamlessly.

Of course, if others have a different experience, it would be good to learn
more about it.

Cheers,

Charles


On 1 December 2017 at 11:00, Julien Chavanton  wrote:

> Forgot to mention
>
> 192.168.150.225 must have another node as notitifcation peer, not himself
> else he will not replicate data to other nodes
>
> On Fri, Dec 1, 2017 at 2:33 AM, Julien Chavanton 
> wrote:
>
>> Hi Aydar,
>>
>> You can have only one notification_address in this case the last one
>> will be used "192.168.150.225" this will be you DMQ master node
>>
>> I know the bus can break if you shutdown multiple nodes at the same time
>> and one of them is the master node and then you restart them. (not sure if
>> this is your case)
>>
>> If this happens the only option I can see is to restart all nodes one by
>> one.
>>
>> I think this case could be handled better maybe by adding a dmq resync
>> command that can recreate the bus .
>>
>>
>> On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov 
>> wrote:
>>
>>> Hello. I have 4  kamailio nodes with same config:
>>>
>>> modparam("dmq", "server_address", "sip:LOCALIP:5050")
>>> modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
>>> modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
>>> modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
>>> modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
>>> modparam("dmq", "num_workers", 6)
>>> modparam("dmq_usrloc", "enable", 1)
>>> modparam("dmq_usrloc", "sync", 1)
>>> modparam("dmq_usrloc", "batch_size", 2000)
>>> modparam("dmq_usrloc", "batch_usleep", 1000)
>>>
>>>
>>> regularly (don't know why) I've got issues - replication is broken, for
>>> example now at all nodes I have:
>>> # kamcmd dmq.list_nodes
>>> {
>>> host: 192.168.8.213
>>> port: 5050
>>> resolved_ip: 192.168.8.213
>>> status: 2
>>> last_notification: 0
>>> local: 0
>>> }
>>> {
>>> host: 192.168.10.54
>>> port: 5050
>>> resolved_ip: 192.168.10.54
>>> status: 2
>>> last_notification: 0
>>> local: 0
>>> }
>>> {
>>> host: 192.168.150.225
>>> port: 5050
>>> resolved_ip: 192.168.150.225
>>> status: 8
>>> last_notification: 0
>>> local: 0
>>> }
>>> {
>>> host: 192.168.107.195
>>> port: 5050
>>> resolved_ip: 192.168.107.195
>>> status: 2
>>> last_notification: 0
>>> local: 1
>>> }
>>>
>>> but at node 192.168.150.225:
>>> # kamcmd dmq.list_nodes
>>> {
>>> host: 192.168.150.225
>>> port: 5050
>>> resolved_ip: 192.168.150.225
>>> status: 2
>>> last_notification: 0
>>> local: 0
>>> }
>>> {
>>> host: 192.168.150.225
>>> port: 5050
>>> resolved_ip: 192.168.150.225
>>> status: 2
>>> last_notification: 0
>>> local: 1
>>> }
>>>
>>> nodes restart doesn't help. but if i exec
>>> systemctl stop kamailio && sleep 15 && systemctl start kamailio
>>> it is partially solve my issue - dmq.list_nodes show all nodes, but at
>>> 192.168.150.225 i see not all user locations: they are added gradually
>>>
>>> There is no errors in logs
>>> Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

Re: [SR-Users] dmq issues

[Julien Chavanton]

Forgot to mention

192.168.150.225 must have another node as notitifcation peer, not himself
else he will not replicate data to other nodes

On Fri, Dec 1, 2017 at 2:33 AM, Julien Chavanton 
wrote:

> Hi Aydar,
>
> You can have only one notification_address in this case the last one will
> be used "192.168.150.225" this will be you DMQ master node
>
> I know the bus can break if you shutdown multiple nodes at the same time
> and one of them is the master node and then you restart them. (not sure if
> this is your case)
>
> If this happens the only option I can see is to restart all nodes one by
> one.
>
> I think this case could be handled better maybe by adding a dmq resync
> command that can recreate the bus .
>
>
> On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov 
> wrote:
>
>> Hello. I have 4  kamailio nodes with same config:
>>
>> modparam("dmq", "server_address", "sip:LOCALIP:5050")
>> modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
>> modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
>> modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
>> modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
>> modparam("dmq", "num_workers", 6)
>> modparam("dmq_usrloc", "enable", 1)
>> modparam("dmq_usrloc", "sync", 1)
>> modparam("dmq_usrloc", "batch_size", 2000)
>> modparam("dmq_usrloc", "batch_usleep", 1000)
>>
>>
>> regularly (don't know why) I've got issues - replication is broken, for
>> example now at all nodes I have:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.8.213
>> port: 5050
>> resolved_ip: 192.168.8.213
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.10.54
>> port: 5050
>> resolved_ip: 192.168.10.54
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 8
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.107.195
>> port: 5050
>> resolved_ip: 192.168.107.195
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> but at node 192.168.150.225:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> nodes restart doesn't help. but if i exec
>> systemctl stop kamailio && sleep 15 && systemctl start kamailio
>> it is partially solve my issue - dmq.list_nodes show all nodes, but at
>> 192.168.150.225 i see not all user locations: they are added gradually
>>
>> There is no errors in logs
>> Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

Re: [SR-Users] dmq issues

[Aidar Kamalov]

It was because at 192.168.150.225 I set this ip for both - server_address
and notification_address and 192.168.150.225 is last sting in config.

After a while this node send KDMQ to other nodes and disable itself :)

So, I must not set all my nodes(except itself) in notification_address at
each node?

U 192.168.150.225:5050 -> 192.168.10.54:5050
KDMQ sip:notification_peer@192.168.10.54:5050 SIP/2.0.
Via: SIP/2.0/UDP 192.168.150.225:5050
;branch=z9hG4bK2174.88e29643.0.
To: .
From: ;tag=a11041f20d719e0a854f96ceebf821bd-99c6.
CSeq: 10 KDMQ.
Call-ID: 621ad9f474b335d6-18221@192.168.150.225.
Content-Length: 198.
User-Agent: kamailio (5.0.4 (x86_64/linux)).
Max-Forwards: 1.
Content-Type: text/plain.
.
sip:192.168.8.213:5050;status=active.
sip:192.168.107.195:5050;status=active.
sip:192.168.10.54:5050;status=active.
sip:192.168.150.225:5050;status=active.
sip:192.168.150.225:5050;status=disabled.


2017-12-01 13:33 GMT+03:00 Julien Chavanton :

> Hi Aydar,
>
> You can have only one notification_address in this case the last one will
> be used "192.168.150.225" this will be you DMQ master node
>
> I know the bus can break if you shutdown multiple nodes at the same time
> and one of them is the master node and then you restart them. (not sure if
> this is your case)
>
> If this happens the only option I can see is to restart all nodes one by
> one.
>
> I think this case could be handled better maybe by adding a dmq resync
> command that can recreate the bus .
>
>
> On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov 
> wrote:
>
>> Hello. I have 4  kamailio nodes with same config:
>>
>> modparam("dmq", "server_address", "sip:LOCALIP:5050")
>> modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
>> modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
>> modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
>> modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
>> modparam("dmq", "num_workers", 6)
>> modparam("dmq_usrloc", "enable", 1)
>> modparam("dmq_usrloc", "sync", 1)
>> modparam("dmq_usrloc", "batch_size", 2000)
>> modparam("dmq_usrloc", "batch_usleep", 1000)
>>
>>
>> regularly (don't know why) I've got issues - replication is broken, for
>> example now at all nodes I have:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.8.213
>> port: 5050
>> resolved_ip: 192.168.8.213
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.10.54
>> port: 5050
>> resolved_ip: 192.168.10.54
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 8
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.107.195
>> port: 5050
>> resolved_ip: 192.168.107.195
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> but at node 192.168.150.225:
>> # kamcmd dmq.list_nodes
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 0
>> }
>> {
>> host: 192.168.150.225
>> port: 5050
>> resolved_ip: 192.168.150.225
>> status: 2
>> last_notification: 0
>> local: 1
>> }
>>
>> nodes restart doesn't help. but if i exec
>> systemctl stop kamailio && sleep 15 && systemctl start kamailio
>> it is partially solve my issue - dmq.list_nodes show all nodes, but at
>> 192.168.150.225 i see not all user locations: they are added gradually
>>
>> There is no errors in logs
>> Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

Re: [SR-Users] dmq issues

[Julien Chavanton]

Hi Aydar,

You can have only one notification_address in this case the last one will
be used "192.168.150.225" this will be you DMQ master node

I know the bus can break if you shutdown multiple nodes at the same time
and one of them is the master node and then you restart them. (not sure if
this is your case)

If this happens the only option I can see is to restart all nodes one by
one.

I think this case could be handled better maybe by adding a dmq resync
command that can recreate the bus .


On Fri, Dec 1, 2017 at 12:32 AM, Aidar Kamalov 
wrote:

> Hello. I have 4  kamailio nodes with same config:
>
> modparam("dmq", "server_address", "sip:LOCALIP:5050")
> modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
> modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
> modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
> modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
> modparam("dmq", "num_workers", 6)
> modparam("dmq_usrloc", "enable", 1)
> modparam("dmq_usrloc", "sync", 1)
> modparam("dmq_usrloc", "batch_size", 2000)
> modparam("dmq_usrloc", "batch_usleep", 1000)
>
>
> regularly (don't know why) I've got issues - replication is broken, for
> example now at all nodes I have:
> # kamcmd dmq.list_nodes
> {
> host: 192.168.8.213
> port: 5050
> resolved_ip: 192.168.8.213
> status: 2
> last_notification: 0
> local: 0
> }
> {
> host: 192.168.10.54
> port: 5050
> resolved_ip: 192.168.10.54
> status: 2
> last_notification: 0
> local: 0
> }
> {
> host: 192.168.150.225
> port: 5050
> resolved_ip: 192.168.150.225
> status: 8
> last_notification: 0
> local: 0
> }
> {
> host: 192.168.107.195
> port: 5050
> resolved_ip: 192.168.107.195
> status: 2
> last_notification: 0
> local: 1
> }
>
> but at node 192.168.150.225:
> # kamcmd dmq.list_nodes
> {
> host: 192.168.150.225
> port: 5050
> resolved_ip: 192.168.150.225
> status: 2
> last_notification: 0
> local: 0
> }
> {
> host: 192.168.150.225
> port: 5050
> resolved_ip: 192.168.150.225
> status: 2
> last_notification: 0
> local: 1
> }
>
> nodes restart doesn't help. but if i exec
> systemctl stop kamailio && sleep 15 && systemctl start kamailio
> it is partially solve my issue - dmq.list_nodes show all nodes, but at
> 192.168.150.225 i see not all user locations: they are added gradually
>
> There is no errors in logs
> Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: 

[SR-Users] dmq issues

[Aidar Kamalov]

Hello. I have 4  kamailio nodes with same config:

modparam("dmq", "server_address", "sip:LOCALIP:5050")
modparam("dmq", "notification_address", "sip:192.168.8.213:5050")
modparam("dmq", "notification_address", "sip:192.168.10.54:5050")
modparam("dmq", "notification_address", "sip:192.168.107.195:5050")
modparam("dmq", "notification_address", "sip:192.168.150.225:5050")
modparam("dmq", "num_workers", 6)
modparam("dmq_usrloc", "enable", 1)
modparam("dmq_usrloc", "sync", 1)
modparam("dmq_usrloc", "batch_size", 2000)
modparam("dmq_usrloc", "batch_usleep", 1000)


regularly (don't know why) I've got issues - replication is broken, for
example now at all nodes I have:
# kamcmd dmq.list_nodes
{
host: 192.168.8.213
port: 5050
resolved_ip: 192.168.8.213
status: 2
last_notification: 0
local: 0
}
{
host: 192.168.10.54
port: 5050
resolved_ip: 192.168.10.54
status: 2
last_notification: 0
local: 0
}
{
host: 192.168.150.225
port: 5050
resolved_ip: 192.168.150.225
status: 8
last_notification: 0
local: 0
}
{
host: 192.168.107.195
port: 5050
resolved_ip: 192.168.107.195
status: 2
last_notification: 0
local: 1
}

but at node 192.168.150.225:
# kamcmd dmq.list_nodes
{
host: 192.168.150.225
port: 5050
resolved_ip: 192.168.150.225
status: 2
last_notification: 0
local: 0
}
{
host: 192.168.150.225
port: 5050
resolved_ip: 192.168.150.225
status: 2
last_notification: 0
local: 1
}

nodes restart doesn't help. but if i exec
systemctl stop kamailio && sleep 15 && systemctl start kamailio
it is partially solve my issue - dmq.list_nodes show all nodes, but at
192.168.150.225 i see not all user locations: they are added gradually

There is no errors in logs
Dec  1 08:28:37 sipufa /usr/sbin/kamailio[18252]: NOTICE: