Re: [mcollective-users] ActiveMQ dropping connection for all the network of brokers after the queue overloaded
Chris, I will try with your recommendation. Thanks On Tuesday, May 30, 2017 at 8:53:25 PM UTC+5:30, Christopher Wood wrote: > > Karthi, I recommend increasing the ActiveMQ log level to debug, turning > everything back on, and seeing what happens when you try the same query. > > I found that activemq clustering would just upchuck and die when I sent > mcollective queries that were too large. 800 messages doesn't seem very > large but with no subscribers it depends on how your version of activemq > handles that with your memory limits. ActiveMQ typically seems happier when > there are active fast subscribers. > > I didn't notice any increased memory limits in the config you posted, > eventually I ended up with these memory limits for activemq: > > > > > > > > > > > > > > > > Usual disclaimer, ultimately I never was able to solve my activemq issues > and improved matters by using nats via choria. > > On Tue, May 30, 2017 at 04:45:11PM +0200, R.I.Pienaar wrote: > > > > > > On Tue, May 30, 2017, at 06:35, Karthi Sweet wrote: > > > I have configured ActiveMQ network of brokers with different locations > > > with > > > MCollective and it was working fine. All the MCollective servers are > > > configured with identity. One of my team member had executed the mco > > > commands with --nodes=/tmp/serverslist, where the serverslist has new > > > line > > > separated servers of 800+ which doesn't exists. So when he executed > none > > > of > > > the servers responded since those were doesn't exists and immediately > the > > > central ActiveMQ server dropped connection with all the network of > > > brokers > > > and it keeps trying reconnect but getting disconnected immediately. Am > > > not > > > sure why this happened when we run the mco query for the servers which > > > doesn't exists. But when I restart the central activemq then > everything > > > came back normal. > > > > > > > This would have made 800+ messages - one per recipient - and they would > > timeout and go away after some time. > > > > Honestly this is a tiny amount of messages it should be no problem at > > all, hard to say what happened though perhaps there were logs produced > > > > -- > > > > --- > > You received this message because you are subscribed to the Google > Groups "mcollective-users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to mcollective-us...@googlegroups.com . > > For more options, visit https://groups.google.com/d/optout. > -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mcollective-users] ActiveMQ dropping connection for all the network of brokers after the queue overloaded
Karthi, I recommend increasing the ActiveMQ log level to debug, turning everything back on, and seeing what happens when you try the same query. I found that activemq clustering would just upchuck and die when I sent mcollective queries that were too large. 800 messages doesn't seem very large but with no subscribers it depends on how your version of activemq handles that with your memory limits. ActiveMQ typically seems happier when there are active fast subscribers. I didn't notice any increased memory limits in the config you posted, eventually I ended up with these memory limits for activemq: Usual disclaimer, ultimately I never was able to solve my activemq issues and improved matters by using nats via choria. On Tue, May 30, 2017 at 04:45:11PM +0200, R.I.Pienaar wrote: > > > On Tue, May 30, 2017, at 06:35, Karthi Sweet wrote: > > I have configured ActiveMQ network of brokers with different locations > > with > > MCollective and it was working fine. All the MCollective servers are > > configured with identity. One of my team member had executed the mco > > commands with --nodes=/tmp/serverslist, where the serverslist has new > > line > > separated servers of 800+ which doesn't exists. So when he executed none > > of > > the servers responded since those were doesn't exists and immediately the > > central ActiveMQ server dropped connection with all the network of > > brokers > > and it keeps trying reconnect but getting disconnected immediately. Am > > not > > sure why this happened when we run the mco query for the servers which > > doesn't exists. But when I restart the central activemq then everything > > came back normal. > > > > This would have made 800+ messages - one per recipient - and they would > timeout and go away after some time. > > Honestly this is a tiny amount of messages it should be no problem at > all, hard to say what happened though perhaps there were logs produced > > -- > > --- > You received this message because you are subscribed to the Google Groups > "mcollective-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to mcollective-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [mcollective-users] ActiveMQ dropping connection for all the network of brokers after the queue overloaded
On Tue, May 30, 2017, at 06:35, Karthi Sweet wrote: > I have configured ActiveMQ network of brokers with different locations > with > MCollective and it was working fine. All the MCollective servers are > configured with identity. One of my team member had executed the mco > commands with --nodes=/tmp/serverslist, where the serverslist has new > line > separated servers of 800+ which doesn't exists. So when he executed none > of > the servers responded since those were doesn't exists and immediately the > central ActiveMQ server dropped connection with all the network of > brokers > and it keeps trying reconnect but getting disconnected immediately. Am > not > sure why this happened when we run the mco query for the servers which > doesn't exists. But when I restart the central activemq then everything > came back normal. > This would have made 800+ messages - one per recipient - and they would timeout and go away after some time. Honestly this is a tiny amount of messages it should be no problem at all, hard to say what happened though perhaps there were logs produced -- --- You received this message because you are subscribed to the Google Groups "mcollective-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to mcollective-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
