Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-20 Thread Vladimir Kuklin
Actually, we are not skipping 'Started' state - we just consider resource as started when beam is powered up and rabbitmq start_app/stop_app action succeeds. Such a node is considered as a good one that can be marked as 'Master' to which the nodes should connect and then all the cluster join/leave

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-19 Thread Andrew Beekhof
On 20 May 2015, at 6:05 am, Andrew Woodward xar...@gmail.com wrote: On Thu, May 7, 2015 at 5:01 PM Andrew Beekhof abeek...@redhat.com wrote: On 5 May 2015, at 1:19 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you Andrew. on 2015/05/05 08:03, Andrew Beekhof

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-19 Thread Andrew Woodward
On Thu, May 7, 2015 at 5:01 PM Andrew Beekhof abeek...@redhat.com wrote: On 5 May 2015, at 1:19 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you Andrew. on 2015/05/05 08:03, Andrew Beekhof wrote: On 28 Apr 2015, at 11:15 pm, Bogdan Dobrelya bdobre...@mirantis.com

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-07 Thread Andrew Beekhof
On 5 May 2015, at 1:19 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you Andrew. on 2015/05/05 08:03, Andrew Beekhof wrote: On 28 Apr 2015, at 11:15 pm, Bogdan Dobrelya bdobre...@mirantis.com wrote: Hello, Hello, Zhou I using Fuel 6.0.1 and find that RabbitMQ

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-07 Thread Andrew Beekhof
On 5 May 2015, at 9:30 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you Andrew. Sorry for misspell your name in the previous email. on 2015/05/05 14:25, Andrew Beekhof wrote: On 5 May 2015, at 2:31 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-07 Thread Andrew Beekhof
On 5 May 2015, at 7:52 pm, Bogdan Dobrelya bdobre...@mirantis.com wrote: On 05.05.2015 04:32, Andrew Beekhof wrote: [snip] Technically it calculates an ordered graph of actions that need to be performed for a set of related resources. You can see an example of the kinds of graphs

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-05 Thread Zhou Zheng Sheng / 周征晟
Thank you Andrew. Sorry for misspell your name in the previous email. on 2015/05/05 14:25, Andrew Beekhof wrote: On 5 May 2015, at 2:31 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you Bogdan for clearing the pacemaker promotion process for me. on 2015/05/05 10:32, Andrew

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-05 Thread Andrew Beekhof
On 5 May 2015, at 2:31 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: Thank you Bogdan for clearing the pacemaker promotion process for me. on 2015/05/05 10:32, Andrew Beekhof wrote: On 29 Apr 2015, at 5:38 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: [snip]

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-05 Thread Bogdan Dobrelya
On 05.05.2015 04:32, Andrew Beekhof wrote: [snip] Technically it calculates an ordered graph of actions that need to be performed for a set of related resources. You can see an example of the kinds of graphs it produces at:

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-04 Thread Zhou Zheng Sheng / 周征晟
Thank you Andrew. on 2015/05/05 08:03, Andrew Beekhof wrote: On 28 Apr 2015, at 11:15 pm, Bogdan Dobrelya bdobre...@mirantis.com wrote: Hello, Hello, Zhou I using Fuel 6.0.1 and find that RabbitMQ recover time is long after power failure. I have a running HA environment, then I reset power

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-04 Thread Zhou Zheng Sheng / 周征晟
Thank you Bogdan for clearing the pacemaker promotion process for me. on 2015/05/05 10:32, Andrew Beekhof wrote: On 29 Apr 2015, at 5:38 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: [snip] Batch is a pacemaker concept I found when I was reading its documentation and code. There

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-04 Thread Andrew Beekhof
On 28 Apr 2015, at 11:15 pm, Bogdan Dobrelya bdobre...@mirantis.com wrote: Hello, Hello, Zhou I using Fuel 6.0.1 and find that RabbitMQ recover time is long after power failure. I have a running HA environment, then I reset power of all the machines at the same time. I observe that

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-04 Thread Andrew Beekhof
On 29 Apr 2015, at 5:38 pm, Zhou Zheng Sheng / 周征晟 zhengsh...@awcloud.com wrote: [snip] Batch is a pacemaker concept I found when I was reading its documentation and code. There is a batch-limit: 30 in the output of pcs property list --all. The pacemaker official documentation

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-05-03 Thread Zhou Zheng Sheng / 周征晟
Hello Sergii, Thank you for the great explanation on Galera OCF script. I replied your question inline. on 2015/05/03 04:49, Sergii Golovatiuk wrote: Hi Zhou, Galera OCF script is a bit special. Since MySQL keeps the most important data we should find the most recent data on all nodes across

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-04-29 Thread Zhou Zheng Sheng / 周征晟
Hi! Thank you very much Vladimir and Bogdan! Thanks for the fast respond and rich information. I backported MySQL and RabbitMQ ocf patches from stable/6.0 and tested again. A full reassemble takes about 5mins, this improves a lot. Adding the force_load trick I mentioned in the previous email, it

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-04-28 Thread Bogdan Dobrelya
On 28.04.2015 15:15, Bogdan Dobrelya wrote: Hello, Zhou Yes, this is a known issue [0]. Note, there were many bugfixes, like [1],[2],[3], merged for MQ OCF script, so you may want to try to backport them as well by the following guide [4] [0]

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-04-28 Thread Bogdan Dobrelya
Hello, Hello, Zhou I using Fuel 6.0.1 and find that RabbitMQ recover time is long after power failure. I have a running HA environment, then I reset power of all the machines at the same time. I observe that after reboot it usually takes 10 minutes for RabittMQ cluster to appear running

Re: [openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-04-28 Thread Vladimir Kuklin
Hi, Zhou Thank you for writing these awesome recommendations. We will look into them and see whether they provide significant impact. BTW, we have found a bunch of issues with our 5.1 and 6.0 RabbitMQ OCF script and fixed them in current master. Would you be so kind as to check out the newest

[openstack-dev] [Fuel] Speed Up RabbitMQ Recovering

2015-04-28 Thread Zhou Zheng Sheng / 周征晟
Hello, I using Fuel 6.0.1 and find that RabbitMQ recover time is long after power failure. I have a running HA environment, then I reset power of all the machines at the same time. I observe that after reboot it usually takes 10 minutes for RabittMQ cluster to appear running master-slave mode in