Re: [Bug 1153077] Re: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default)
I haven't used MAAS for about 6 months precisely because of this issue. It may have been fixed in the interim. On Tue, Oct 15, 2013 at 6:43 AM, Julian Edwards 1153...@bugs.launchpad.net wrote: Have any you seen this again lately? -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default) Status in MAAS: Triaged Status in “maas” package in Ubuntu: Confirmed Bug description: With the default MAAS install I was having a problem where celery would become unresponsive and stop running new tasks (including powering compute nodes on/off) after a while, and eventually the system would have out of memory errors. The problem seems to have been that the queue was filling up with dhcp_upload_leases tasks (I checked on this by getting tasks out with the amqp tools...). This problem goes away if the timedelta in /usr/share/maas/celeryconfig_cluster.py is changed from 1 minute to 5 minutes. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1153077/+subscriptions -- Mark Lee Stillwell mark...@fortawesome.org -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to maas in Ubuntu. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1153077/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
Re: [Bug 1153077] Re: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default)
For the moment at least the system I had tried to install MAAS on is in use. I may look at it again during the winter break. On Tue, Oct 15, 2013 at 10:01 AM, Julian Edwards 1153...@bugs.launchpad.net wrote: On Tuesday 15 Oct 2013 08:48:08 you wrote: I haven't used MAAS for about 6 months precisely because of this issue. It may have been fixed in the interim. Are you able to try it again? I have not seen the problem in a long time, although it may have been a RabbitMQ bug in precise. -- You received this bug notification because you are subscribed to the bug report. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default) Status in MAAS: Triaged Status in “maas” package in Ubuntu: Confirmed Bug description: With the default MAAS install I was having a problem where celery would become unresponsive and stop running new tasks (including powering compute nodes on/off) after a while, and eventually the system would have out of memory errors. The problem seems to have been that the queue was filling up with dhcp_upload_leases tasks (I checked on this by getting tasks out with the amqp tools...). This problem goes away if the timedelta in /usr/share/maas/celeryconfig_cluster.py is changed from 1 minute to 5 minutes. To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1153077/+subscriptions -- Mark Lee Stillwell mark...@fortawesome.org -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to maas in Ubuntu. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1153077/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1153077] Re: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default)
Unfortunately, due in part to this bug, we aren't currently using MAAS. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to maas in Ubuntu. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and celery stops executing tasks when upload_dhcp_leases is done every minute (by default) To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1153077/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1153077] Re: rabbitmq queue fills up and salary stops executing tasks when upload_dhcp_leases is done every minute (by default)
As an update, the problem does not go away actually, it just takes longer to develop. So, for some reason a task is getting hung or something, causing all tasks behind it to not be executed and filling the queue. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to maas in Ubuntu. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and salary stops executing tasks when upload_dhcp_leases is done every minute (by default) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1153077/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1153077] [NEW] rabbitmq queue fills up and salary stops executing tasks when upload_dhcp_leases is done every minute (by default)
Public bug reported: With the default MAAS install I was having a problem where celery would become unresponsive and stop running new tasks (including powering compute nodes on/off) after a while, and eventually the system would have out of memory errors. The problem seems to have been that the queue was filling up with dhcp_upload_leases tasks (I checked on this by getting tasks out with the amqp tools...). This problem goes away if the timedelta in /usr/share/maas/celeryconfig_cluster.py is changed from 1 minute to 5 minutes. ** Affects: maas (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to maas in Ubuntu. https://bugs.launchpad.net/bugs/1153077 Title: rabbitmq queue fills up and salary stops executing tasks when upload_dhcp_leases is done every minute (by default) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/maas/+bug/1153077/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
