AW: ALL Hosts Stuck in Maintenance

2015-01-29 Thread Martin Emrich
...@gmail.com] Gesendet: Mittwoch, 28. Januar 2015 11:34 An: users@cloudstack.apache.org Betreff: Re: ALL Hosts Stuck in Maintenance Martin, My diagnosis hasn't been altered studied or challanged ;) for a quick fix vm != null should be added in the expression line 2083 of ResourceManagerImpl. A more robust

Re: ALL Hosts Stuck in Maintenance

2015-01-28 Thread Daan Hoogland
Betreff: Re: ALL Hosts Stuck in Maintenance Hi! Same problem here with CS 4.4.1 and 5 XenServers in one Cluster: Two hosts are in maintenance mode, and canceling maintenance mode results in Internal Server Error. Here's my backtrace: 2014-11-25 13:08:45,448 ERROR [c.c.a.ApiAsyncJobDispatcher

AW: ALL Hosts Stuck in Maintenance

2015-01-27 Thread Martin Emrich
Hi! Any news on this issue? I just fell in to this pit again :( Regards, Martin -Ursprüngliche Nachricht- Von: Martin Emrich [mailto:martin.emr...@empolis.com] Gesendet: Dienstag, 25. November 2014 13:39 An: users@cloudstack.apache.org Betreff: Re: ALL Hosts Stuck in Maintenance Hi

Re: ALL Hosts Stuck in Maintenance

2014-12-02 Thread Steve Searles
Thanks for the reply. Yes that did seem to be the issue. I did a manual cleanup of all of the vm’s that were assigned to that host, cleared out the async job queue and restarted the management server. This seems to have resolved the issue system wide. Steven Searles, CTO |

Re: ALL Hosts Stuck in Maintenance

2014-11-25 Thread Martin Emrich
Hi! Same problem here with CS 4.4.1 and 5 XenServers in one Cluster: Two hosts are in maintenance mode, and canceling maintenance mode results in Internal Server Error. Here's my backtrace: 2014-11-25 13:08:45,448 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-54:ctx-a076520e

Re: ALL Hosts Stuck in Maintenance

2014-11-22 Thread Daan Hoogland
Steven, I had a look at your stacktrace. It seems a vm is marked to be migrated back during maintenance mode cancel and it doesn't exist. Did you destroy any of the vms on the host while it was in maintenance? There is a number of jobs for migration each of which pertains to a vm. For one of

Re: ALL Hosts Stuck in Maintenance

2014-11-21 Thread Steve Searles
For some reason it is affecting every host. VMware, KVM, and XenServer. No hosts will come out of maintenance same NPE for all. Storage will go in and out of maintenance fine. Weird. Any ideas? The only way to get the host back online is to remove it and re-add it. Steven Searles

Re: ALL Hosts Stuck in Maintenance

2014-11-21 Thread Andrija Panic
Experienced similar behaviour, for kvm - seems like restarting libvirtd and give it some time to settle, and than agent connects ... on its own... Sent from Google Nexus 4 On Nov 21, 2014 4:43 PM, Steve Searles ssear...@zimcom.net wrote: For some reason it is affecting every host. VMware,

Re: ALL Hosts Stuck in Maintenance

2014-11-21 Thread Steve Searles
In my case the agent is connected fine. I have tried even restarting the hosts. I have the agent log in debug for the KVM server and am not seeing the mgmt server even trying to execute anything when coming out of maintenance. I just get the NPE immediately in the management server logs.

Re: ALL Hosts Stuck in Maintenance

2014-11-21 Thread Steve Searles
Below is the full NPE from the catalina.out if that would help. This is the same NPE for all hypervisor types. INFO [o.a.c.f.j.i.AsyncJobMonitor] (API-Job-Executor-10:ctx-208629c7 job-12679) Add job-12679 into job monitoring ERROR [c.c.a.ApiAsyncJobDispatcher]