Hi Rafael, I just want to thank you for your help.
I tried to use the hint of fixing state.db but it did not work. The XML validator return the message that no errors was found on file. I had to recreate the XenServer's Pool. As I had no-critical instances on this pool the downtime impact was less. Thanks again for your help. --- FABIANO A. SOUZA _Assistência e Consultoria em TI_ f.so...@fdsouza.com.br http://www.fdsouza.com.br Em 15/03/2017 15:27, Rafael Weingärtner escreveu: > de nada ;) > > On Wed, Mar 15, 2017 at 2:26 PM, Fabiano A. Souza <f.so...@fdsouza.com.br> > wrote: > > I'm so sorry. > > Thanks > > --- > FABIANO A. SOUZA > _Assistência e Consultoria em TI_ > f.so...@fdsouza.com.br > http://www.fdsouza.com.br > > Em 15/03/2017 15:21, Rafael Weingärtner escreveu: > > The answer is in my previous email, last line: > > You can check if there is something wrong in state.db by passing the > contents of it in an XML validator. There will probably be some broken XML >> tags there. > > Last time I faced this problem, I put the host in maintenance, powered off > or migrate host's VMs, then I opened the file with an XML validator, found > the entries that were corrupted and removed them. Then I replaced the old > state.db file with this modified file. Afterward, reboot the server and > everything should be ok. > > No need to tell you to do a backup of things, check hypervisors > community...., right? > http://discussions.citrix.com/topic/241761-help-power- failed-now-xapi-will-not-start/ > <http://www.google.com/url?q=http%3A%2F%2Fdiscussions. citrix.com%2Ftopic%2F241761-help-power-failed-now-xapi- will-not-start%2F&sa=D&sntz=1&usg=AFQjCNEMwpDSMLgdddVdD6iDcZ4--yxtkQ> > On Wed, Mar 15, 2017 at 2:05 PM, Fabiano A. Souza < f.so...@fdsouza.com.br> > wrote: > > Very thanks Rafael. I was already imagining that this would be the > problem. Do you know any way to fix this problem in XenServer? > > By your experience do you believe that recreating the pool in XenServer > will correct this problem? > > Again thanks so much for your help. > > --- > FABIANO A. SOUZA > _Assistência e Consultoria em TI_ > f.so...@fdsouza.com.br > http://www.fdsouza.com.br > > Em 15/03/2017 14:36, Rafael Weingärtner escreveu: > > Looking at the log from your XenServer, and as I said before the problem is not in ACS. > This kind of log entries: > > Db_exn.DBCache_NotFound("missing row", "VBD", > > "OpaqueRef:e1136dfc-cdbc-ba21-3eb8-94838cc75030") > or > > D:424e469aca54 failed with exception Db_exn.DBCache_NotFound("missing > row", "task", > indicate a problem in the hypervisor metadata. In your case, XenServer uses > a file called state.db. This file contains metadata in XML format. Probably > your force reboot corrupted some entries there (I have already had similar > problems before). You can check if there is something wrong in state.db by > passing the contents of it in a XML validator. There will probably be some > broken XML tags there. > > On Wed, Mar 15, 2017 at 12:15 PM, Fabiano A. Souza < f.so...@fdsouza.com.br> > wrote: > >> Yes. I set up the DataCenter with the Advanced Network configuration. >> >> I'm attaching XenServer file log detailed sample. >> >> Thank you so much for your help. >> --- >> >> *Fabiano A. Souza* >> *Assistência e Consultoria em TI* >> f.so...@fdsouza.com.br >> http://www.fdsouza.com.br >> >> Em 15/03/2017 12:52, Rafael Weingärtner escreveu: >> >> I am not sure if this is a complete destroy and if ACS re-uses something >> from the old VMs. >> >> From your logs, >> >> VIF.unplug >> R:68394c4e8e7c failed with exception Db_exn.DBCache_NotFound("missing >> >> This probably happened because when the VM was being created and the VIF >> was not created in the hypervisor. There are probably other error log >> entries before this one. From your other log: >> >> errorInfo: [HOST_CANNOT_ATTACH_NETWORK, >> >> It is clear that there is something wrong in your XenServer. If you want us >> to help you, you need to provide a more log entries (not scattered bits). >> Also, in the XenServer logs, there will be plenty of exceptions there >> regarding this problem. >> You said that user VMs are working. I am guessing you have an advanced >> networking set up. Maybe the Guest network was not affected, but the >> Management or Storage network configuration in the XenServer may be >> affected by some misconfiguration/corruption that happened during the >> force-reboot. >> >> On Wed, Mar 15, 2017 at 11:42 AM, Fabiano A. Souza <f.so...@fdsouza.com.brwrote: >> The ACS is trying to do this itself. >> >> ACS create, trying start, after error stop and destroy the systemvms >> instance. The instance name or ID is being incremented at each attempt >> to create these instances (s-300-VM, s-301-VM...). >> >> --- >> FABIANO A. SOUZA >> _Assistência e Consultoria em TI_ >> f.so...@fdsouza.com.br >> http://www.fdsouza.com.br >> >> Em 15/03/2017 12:27, Rafael Weingärtner escreveu: >> >> You said you forced a reboot. It looks like something got corrupted in >> >> your >> >> host. >> >> I would not do some drastic measure like destroying pool. >> >> Have you tried to destroy the system VMs and then let ACS re-create them? >> On Wed, Mar 15, 2017 at 11:23 AM, Fabiano A. Souza < >> >> f.so...@fdsouza.com.br> >> >> wrote: >> >> Hi Rafael >> >> I have some other Guest (Windows/Linux) include VRs that works normally. >> Only systemvms has problem to deploy. I use one pool with two XenServer >> 7 host. In xenresource.log I see this error below. >> >> Mar 15 08:31:32 fl-mia-compute1 xapi: [error|fl-mia-compute1|605 INET >> :::80|dispatch:VIF.unplug D:6cce7f031461|backtrace] VIF.unplug >> R:68394c4e8e7c failed with exception Db_exn.DBCache_NotFound("missing >> row", "VIF", "OpaqueRef:20e49fbf-5c92-ef21-805b-13b1997e326e") >> >> Mar 15 08:31:32 fl-mia-compute1 xapi: [error|fl-mia-compute1|605 INET >> :::80|dispatch:VIF.unplug D:b37f4fd2de0c|backtrace] VIF.unplug >> R:fcca316e07d3 failed with exception Db_exn.DBCache_NotFound("missing >> row", "VIF", "OpaqueRef:4d7c3a36-982a-825e-e71f-8978108ec56b") >> >> I'm thinking remove the hosts from CloudStack and destroy pool. Create >> again the pool and re-attach them again on CloudStack configuration. Do >> you believe that can I have problem to start the guests instance after >> that? >> >> --- >> FABIANO A. SOUZA >> _Assistência e Consultoria em TI_ >> f.so...@fdsouza.com.br >> http://www.fdsouza.com.br >> >> Em 15/03/2017 11:28, Rafael Weingärtner escreveu: >> >> This is not an Apache CloudStack (ACS) problem per se. >> "HOST_CANNOT_ATTACH_NETWOR" >> Something is wrong with your hosts. Are all of the networks of yours >> >> hosts working? is everything in place as ACS expects? >> >> On Wed, Mar 15, 2017 at 10:26 AM, Fabiano A. Souza < >> >> f.so...@fdsouza.com.br> wrote: >> >> After my host with XenServer 7 forced reboot the systemvms can't start >> anymore. >> >> Can you help me? >> >> 2017-03-14 09:42:10,724 WARN [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-184:ctx-f12cdd04) (logid:4552cd9d) Task failed! Task >> record: uuid: 84f3483d-1021-22fb-be39-2eb15366e03e >> nameLabel: Async.VM.start_on >> nameDescription: >> allowedOperations: [] >> currentOperations: {} >> created: Tue Mar 14 09:42:46 BRT 2017 >> finished: Tue Mar 14 09:42:46 BRT 2017 >> status: failure >> residentOn: com.xensource.xenapi.Host@4e05ddb9 >> progress: 1.0 >> type: <none/> >> result: >> errorInfo: [HOST_CANNOT_ATTACH_NETWORK, >> OpaqueRef:3269022e-9b3d-88ca-8fb1-b15f3948b3a9, >> OpaqueRef:cd7b3c3e-92fd-18a3-eaa7-099aa066bb46] >> otherConfig: {} >> subtaskOf: com.xensource.xenapi.Task@aaf13f6f >> subtasks: [] >> >> 2017-03-14 09:42:10,730 WARN [c.c.h.x.r.CitrixResourceBase] >> (DirectAgent-184:ctx-f12cdd04) (logid:4552cd9d) Unable to start >> VM(s-294-VM) on host(b8324267-2cbe-4aa9-9b26-215375cb5553) due to Task >> failed! Task record: uuid: 84f3483d-1021-22fb-be39-2eb15366e03e >> nameLabel: Async.VM.start_on >> nameDescription: >> allowedOperations: [] >> currentOperations: {} >> created: Tue Mar 14 09:42:46 BRT 2017 >> finished: Tue Mar 14 09:42:46 BRT 2017 >> status: failure >> residentOn: com.xensource.xenapi.Host@4e05ddb9 >> progress: 1.0 >> type: <none/> >> result: >> errorInfo: [HOST_CANNOT_ATTACH_NETWORK, >> OpaqueRef:3269022e-9b3d-88ca-8fb1-b15f3948b3a9, >> OpaqueRef:cd7b3c3e-92fd-18a3-eaa7-099aa066bb46] >> otherConfig: {} >> subtaskOf: com.xensource.xenapi.Task@aaf13f6f >> subtasks: [] >> >> Task failed! Task record: uuid: 84f3483d-1021-22fb-be39-2eb15366e03e >> nameLabel: Async.VM.start_on >> nameDescription: >> allowedOperations: [] >> currentOperations: {} >> created: Tue Mar 14 09:42:46 BRT 2017 >> finished: Tue Mar 14 09:42:46 BRT 2017 >> status: failure >> residentOn: com.xensource.xenapi.Host@4e05ddb9 >> progress: 1.0 >> type: <none/> >> result: >> errorInfo: [HOST_CANNOT_ATTACH_NETWORK, >> OpaqueRef:3269022e-9b3d-88ca-8fb1-b15f3948b3a9, >> OpaqueRef:cd7b3c3e-92fd-18a3-eaa7-099aa066bb46] >> otherConfig: {} >> subtaskOf: com.xensource.xenapi.Task@aaf13f6f >> subtasks: [] >> >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase. >> checkForSuccess(CitrixResourceBase.java:471) >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM( >> CitrixResourceBase.java:4900) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase. >> CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:128) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper.xenbase. >> CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53) >> at >> com.cloud.hypervisor.xenserver.resource.wrapper. >> xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122) >> at >> com.cloud.hypervisor.xenserver.resource.CitrixResourceBase. >> >> executeRequest( >> >> CitrixResourceBase.java:1687) >> at >> com.cloud.agent.manager.DirectAgentAttache$Task.runInContext( >> DirectAgentAttache.java:315) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run( >> ManagedContextRunnable.java:49) >> at >> org.apache.cloudstack.managed.context.impl. >> >> DefaultManagedContext$1.call( >> >> DefaultManagedContext.java:56) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext. >> callWithContext(DefaultManagedContext.java:103) >> at >> org.apache.cloudstack.managed.context.impl.DefaultManagedContext. >> runWithContext(DefaultManagedContext.java:53) >> at >> org.apache.cloudstack.managed.context.ManagedContextRunnable.run( >> ManagedContextRunnable.java:46) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at java.util.concurrent.FutureTask.run(FutureTask.java:266) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ >> ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ >> >> ScheduledFutureTask.run( >> >> ScheduledThreadPoolExecutor.java:293) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker( >> ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run( >> ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> >> -- >> FABIANO A. SOUZA >> _Assistência e Consultoria em TI_ >> f.so...@fdsouza.com.br >> http://www.fdsouza.com.br
