It's a full install from an XCP CD. # service xapissl status xapissl (pid 2323) is running...
I performed a restart of xapissl anyway, and xe-toolstack-restart still fails starting the xapi service. Any other ideas? What would happen if I were to restart the system or perform a reinstall? Our customer and us are pretty nervous now, as they have not had a successful backup since last week Friday. So what else can I try? Thanks, Dave! On 20 January 2012 15:41, Dave Scott <[email protected]> wrote: > Hi,**** > > ** ** > > I should have asked earlier: is this a host installed via the XCP CD, or > is this a Debian system running the xcp- packages?**** > > ** ** > > It looks like xapi can’t find a running stunnel, and it looks like the > xe-toolstack-restart failed to run “/sbin/service xapissl restart”. Do you > have stunnel running, and listening on port 443? On a system installed via > the XCP CD, “service xapissl restart” should start stunnel. On a Debian > system – I believe – the xapi init.d script itself starts stunnel.**** > > ** ** > > Dave**** > > ** ** > > *From:* Lars Seeliger [mailto:[email protected]] > *Sent:* 20 January 2012 12:15 > *To:* Dave Scott > *Cc:* [email protected] > *Subject:* Re: [Xen-API] xapi will no longer start - what are my options?* > *** > > ** ** > > Hey, Dave > > Thanks for the prompt response. For fear of not including enough info, > I've pasted the entire xensource.log contents here: > http://pastebin.com/AW12gfM0 > > If you need anything else, just shout; this problem has ruined my day! :p* > *** > > On 20 January 2012 12:53, Dave Scott <[email protected]> wrote:**** > > Hi Lars,**** > > **** > > Have a look in the main xapi logs (in the confusingly-named file > /var/log/xensource.log). Start from the bottom and reverse-search to the > string “XAPI SERVER STARTING”. The lines after that will show how far the > startup sequence got.**** > > **** > > Cheers,**** > > Dave**** > > **** > > **** > > **** > > *From:* [email protected] [mailto: > [email protected]] *On Behalf Of *Lars Seeliger > *Sent:* 20 January 2012 10:23 > *To:* [email protected] > *Subject:* [Xen-API] xapi will no longer start - what are my options?**** > > **** > > Hi there > > A scripted backup running on one of our XCP installations failed a few > days ago, while exporting a snapshot. After the failure I tried to delete > the snapshot in question, to no avail (something about the VDI being in > use). > > Anyway, I thought an xe-toolstack-restart would reset any lock on that > file, allowing me to delete the no longer needed snapshot. Unfortunately, > xapi is now unable to start. > > I've just tried again, and this appears in /var/log/messages: > > Jan 20 11:01:21 xcp-hoppe xapi: [ info|xcp-hoppe|0 thread_zero||watchdog] > (Re)starting xapi... > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.850Z||1172|About to bind > to /var/xapi/forker/fd_e8f89481-9aae-05a5-1d73-fbb713f58ea3 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.850Z||1172|bound, > listening > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.851Z||2300|Child here! > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.851Z||2301|Grandchild > here! > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.851Z||2301|Started: > state.cmdargs = [/sbin/service;xapissl;restart] > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.851Z||2301|Started: > state.env = [PATH=/sbin:/usr/sbin:/bin:/usr/bin] > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Selecting in > handle_comms_no_fd_sock2 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Done > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|fd sock > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Selecting in > handle_comms_with_fd_sock2 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Done > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|fd sock2 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Received fd > named: ed350b47-3eb6-63e0-38c5-3beaaefb65dd - duping to 1 (from 6) > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Selecting in > handle_comms_with_fd_sock2 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Done > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|fd sock2 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Received fd > named: d810a903-961d-bb1e-aeb7-b39c98e5eefa - duping to 2 (from 6) > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Selecting in > handle_comms_with_fd_sock2 > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Done > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|comms sock > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Exec > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Finished... > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|Args after > replacement = [/sbin/service;xapissl;restart] > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:25.852Z||2301|I've received > the following fds: [2;1] > Jan 20 11:04:26 xcp-hoppe fe: 20120120T10:04:26.942Z||2301|Caught > unexpected exception: Unix.Unix_error(31, "write", "") > Jan 20 10:05:41 xcp-hoppe HVM5652[12007]: XENNET: WARNING: No handler > for oid 0xffda0014. > Jan 20 10:05:41 xcp-hoppe HVM5652[12007]: XENNET: WARNING: No handler > for oid 0xffa0ffa1. > Jan 20 10:05:41 xcp-hoppe HVM5652[12007]: XENNET: WARNING: Bad tcp task > offload header? > Jan 20 10:05:41 xcp-hoppe HVM5652[12007]: XENNET: WARNING: Bad tcp task > offload header? > Jan 20 11:06:16 xcp-hoppe snmpd[15173]: Received SNMP packet(s) from UDP: > [192.168.1.1]:2227 > Jan 20 11:06:16 xcp-hoppe snmpd[15173]: Received SNMP packet(s) from UDP: > [192.168.1.1]:2228 > Jan 20 11:06:17 xcp-hoppe snmpd[15173]: Received SNMP packet(s) from UDP: > [192.168.1.1]:2229 > Jan 20 11:06:17 xcp-hoppe snmpd[15173]: Received SNMP packet(s) from UDP: > [192.168.1.1]:2230 > Jan 20 11:06:25 xcp-hoppe python: PERFMON: caught socket.error: (111 > Connection refused) - restarting XAPI session > Jan 20 10:08:36 xcp-hoppe HVM5652[12007]: Time offset set 3569, added > offset -1 > Jan 20 11:08:54 xcp-hoppe python: PERFMON: Caught signal 15 - exiting > Jan 20 11:08:54 xcp-hoppe python: PERFMON: 11 Resource temporarily > unavailable > Jan 20 11:08:54 xcp-hoppe python: PERFMON: Traceback (most recent call > last): > Jan 20 11:08:54 xcp-hoppe python: PERFMON: File > "/opt/xensource/bin/perfmon", line 930, in ? rc = main() > Jan 20 11:08:54 xcp-hoppe python: PERFMON: File > "/opt/xensource/bin/perfmon", line 880, in main cmd = > cmdsock.recv(cmdmaxlen) > Jan 20 11:08:54 xcp-hoppe python: PERFMON: error: (11, 'Resource > temporarily unavailable') > Jan 20 11:08:54 xcp-hoppe python: PERFMON: caught socket.error: (111 > Connection refused) - restarting XAPI session > Jan 20 11:08:55 xcp-hoppe v6d: [ info|xcp-hoppe|0||watchdog] (Re)starting > v6d... > Jan 20 11:08:55 xcp-hoppe xapi: [ info|xcp-hoppe|0 thread_zero||watchdog] > (Re)starting xapi... > Jan 20 10:10:45 xcp-hoppe HVM5641[28792]: Time offset set 3563, added > offset -1 > Jan 20 11:13:45 xcp-hoppe python: PERFMON: caught socket.error: (111 > Connection refused) - restarting XAPI session > > > /var/log/SMI contains: > > [2598] 2012-01-20 11:08:55.277830 VASSR run > ['/opt/xensource/sm/VASSR', > '<methodCall><methodName>sr_get_driver_info</methodName><params><param><value><struct><member><name>host_ref</name><value>OpaqueRef:NULL</value></member><member><name>command</name><value>sr_get_driver_info</value></member><member><name>args</name><value><array><data/></array></value></member><member><name>device_config</name><value><struct/></value></member></struct></value></param></params></methodCall>'] > [2598] 2012-01-20 11:08:55.278332 Warning: vdi_[de]activate present > for vastsky > [2619] 2012-01-20 11:08:55.858537 Warning: vdi_[de]activate present > for dummy > > Not sure there's anything of value in those logs... > > I'm somewhat desperate, as I'm unable to perform any xe commands and am > worried a reboot of the server will not magically bring xapi back online, > meaning the VMs will not start. This XCP host is critical to one of our > customer's infrastructure. It's Friday and I could possibly go there this > evening and perform tasks necessary to bring everything back online, I just > don't quite know what my options are, aside from reboot and perhaps an > in-place install of XCP. > > Does anyone have any bright ideas? I'm all ears!**** > > ** ** >
_______________________________________________ xen-api mailing list [email protected] http://lists.xensource.com/mailman/listinfo/xen-api
