I'm not sure I'd have felt guilty for charging them for the full hour. This seems like a rather obscure issue-not the type of thing you'd have been expected to know (if the latter, then I definitely wouldn't charge). Of course, the next time this happens the client (whoever that client may be) will benefit since you'll be armed with the extra knowledge. :)
John Hornbuckle, MSMIS, PMP MIS Department Taylor County School District www.taylor.k12.fl.us From: David Lum [mailto:[email protected]]<mailto:[mailto:[email protected]]> Sent: Saturday, October 29, 2011 2:31 AM To: NT System Admin Issues Subject: Hyper-V VM's and unnecessary heart failure Remotely working on a %nightjob% client tonight, both a VM and corresponding host unexpectedly drop my LogMeIn connection. I see from LogMeIn that other systems in that room are online, so I know it wasn't the circuit that dropped. Oh joy, I get to drive in (thankfully a short 20 min drive). I get onsite and the host server is halted at the POST screen for the eSATA RAID controller, and the eSATA RAID controller reports a degraded disk on one of the two volumes. Power everything off, pull the drives, disconnect/reconnect the cables, etc. Power it back up and everything shows good. So the host comes up (YAY ½ way there! Well...) and I log in and watch for the VM to start...it gets to 50% then stops, and after 15 minutes (and you know how long 15 minutes is when you're waiting for a *VERY* critical server to come up don'tcha?) the VM goes back to "stopped". As I do full volume backups nightly to the eSATA I'm not too worried yet, but even recovering to that this client would lose a day of work (Internet backups start at 7pm, servers went offline at 5:13pm). A cursory look at the event logs shows nothing exciting, so I change the VM "autostart" from 60 seconds after host OS to 500 seconds and then reboot the host. No change. Joy. Thinking maybe it's an issue on the host I pull a two week old DISK2VHD file that was handier than the backups, I create a new VM on the host and use this VHD. That VM fires up just fine, but it makes me wonder if I can just create a new VM and point to the existing disk files for this critical server. I file that away for plan B. I hit the event logs again, I went through both system and app logs for the timeframe including 30 mins on either side of the start failures (and you know I tried to start that VM more than just those two times...). Somehow I stumbled upon one of Windows 2008's 1 zillion new logs, under Windows logs\Applicaitons and Services logs\Microsoft\Windows\Hyper-V Worker and I found my golden nugget: Log Name: Microsoft-Windows-Hyper-V-Worker-Admin Source: Microsoft-Windows-Hyper-V-Worker Date: 10/28/2011 7:02:45 PM Event ID: 12140 Task Category: None Level: Error Keywords: User: NETWORK SERVICE Computer: Host4.thehosed.one.local Description: 'thehosed.one': Failed to open attachment '\\192.168.116.249\Inst-server\Windows 2008 R2\SW_DVD5_Windows_Svr_DC_EE_SE_Web_2008R2_64-bit_English_X15-59754.ISO'. Error: 'The specified network name is no longer available.' (0x80070040). (Virtual machine 97527135-A765-4700-AF66-C6FE2143391D) Event Xml: Google-Fu then returned a thread to me where someone else was having the same issue because about a VM not starting and it turned out to be a CD-ROM driver issue. Was the VM was failing to start because I had the CD-ROM mapped to a network location that was no longer valid? ARE YOU KIDDING ME? Go into VM settings, remove the CD-ROM from the config and boot the VM. Presto! Took me just over four hours to find the necessary 2-second config change... I charge 1.5x my normal hourly rate to break my routine and drive onsite, somehow I think just one hour is fairhere - sometimes the lesson and the relief that there was zero data loss for the client is reward enough! David Lum Systems Engineer // NWEATM Office 503.548.5229 // Cell (voice/text) 503.267.9764 ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to [email protected]<mailto:[email protected]> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to [email protected]<mailto:[email protected]> with the body: unsubscribe ntsysadmin ~ Finally, powerful endpoint security that ISN'T a resource hog! ~ ~ <http://www.sunbeltsoftware.com/Business/VIPRE-Enterprise/> ~ --- To manage subscriptions click here: http://lyris.sunbelt-software.com/read/my_forums/ or send an email to [email protected] with the body: unsubscribe ntsysadmin
