Re: Using KVM for Windows kernel debugging
On 08/19/2009 01:31 AM, duck wrote: I have never got this to work reliably. Occasionally I can get as far as making a debugger connection at boot-time, IIRC, but have never managed to use the deugger at all. You always seem to end up in some debugger-debuggee deadlock. It works. However, getting into debug session is real pain in the neck. Especially for the first time. I suspect that the serial link simulation is imperfect enough (lost interrupts and thus lost characters?) that you are out of luck for serial-link-based remote debugging, even though userland serial comms, e.g. using Hyperterm, seem to work fine. I was able to get debugging working, a year or two ago, under QEMU using the MIN_CYCLE_COUNT patch (I made a command line option called '-mcc' to allow this to be tuned at run-time) but IIRC this patch is irrelevant when KVM is used. Can anyone suggest a KVM patch which might help, even if it slows the debuggee VM down? Pity SoftICE got killed off :-) You could try http://www.sysersoft.com/ for a one-PC Windows kernel debugging solution. Seems to be a small Beijing software house which just sells on-line. Their product is not free, though there is a free trial. (I haven't tried it under KVM, though they advertise that it works under VxWxxx [censored]). Maybe we can use vm kd extensions project (http://www.nynaeve.net/?page_id=168) as a starting point and fit it into kvm. AFAIK, VmWare and Virtual Box have done it already. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
On 08/18/2009 01:52 PM, Tom Parkin wrote: 2009/8/17 Tom Parkintom.par...@gmail.com: Thanks so much for that, Yan, it looks exactly like what I need. I'll give it a try when I'm back in the office. Having given it a try, I'm having some troubles which I hope someone may be able to assist with ? Here's my configuration : I have two Windows XP hosts running in two virtual machines[0]. One is set up as the debugee to export debugging information via. COM1, the other is set up as the debugger with the WinDBG kernel debugger installed. I have followed the instructions on the Wiki[1] for creating a virtual serial connection between the two VMs, and I am able to send messages between the two VMs using Hyperterm. However, I am unable to successfully establish a connection between the WinDBG debugger process and the debugee machine. The best I've managed so far is as follows : o Boot the debugger VM and start WinDBG o Boot the debugee VM o The debugee boots to the Windows bootloader screen. Immediately after that it appears to hang with a black screen, and it starts chewing CPU o Wait for a short time (~1-2min), after which the WinDBG process crashes on the debugger VM o Restart WinDBG and wait again for a short time (~1-2min). Again, WinDBG crashes o Restart WinDBG a third time. This time the debugger window shows Kernel debugger connection established, although the window status bar still shows Debugee not connected Try to get MS symbols first. o Wait for some time (~5min), during which some further messages come up in the debugger. Eventually it seems to settle into a loop of GetContextState failed with the occasional Unable to read KTHREAD address. Could be a timing issue. Probably host (WinDbg) and target are running out-of-sync. Try to add /break switch to boot.ini or bcdedit. And that appears to be that. I've left it to run for up to ~15 min, during which time the debugee VM window never comes out of the apparent black screen hang, and the debugee kvm process continues to chew CPU, pretty much pegging one of my cores at 100%. My questions: + The Wiki mentions a patch to the kvm-qemu sources[2]. Looking at the git tree it seems this change may be merged, so possibly this patch isn't required any more. Can anyone confirm this ? + Does anyone have a working Windows guest debugging setup working ? Could you share the details ? Of course, any suggestions on how to debug the entire configuration would be gratefully received ! Thanks, Tom [0]. I'm running ubuntu 9.04 with the distro-provided kvm package version 1:84+dfsg-0ubuntu12.3. I'm at somewhat of a loss to relate this to actual kvm-qemu releases... My kernel version is 2.6.28-14-generic. My cpu is a AMD Turion(tm)X2 Ultra DualCore Mobile ZM-86, and I'm running kvm_amd with the option npt=0 to avoid kernel oopses when starting VM images. [1]. As provided by Yan previously; http://kvm.qumranet.com/kvmwiki/WindowsGuestDebug [2]. The link in the Wiki is for a private IP (10.0.0.1) but I think the patch is probably the same as the one referenced here: http://www.damogran.de/blog/archives/14-WinDbg-and-QEMU.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
Hi Vadim, 2009/8/18 Vadim Rozenfeld vroze...@redhat.com: Try to download symbols first. Thanks for the tip, it gets me a bit closer -- although still not fully up and running ! With the symbols installed, and the windbg symbol path set, the windbg process doesn't exit, and does print Connected to Windows XP ... (although the window status bar still reports Debuggee not connected). However, other than that I have similar behavior as previously reported -- the target VM hangs on a black screen chewing CPU, and in time the windbg process starts reporting GetContextState failed Furthermore, I tried using kd rather than windbg to attempt to connect to the target machine: kd.exe -y my-symbols -k comport spec I get different errors from kd: Unable to read head of debugger data list, Win32 error 0n56 Tom -- Tom Parkin www.thhp.org.uk The worst moment for the atheist is when he is really thankful and has nobody to thank /Rossetti/ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
Hi Yan, 2009/8/18 Yan Vugenfirer yvuge...@redhat.com: In case you don’t need to debug the boot process you could wait for the Windows to start and only then start Windbg configure the communication options and hit break (ctrl+break) to initiate debug session. That would be fine; I have no requirement to debug the boot process currently. I'm actually trying to debug a driver segfault which causes a bluescreen on load, and I can control when the driver is loaded using userspace applications. However, when I tried the above, I get: Failed to open \\.\com1 Kernel debugger failed initialisation, Win32 error 0n5 Access is denied. However, I have administrative rights on the machine. Just in case the error actually relates to the target machine, I double checked with kdbgctl.exe that kernel debugging is allowed on the target, and it reports that full access is permitted. Tom -- Tom Parkin www.thhp.org.uk The worst moment for the atheist is when he is really thankful and has nobody to thank /Rossetti/ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
On 08/18/2009 04:54 PM, Tom Parkin wrote: Hi Vadim, 2009/8/18 Vadim Rozenfeldvroze...@redhat.com: Try to download symbols first. Thanks for the tip, it gets me a bit closer -- although still not fully up and running ! With the symbols installed, and the windbg symbol path set, the windbg process doesn't exit, and does print Connected to Windows XP ... (although the window status bar still reports Debuggee not connected). did you try /break switch in boot.ini? However, other than that I have similar behavior as previously reported -- the target VM hangs on a black screen chewing CPU, and in time the windbg process starts reporting GetContextState failed Furthermore, I tried using kd rather than windbg to attempt to connect to the target machine: kd and WinDbg use the same debug engine the only one difference is in user interface. However, kd is much faster. kd.exe -ymy-symbols -kcomport spec I get different errors from kd: Unable to read head of debugger data list, Win32 error 0n56 What is your guest OS? Are you sure you have correct symbols? If you don't need live debugging, you can set up your system to generate crash dump file and use it for post-mortem debugging session. Regards, Vadim Tom -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
2009/8/18 Vadim Rozenfeld vroze...@redhat.com: did you try /break switch in boot.ini? It's a new one to me. I've just given it a go and found no change in behavior, from which I deduce that the target machine isn't getting as far even as the HAL init (although this might be an incorrect reading of the /break option). kd and WinDbg use the same debug engine the only one difference is in user interface. However, kd is much faster. That's worthwhile to know, thanks :-) I was mainly trying kd to see whether it reported any different/extra information on stderr that windbg was silently swallowing. What is your guest OS? Are you sure you got right symbols? My guest OS is Windows XP SP3. I have downloaded WindowsXP-KB936929-SP3-x86-symbols-full-ENU.exe from the Microsoft website, which I think should be correct. I'll keep trying to get the connection up and running. In the meantime, a kernel memory dump combined with after-the-fact debugging should be enough to fix the errors I have currently. Thanks again for your suggestions and advice, Tom -- Tom Parkin www.thhp.org.uk The worst moment for the atheist is when he is really thankful and has nobody to thank /Rossetti/ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
I have never got this to work reliably. Occasionally I can get as far as making a debugger connection at boot-time, IIRC, but have never managed to use the deugger at all. You always seem to end up in some debugger-debuggee deadlock. I suspect that the serial link simulation is imperfect enough (lost interrupts and thus lost characters?) that you are out of luck for serial-link-based remote debugging, even though userland serial comms, e.g. using Hyperterm, seem to work fine. I was able to get debugging working, a year or two ago, under QEMU using the MIN_CYCLE_COUNT patch (I made a command line option called '-mcc' to allow this to be tuned at run-time) but IIRC this patch is irrelevant when KVM is used. Can anyone suggest a KVM patch which might help, even if it slows the debuggee VM down? Pity SoftICE got killed off :-) You could try http://www.sysersoft.com/ for a one-PC Windows kernel debugging solution. Seems to be a small Beijing software house which just sells on-line. Their product is not free, though there is a free trial. (I haven't tried it under KVM, though they advertise that it works under VxWxxx [censored]). -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
2009/8/17 Tom Parkin tom.par...@gmail.com: Has anyone on the list got any experience of this kind of setup, or any suggestions that might help ? Sorry for the spam, but I forgot to mention that I found this mail in the archives which looks pertinent: http://article.gmane.org/gmane.comp.emulators.kvm.devel/20770/match=windows Sadly, the wiki link is no longer valid: http://kvm.qumranet.com/kvmwiki/WindowsGuestDebug If anyone could tell me where that information now resides I'd appreciate it. Thanks again ! Tom -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
(copying smintz) On 08/17/2009 06:16 PM, Tom Parkin wrote: 2009/8/17 Tom Parkintom.par...@gmail.com: Has anyone on the list got any experience of this kind of setup, or any suggestions that might help ? Sorry for the spam, but I forgot to mention that I found this mail in the archives which looks pertinent: http://article.gmane.org/gmane.comp.emulators.kvm.devel/20770/match=windows Sadly, the wiki link is no longer valid: http://kvm.qumranet.com/kvmwiki/WindowsGuestDebug If anyone could tell me where that information now resides I'd appreciate it. This wiki is not in http://www.linux-kvm.org. Shahar, have we lost that page? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Using KVM for Windows kernel debugging
-Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Tom Parkin Sent: Monday, August 17, 2009 6:16 PM To: kvm@vger.kernel.org Cc: tom.par...@gmail.com Subject: Re: Using KVM for Windows kernel debugging 2009/8/17 Tom Parkin tom.par...@gmail.com: Has anyone on the list got any experience of this kind of setup, or any suggestions that might help ? Sorry for the spam, but I forgot to mention that I found this mail in the archives which looks pertinent: http://article.gmane.org/gmane.comp.emulators.kvm.devel/20770/match=win dows Sadly, the wiki link is no longer valid: http://kvm.qumranet.com/kvmwiki/WindowsGuestDebug If anyone could tell me where that information now resides I'd appreciate it. [YV] I recreated the page: http://www.linux-kvm.org/page/WindowsGuestDrivers/GuestDebugging Thanks again ! Tom -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Using KVM for Windows kernel debugging
Thanks so much for that, Yan, it looks exactly like what I need. I'll give it a try when I'm back in the office. Best regards, Tom On 18/08/2009, Yan Vugenfirer yvuge...@redhat.com wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Tom Parkin Sent: Monday, August 17, 2009 6:16 PM To: kvm@vger.kernel.org Cc: tom.par...@gmail.com Subject: Re: Using KVM for Windows kernel debugging 2009/8/17 Tom Parkin tom.par...@gmail.com: Has anyone on the list got any experience of this kind of setup, or any suggestions that might help ? Sorry for the spam, but I forgot to mention that I found this mail in the archives which looks pertinent: http://article.gmane.org/gmane.comp.emulators.kvm.devel/20770/match=win dows Sadly, the wiki link is no longer valid: http://kvm.qumranet.com/kvmwiki/WindowsGuestDebug If anyone could tell me where that information now resides I'd appreciate it. [YV] I recreated the page: http://www.linux-kvm.org/page/WindowsGuestDrivers/GuestDebugging Thanks again ! Tom -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- Sent from my mobile device Tom Parkin www.thhp.org.uk The worst moment for the atheist is when he is really thankful and has nobody to thank /Rossetti/ -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
