Re: Exposing host debug capabilities to userspace
Paolo Bonzini pbonz...@redhat.com writes: On 25/11/2014 17:35, Alexander Graf wrote: Unfortunately on ARM you also have a few other constraints - the debug register space is partitioned into magic super debug registers at the top (with an implementation specific amount) and normal debug registers in the lower end of the space. Does the gdbstub ever need to use the magic super debug registers? Even if it does, the logic is the same as x86. On x86 you might run out of breakpoints, on ARM you might run out of breakpoints in general or magic super breakpoints in particular. By default gdbstub uses normal software breakpoints unless a) the user asks for a HW one or b) the memory is RO (i.e a ROM). For watchpoints the situation is reversed and they are used by default if available. I would just treat gdbstub debugging as the ugly step child that it is and not introduce anything special for it (except for debug event deflection to user space). Then only ever work on guest debug registers and call it a day. Chances are just too high that we get the interfaces wrong and shoot ourselves in the foot. I agree. But we do need a way to retrieve the number of valid fields in struct kvm_guest_debug_arch, if that is not architecturally defined (e.g. on x86 it's just always 4). It's IMPDEF (implementation defined) up to a maximum of 16. A hidden ONE_REG, whose existence is determined by (ARM || ARM64) kvm_check_extension(KVM_CAP_SET_GUEST_DEBUG), is certainly fine and I like it because it doesn't pollute the global KVM_CAP_* namespace. My original implementation more or less did that but my main worry is migration code tends to iterate over the whole ONE_REG list and introducing a value that doesn't happen to be strictly the guests is semantically impure. Of course if you mean by hidden don't export it via GET_REG_LIST then I guess that would work. Does seem a little ugly as the internal KVM code now needs to learn about hidden and non-hidden registers. The alternative is a capability; however, if you start with two capabilities you'll end up needing four, and then realize that returning a struct via ONE_REG would have been a better idea. :) We already have how many breakpoints, how many watchpoints, how many magic super debug registers,... Only two, not sure what the super debug registers are? Certainly on the QEMU side the capability based approach is beautifully simple: max_hw_wp = kvm_check_extension(cs-kvm_state, KVM_CAP_GUEST_DEBUG_HW_WPS); max_hw_bp = kvm_check_extension(cs-kvm_state, KVM_CAP_GUEST_DEBUG_HW_BPS); thanks to kvm_check_extension zeroing failure modes. And in it's defence it's a generic enough capability to be used across any other architectures that need to expose this information compared to the many PPC specific capabilities. Perhaps a KVM_ARCH_EXTENSION ioctl would be more useful in reducing the growth of the space? Paolo -- Alex Bennée -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 25 November 2014 at 16:50, Paolo Bonzini pbonz...@redhat.com wrote: On 25/11/2014 17:35, Alexander Graf wrote: Unfortunately on ARM you also have a few other constraints - the debug register space is partitioned into magic super debug registers at the top (with an implementation specific amount) and normal debug registers in the lower end of the space. Does the gdbstub ever need to use the magic super debug registers? No. The extra features of the trailing registers in the range are context matching, which means you can set them up as breakpoint on this context ID or breakpoint on this VMID [read: breakpoints specific to a particular process ID or particular VM], generally as linked breakpoints adding a constraint to a plain address-match breakpoint register. The gdbstub doesn't use context bps and I don't think Linux does either (though of course it might in future). -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24/11/2014 14:59, Alex Bennée wrote: Alexander Graf pointed out that KVM_CHECK_EXTENSION can return any positive number for success. How about using: max_hw_bps = kvm_check_extension(kvm_state, KVM_CAP_GUEST_DEBUG_HW_BPS); max_hw_wps = kvm_check_extension(kvm_state, KVM_CAP_GUEST_DEBUG_HW_WPS); Seems pretty sane, doesn't change the semantics of an API and is architecture agnostic if others need the number? Yes, this was going to be my suggestion as well. Just I would use a bitmask in case some register can act as both breakpoint and watchpoint. On x86, each of the four bp/wp registers (each register can act as both) can be used for either guest or gdbstub debugging. If the KVM_GUESTDBG_USE_HW_BP feature is enabled, the guest is entered with made up debug register contents, that we pass via KVM_SET_GUEST_DEBUG's struct kvm_guest_debug_arch. Otherwise, the guest is entered with real debug register contents passed via KVM_SET_DEBUGREGS. Reads/writes of the debug registers trap to KVM (which helps the guest see the expected values of the debug registers in the former case). There is no KVM_GET_GUEST_DEBUG because the corresponding info is passed via struct kvm_debug_exit_arch. If gdbstub hardware breakpoints are enabled, all hardware breakpoints exit to userspace. QEMU then decides whether the breakpoint came from guest debugging (and then injects an exception), or from gdbstub debugging (and then suspends execution). Same for software breakpoints. If the total request is 4 hardware breakpoints, someone will have to lose and some gdbstub breakpoints will be missed. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 25.11.14 17:21, Paolo Bonzini wrote: On 24/11/2014 14:59, Alex Bennée wrote: Alexander Graf pointed out that KVM_CHECK_EXTENSION can return any positive number for success. How about using: max_hw_bps = kvm_check_extension(kvm_state, KVM_CAP_GUEST_DEBUG_HW_BPS); max_hw_wps = kvm_check_extension(kvm_state, KVM_CAP_GUEST_DEBUG_HW_WPS); Seems pretty sane, doesn't change the semantics of an API and is architecture agnostic if others need the number? Yes, this was going to be my suggestion as well. Just I would use a bitmask in case some register can act as both breakpoint and watchpoint. On x86, each of the four bp/wp registers (each register can act as both) can be used for either guest or gdbstub debugging. If the KVM_GUESTDBG_USE_HW_BP feature is enabled, the guest is entered with made up debug register contents, that we pass via KVM_SET_GUEST_DEBUG's struct kvm_guest_debug_arch. Otherwise, the guest is entered with real debug register contents passed via KVM_SET_DEBUGREGS. Reads/writes of the debug registers trap to KVM (which helps the guest see the expected values of the debug registers in the former case). There is no KVM_GET_GUEST_DEBUG because the corresponding info is passed via struct kvm_debug_exit_arch. If gdbstub hardware breakpoints are enabled, all hardware breakpoints exit to userspace. QEMU then decides whether the breakpoint came from guest debugging (and then injects an exception), or from gdbstub debugging (and then suspends execution). Same for software breakpoints. If the total request is 4 hardware breakpoints, someone will have to lose and some gdbstub breakpoints will be missed. Unfortunately on ARM you also have a few other constraints - the debug register space is partitioned into magic super debug registers at the top (with an implementation specific amount) and normal debug registers in the lower end of the space. The main pain I have with exposing host information is that it's going to be interesting and challenging enough to get all of this right merely for the guest debug register space. Exposing the host debug register space as well means there is even more space for breakage. I would just treat gdbstub debugging as the ugly step child that it is and not introduce anything special for it (except for debug event deflection to user space). Then only ever work on guest debug registers and call it a day. Chances are just too high that we get the interfaces wrong and shoot ourselves in the foot. Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24/11/2014 15:52, Christoffer Dall wrote: We had a lenghty IRC discussion on this, for the curious, go read it here: http://irclogs.linaro.org/2014/11/24/%23kvm-arm.html Some notes... chazy but saying that you want to design an ABI that potentially exposes fewer debug registers to gdbstubs than your hardware supports and breaks gdbstub support if you happen to support emulation of a cpu with more debug registers in the future, because you want to re-use ONE_REG is not a great approach either agraf chazy: but you wouldn't even remotely think of doing it, no? agraf chazy: I'm saying that QEMU shouldn't have access to the excess registers chazy agraf: I wouldn’t even remotely think of doing nested virtualization, but people have agraf chazy: QEMU spawns an A57, it gets 8 debug registers chazy agraf: I understand, but I don’t agree with your rationale agraf chazy: not QEMU spawns an A57 on an A67, so it gets 8 real and 8 hidden debug registers that it also needs to be aware of mappings of I disagree with Alex. QEMU can use the 16 debug registers as it wishes, since it sets them with KVM_SET_GUEST_DEBUG. The guest's eight can be mapped to the last 8 if those are the required semantics for the architecture. Just exit to QEMU if you do a debug register write while gdbstub debugging is active; QEMU gets the contents of the 8 VCPU-visible registers with ONE_REG (equivalent of x86 GET_DEBUGREGS); calls KVM_SET_GUEST_DEBUG to reflect them in the 16-register state; and restarts. It works as long as you can trap both reads and writes. chazy ajb-linaro: don’t migrate a guest that is getting debugged is the answer to that one I believe ajb-linarochazy: at least with the overlay approach that happens automatically ajb-linarothe overlay isn't migrated and the new host isn't calling SET_GUEST_DEBUG so whatever the debug registers where before are restorerd Right. The destination will lose the hardware breakpoints/watchpoints because it starts the guest with KVM_SET_GUEST_DEBUG. Apart from losing them, everything should work fine. The dest sets the 8 VCPU-visible registers with ONE_REG, the hypervisor reflects them in a subset of the 16 physical registers, and that's it. Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 25/11/2014 17:35, Alexander Graf wrote: Unfortunately on ARM you also have a few other constraints - the debug register space is partitioned into magic super debug registers at the top (with an implementation specific amount) and normal debug registers in the lower end of the space. Does the gdbstub ever need to use the magic super debug registers? Even if it does, the logic is the same as x86. On x86 you might run out of breakpoints, on ARM you might run out of breakpoints in general or magic super breakpoints in particular. I would just treat gdbstub debugging as the ugly step child that it is and not introduce anything special for it (except for debug event deflection to user space). Then only ever work on guest debug registers and call it a day. Chances are just too high that we get the interfaces wrong and shoot ourselves in the foot. I agree. But we do need a way to retrieve the number of valid fields in struct kvm_guest_debug_arch, if that is not architecturally defined (e.g. on x86 it's just always 4). A hidden ONE_REG, whose existence is determined by (ARM || ARM64) kvm_check_extension(KVM_CAP_SET_GUEST_DEBUG), is certainly fine and I like it because it doesn't pollute the global KVM_CAP_* namespace. The alternative is a capability; however, if you start with two capabilities you'll end up needing four, and then realize that returning a struct via ONE_REG would have been a better idea. :) We already have how many breakpoints, how many watchpoints, how many magic super debug registers,... Paolo -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24 November 2014 at 11:16, Alex Bennée alex@zen.linaro.local.i-did-not-set--mail-host-address--so-tickle-me wrote: ^^^ :-) Alex Bennée alex.ben...@linaro.org writes: * KVM ioctl KVM_GET_DEBUGREGS This is currently x86 only and looks like it's more aimed at debug registers than capability stuff. Also I'm not sure what the state of this ioctl is compared to KVM_SET_GUEST_DEBUG. Do these APIs overlap or is one an older deprecated x86 only API? I'm minded to re-use this ioctl and define it for ARM as reading the host debug architecture state ID_AA64DFR0/1_EL1. Currently for x86 it's used for getting vcpu debug registers which on ARM is handled via the GET/SET one reg interface. This seems a bit odd. Either the x86 use of this ioctl is for accessing guest state, in which case using it on ARM for host state is a bit weird, or else why is x86 doing its debug via host state and ARM using guest state? It may well still be the best choice, but it just feels like maybe something isn't lined up right... -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
Fixed CC:kvmarm, Added: Alexander Graf, Fixed: my From: Replying to myself with additional information on each option Alex Bennée alex.ben...@linaro.org writes: Hi, I've almost finished the ARMv8 guest debug support but I have one problem left to solve. userspace needs to know how many hardware debug registers are available for GDB to use. This information is available from the ID_AA64DFR0_EL1 register. Currently I abuse GET_ONE_REG to fetch it's value however semantically this is poor as it's API is for getting guest state not host state and they could theoretically have different values. So far the options I've examined are: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) Nope, guest state API * ptrace(PTRACE_GETREGSET, NT_ARM_HW_WATCH) Nope, ptrace requires attachment and you can't attach to your own thread group. * KVM ioctl KVM_GET_DEBUGREGS This is currently x86 only and looks like it's more aimed at debug registers than capability stuff. Also I'm not sure what the state of this ioctl is compared to KVM_SET_GUEST_DEBUG. Do these APIs overlap or is one an older deprecated x86 only API? I'm minded to re-use this ioctl and define it for ARM as reading the host debug architecture state ID_AA64DFR0/1_EL1. Currently for x86 it's used for getting vcpu debug registers which on ARM is handled via the GET/SET one reg interface. * Export the information via sysfs I suppose the correct canonical non-subsystem specific way to make this information available it to expose the data in some sort of sysfs node? However I don't see any existing sysfs structure for the CPU. I suspect this would get complicated depending on the architecture. * Expand /proc/cpuinfo I suspect adding extra text to be badly parsed by userspace is just horrid and unacceptable behaviour ;-) * Add another KVM ioctl? This would have the downside of being specific to KVM and of course proliferating the API space again. So unless there are any objections my intention to re-use the existing API calls for ARM architectures. -- Alex Bennée -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On Mon, Nov 24, 2014 at 12:21 PM, Peter Maydell peter.mayd...@linaro.org wrote: On 24 November 2014 at 11:16, Alex Bennée alex@zen.linaro.local.i-did-not-set--mail-host-address--so-tickle-me wrote: ^^^ :-) Alex Bennée alex.ben...@linaro.org writes: * KVM ioctl KVM_GET_DEBUGREGS This is currently x86 only and looks like it's more aimed at debug registers than capability stuff. Also I'm not sure what the state of this ioctl is compared to KVM_SET_GUEST_DEBUG. Do these APIs overlap or is one an older deprecated x86 only API? I'm minded to re-use this ioctl and define it for ARM as reading the host debug architecture state ID_AA64DFR0/1_EL1. Currently for x86 it's used for getting vcpu debug registers which on ARM is handled via the GET/SET one reg interface. This seems a bit odd. Either the x86 use of this ioctl is for accessing guest state, in which case using it on ARM for host state is a bit weird, or else why is x86 doing its debug via host state and ARM using guest state? It may well still be the best choice, but it just feels like maybe something isn't lined up right... It seems weird, agreed, but somehow what we're left with except for adding a new ioctl. I think part of the explanation may simply be that x86 solves this problem in an inherently different way. -Christoffer -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24.11.14 12:35, Alex Bennée wrote: Fixed CC:kvmarm, Added: Alexander Graf, Fixed: my From: Replying to myself with additional information on each option Alex Bennée alex.ben...@linaro.org writes: Hi, I've almost finished the ARMv8 guest debug support but I have one problem left to solve. userspace needs to know how many hardware debug registers are available for GDB to use. This information is available from the ID_AA64DFR0_EL1 register. Currently I abuse GET_ONE_REG to fetch it's value however semantically this is poor as it's API is for getting guest state not host state and they could theoretically have different values. So far the options I've examined are: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) Nope, guest state API What's the problem with using ONE_REG for this? After all, the total number of available guest debug register is a guest vcpu property of some sort. Btw, looking through the mess that we have today with asymmetric SET_GUEST_DEBUG and GET_DEBUGREGS ioctls I can't shake off the feeling that we're best off just doing all of the debug register sync via ONE_REGs. That way we at least have a guaranteed symmetric interface that can get and set values, so we won't trip over it on live migration. Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24 November 2014 at 12:26, Alexander Graf ag...@suse.de wrote: On 24.11.14 12:35, Alex Bennée wrote: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) Nope, guest state API What's the problem with using ONE_REG for this? After all, the total number of available guest debug register is a guest vcpu property of some sort. Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: On 24 November 2014 at 12:26, Alexander Graf ag...@suse.de wrote: On 24.11.14 12:35, Alex Bennée wrote: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) Nope, guest state API What's the problem with using ONE_REG for this? After all, the total number of available guest debug register is a guest vcpu property of some sort. Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. Alex -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. -- PMM -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24.11.14 13:44, Peter Maydell wrote: On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. No, we don't want to know how many debug registers the host has. We want to know how many debug registers the guest has. Imagine you're running on A57 today with 8 debug registers (no idea if that's true, but assume it is). Tomorrow there will be a new core - let's call it A67 - with 16 debug registers. To make sure your legacy, badly written guest behaves exactly the same - especially after live migration - you want to spawn a VM with -cpu A57. That implies you want to expose 8 debug registers into the guest. So debug register synchronization should only be aware of those 8 registers. So what we really care about is the number of debug registers available to a guest vcpu. That in turn means it's guest state and as such can easily go into ONE_REG. Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
Alexander Graf ag...@suse.de writes: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: On 24 November 2014 at 12:26, Alexander Graf ag...@suse.de wrote: On 24.11.14 12:35, Alex Bennée wrote: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) Nope, guest state API What's the problem with using ONE_REG for this? After all, the total number of available guest debug register is a guest vcpu property of some sort. Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. Does the GET/SET_ONE_REG have the concept of a read-only register? I'd be worried by code just blindly iterating over the lists getting thrown when one doesn't work. Alex -- PMM -- Alex Bennée -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On Mon, Nov 24, 2014 at 1:51 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:44, Peter Maydell wrote: On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. No, we don't want to know how many debug registers the host has. We want to know how many debug registers the guest has. Imagine you're running on A57 today with 8 debug registers (no idea if that's true, but assume it is). Tomorrow there will be a new core - let's call it A67 - with 16 debug registers. To make sure your legacy, badly written guest behaves exactly the same - especially after live migration - you want to spawn a VM with -cpu A57. That implies you want to expose 8 debug registers into the guest. So debug register synchronization should only be aware of those 8 registers. So what we really care about is the number of debug registers available to a guest vcpu. That in turn means it's guest state and as such can easily go into ONE_REG. We already export this for the guest via ONE_REG. What we want to do is support gdbstubs in QEMU to debug the guest, and to do this, QEMU needs to know how many hardware registers on the host there is; the guest will never see this information. So this is really about the host, the guest side is trivially handled through ONE_REG. -Christoffer -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On Mon, Nov 24, 2014 at 1:53 PM, Alex Bennée alex.ben...@linaro.org wrote: Alexander Graf ag...@suse.de writes: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: On 24 November 2014 at 12:26, Alexander Graf ag...@suse.de wrote: On 24.11.14 12:35, Alex Bennée wrote: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) Nope, guest state API What's the problem with using ONE_REG for this? After all, the total number of available guest debug register is a guest vcpu property of some sort. Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. Does the GET/SET_ONE_REG have the concept of a read-only register? I'd be worried by code just blindly iterating over the lists getting thrown when one doesn't work. yes, we have invariant cp15 registers. -Christoffer -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24.11.14 13:53, Christoffer Dall wrote: On Mon, Nov 24, 2014 at 1:51 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:44, Peter Maydell wrote: On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. No, we don't want to know how many debug registers the host has. We want to know how many debug registers the guest has. Imagine you're running on A57 today with 8 debug registers (no idea if that's true, but assume it is). Tomorrow there will be a new core - let's call it A67 - with 16 debug registers. To make sure your legacy, badly written guest behaves exactly the same - especially after live migration - you want to spawn a VM with -cpu A57. That implies you want to expose 8 debug registers into the guest. So debug register synchronization should only be aware of those 8 registers. So what we really care about is the number of debug registers available to a guest vcpu. That in turn means it's guest state and as such can easily go into ONE_REG. We already export this for the guest via ONE_REG. What we want to do is support gdbstubs in QEMU to debug the guest, and to do this, QEMU needs to know how many hardware registers on the host there is; the guest will never see this information. So this is really about the host, the guest side is trivially handled through ONE_REG. That's the cp15 register that happens to get exposed to the guest. You can just add another ONE_REG that does not have a cp15 equivalent to expose the number of the vcpu's actually available debug registers. Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On Mon, Nov 24, 2014 at 1:56 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:53, Christoffer Dall wrote: On Mon, Nov 24, 2014 at 1:51 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:44, Peter Maydell wrote: On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. No, we don't want to know how many debug registers the host has. We want to know how many debug registers the guest has. Imagine you're running on A57 today with 8 debug registers (no idea if that's true, but assume it is). Tomorrow there will be a new core - let's call it A67 - with 16 debug registers. To make sure your legacy, badly written guest behaves exactly the same - especially after live migration - you want to spawn a VM with -cpu A57. That implies you want to expose 8 debug registers into the guest. So debug register synchronization should only be aware of those 8 registers. So what we really care about is the number of debug registers available to a guest vcpu. That in turn means it's guest state and as such can easily go into ONE_REG. We already export this for the guest via ONE_REG. What we want to do is support gdbstubs in QEMU to debug the guest, and to do this, QEMU needs to know how many hardware registers on the host there is; the guest will never see this information. So this is really about the host, the guest side is trivially handled through ONE_REG. That's the cp15 register that happens to get exposed to the guest. You can just add another ONE_REG that does not have a cp15 equivalent to expose the number of the vcpu's actually available debug registers. The fact that we currently map the guest vcpu registers to that of the host doesn't mean that will always be the case (which you argued for above). If you migrate a VM from a CPU with 16 debug registers (or emulate a CPU with 16 debug registers) on a physical CPU with only 8 debug registers, you cannot tell QEMU that it has 16 debug registers on the hardware to configure to debug the guest, which is what ONE_REG would give you. You could add another set of registers to ONE_REG which says these are the host versions, or you could say that you don't ever support a setup where the guest number of debug registers is not the same as the host number, but both would be wrong. -Christoffer -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
Alex Bennée alex.ben...@linaro.org writes: snip Alex Bennée alex.ben...@linaro.org writes: Hi, I've almost finished the ARMv8 guest debug support but I have one problem left to solve. userspace needs to know how many hardware debug registers are available for GDB to use. This information is available from the ID_AA64DFR0_EL1 register. Snip So far the options I've examined are: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) * ptrace(PTRACE_GETREGSET, NT_ARM_HW_WATCH) * KVM ioctl KVM_GET_DEBUGREGS * Export the information via sysfs * Expand /proc/cpuinfo * Add another KVM ioctl? Alexander Graf pointed out that KVM_CHECK_EXTENSION can return any positive number for success. How about using: max_hw_bps = kvm_check_extension(kvm_state, KVM_CAP_GUEST_DEBUG_HW_BPS); max_hw_wps = kvm_check_extension(kvm_state, KVM_CAP_GUEST_DEBUG_HW_WPS); Seems pretty sane, doesn't change the semantics of an API and is architecture agnostic if others need the number? -- Alex Bennée -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On 24.11.14 14:10, Christoffer Dall wrote: On Mon, Nov 24, 2014 at 1:56 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:53, Christoffer Dall wrote: On Mon, Nov 24, 2014 at 1:51 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:44, Peter Maydell wrote: On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. No, we don't want to know how many debug registers the host has. We want to know how many debug registers the guest has. Imagine you're running on A57 today with 8 debug registers (no idea if that's true, but assume it is). Tomorrow there will be a new core - let's call it A67 - with 16 debug registers. To make sure your legacy, badly written guest behaves exactly the same - especially after live migration - you want to spawn a VM with -cpu A57. That implies you want to expose 8 debug registers into the guest. So debug register synchronization should only be aware of those 8 registers. So what we really care about is the number of debug registers available to a guest vcpu. That in turn means it's guest state and as such can easily go into ONE_REG. We already export this for the guest via ONE_REG. What we want to do is support gdbstubs in QEMU to debug the guest, and to do this, QEMU needs to know how many hardware registers on the host there is; the guest will never see this information. So this is really about the host, the guest side is trivially handled through ONE_REG. That's the cp15 register that happens to get exposed to the guest. You can just add another ONE_REG that does not have a cp15 equivalent to expose the number of the vcpu's actually available debug registers. The fact that we currently map the guest vcpu registers to that of the host doesn't mean that will always be the case (which you argued for above). If you migrate a VM from a CPU with 16 debug registers (or emulate a CPU with 16 debug registers) on a physical CPU with only 8 debug registers, you cannot tell QEMU that it has 16 debug registers on the hardware to configure to debug the guest, which is what ONE_REG would give you. You could add another set of registers to ONE_REG which says these are the host versions, or you could say that you don't ever support a setup where the guest number of debug registers is not the same as the host number, but both would be wrong. Why would QEMU ever want to access debug registers that it didn't ask for in the first place? If we simply limit ourselves to the register set the vcpu has we don't have any problems and the complexity matrix shrinks noticably, no? Alex -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
On Mon, Nov 24, 2014 at 03:07:35PM +0100, Alexander Graf wrote: On 24.11.14 14:10, Christoffer Dall wrote: On Mon, Nov 24, 2014 at 1:56 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:53, Christoffer Dall wrote: On Mon, Nov 24, 2014 at 1:51 PM, Alexander Graf ag...@suse.de wrote: On 24.11.14 13:44, Peter Maydell wrote: On 24 November 2014 at 12:41, Alexander Graf ag...@suse.de wrote: Am 24.11.2014 um 13:32 schrieb Peter Maydell peter.mayd...@linaro.org: Yes, but we don't want to know about properties of the guest vCPU. In an ideal world QEMU could reserve say half the debug registers for debugging the VM on startup and have KVM expose ID registers indicating to the guest that it only had the other half... Yup, so create another (read-only) ONE_REG that exposes the number of actual guest debug registers. I'm confused. ONE_REG is for guest state, and the ID register by definition is how we tell the guest how many debug registers it has. What we want to know (and perhaps even control) for debugging the VM is how many debug registers the host has. No, we don't want to know how many debug registers the host has. We want to know how many debug registers the guest has. Imagine you're running on A57 today with 8 debug registers (no idea if that's true, but assume it is). Tomorrow there will be a new core - let's call it A67 - with 16 debug registers. To make sure your legacy, badly written guest behaves exactly the same - especially after live migration - you want to spawn a VM with -cpu A57. That implies you want to expose 8 debug registers into the guest. So debug register synchronization should only be aware of those 8 registers. So what we really care about is the number of debug registers available to a guest vcpu. That in turn means it's guest state and as such can easily go into ONE_REG. We already export this for the guest via ONE_REG. What we want to do is support gdbstubs in QEMU to debug the guest, and to do this, QEMU needs to know how many hardware registers on the host there is; the guest will never see this information. So this is really about the host, the guest side is trivially handled through ONE_REG. That's the cp15 register that happens to get exposed to the guest. You can just add another ONE_REG that does not have a cp15 equivalent to expose the number of the vcpu's actually available debug registers. The fact that we currently map the guest vcpu registers to that of the host doesn't mean that will always be the case (which you argued for above). If you migrate a VM from a CPU with 16 debug registers (or emulate a CPU with 16 debug registers) on a physical CPU with only 8 debug registers, you cannot tell QEMU that it has 16 debug registers on the hardware to configure to debug the guest, which is what ONE_REG would give you. You could add another set of registers to ONE_REG which says these are the host versions, or you could say that you don't ever support a setup where the guest number of debug registers is not the same as the host number, but both would be wrong. Why would QEMU ever want to access debug registers that it didn't ask for in the first place? If we simply limit ourselves to the register set the vcpu has we don't have any problems and the complexity matrix shrinks noticably, no? (For others following this, not hanging out on #kvm-arm, I provide a summary). We had a lenghty IRC discussion on this, for the curious, go read it here: http://irclogs.linaro.org/2014/11/24/%23kvm-arm.html The point of confusion is that other KVM architectures use the ONE_REG interface to set the debug registers, for both guest and host state, and in fact this can be overlayed. The propose idea of using SET_DEBUGREGS was flawed, because this also pertains to guest state and is a deprecated ABI. So that left us with two choices: (1) Implement a new ABI to retrieve and set host debugging independently of the guest state. (2) Overlay the host debugging of a guest with the guest's internal debugging state (a process inside the guest is debugging a process). Option (1) has the usual drawback of having to design an ABI, consumes the ioctl number space etc. But it also has (the more severe) drawback that debugging the guest using QEMU gdbstubs breaks guest debugging, because we would keep completely distinct register sets. Option (2) feels semantically slightly weird in the cases where the number of guest debug regs differs from the number of host debug regs, but as Alex Graf pointed out, we probably wouldn't dream of supporting more guest debug registers than the host has (refuse migration, don't emulate that VCPU on the PCPU etc.), and we don't care deeply about exposing fewer host debug regs than actually available. Migrating a VM that is being debugged by the host is simply not supported, so any state obtained via ONE_REG for
Re: Exposing host debug capabilities to userspace
On Thu, Nov 20, 2014 at 04:55:14PM +, Alex Bennée wrote: Hi, I've almost finished the ARMv8 guest debug support but I have one problem left to solve. userspace needs to know how many hardware debug registers are available for GDB to use. This information is available from the ID_AA64DFR0_EL1 register. Currently I abuse GET_ONE_REG to fetch it's value however semantically this is poor as it's API is for getting guest state not host state and they could theoretically have different values. So far the options I've examined are: * KVM ioctl GET_ONE_REG(ID_AA64DFR0_EL1) As explained above, abusing a guest state API for host configuration. It's just wrong, and we should only do this if there's absolutely no other way to do this. * ptrace(PTRACE_GETREGSET, NT_ARM_HW_WATCH) This is used by GDB to access the host details in debug-monitors. However the ptrace API really wants you to attach to a process before calling PTRACE_GETREGSET. Currently I've tried attaching to the thread_id of the vCPU but this fails with EPERM, I suspect because attaching to your own threads likely upsets the kernel. Can you confirm your suspicion? This seems like a rather good approach so we should really investigate why this doesn't work and explore ways to get it working. * KVM ioctl KVM_GET_DEBUGREGS This is currently x86 only and looks like it's more aimed at debug registers than capability stuff. Also I'm not sure what the state of this ioctl is compared to KVM_SET_GUEST_DEBUG. Do these APIs overlap or is one an older deprecated x86 only API? The API text and a brief glance of the x86 code seems to indicate that this is also the vcpu state... * Export the information via sysfs I suppose the correct canonical non-subsystem specific way to make this information available it to expose the data in some sort of sysfs node? However I don't see any existing sysfs structure for the CPU. * Expand /proc/cpuinfo I suspect adding extra text to be badly parsed by userspace is just horrid and unacceptable behaviour ;-) * Add another KVM ioctl? This would have the downside of being specific to KVM and of course proliferating the API space again. This may not be that bad, for example, could we ever imaging that we'd only want to export a few of the debug registers for host gdbstub usage? -Christoffer -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
Christoffer Dall christoffer.d...@linaro.org writes: On Thu, Nov 20, 2014 at 04:55:14PM +, Alex Bennée wrote: Hi, I've almost finished the ARMv8 guest debug support but I have one problem left to solve. userspace needs to know how many hardware debug registers are available for GDB to use. * KVM ioctl KVM_GET_DEBUGREGS This is currently x86 only and looks like it's more aimed at debug registers than capability stuff. Also I'm not sure what the state of this ioctl is compared to KVM_SET_GUEST_DEBUG. Do these APIs overlap or is one an older deprecated x86 only API? The API text and a brief glance of the x86 code seems to indicate that this is also the vcpu state... Yeah I was getting confused as to the difference between the two API calls. Is this just an x86 version of what GET/SET_ONE_REG replaced? * Add another KVM ioctl? This would have the downside of being specific to KVM and of course proliferating the API space again. This may not be that bad, for example, could we ever imaging that we'd only want to export a few of the debug registers for host gdbstub usage? However it is general information which might be useful to the whole system (although I suspect KVM and PTRACE are the only two). It would be a shame to have an informational API wrapped up in the extra boiler-plate of a specific API. -- Alex Bennée -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exposing host debug capabilities to userspace
Christoffer Dall christoffer.d...@linaro.org writes: On Thu, Nov 20, 2014 at 04:55:14PM +, Alex Bennée wrote: snip * ptrace(PTRACE_GETREGSET, NT_ARM_HW_WATCH) This is used by GDB to access the host details in debug-monitors. However the ptrace API really wants you to attach to a process before calling PTRACE_GETREGSET. Currently I've tried attaching to the thread_id of the vCPU but this fails with EPERM, I suspect because attaching to your own threads likely upsets the kernel. Can you confirm your suspicion? This seems like a rather good approach so we should really investigate why this doesn't work and explore ways to get it working. From ptrace_attach: retval = -EPERM; if (unlikely(task-flags PF_KTHREAD)) goto out; if (same_thread_group(task, current)) goto out; I think this is what is triggering my EPERM. I'm going to dig into the history of code around that bit. While I can see it might be undesirable I'm not sure if it has to be verbotten... -- Alex Bennée -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
