[XenPPC] Re: [Xen-devel] schedule() vs softirqs
On Fri, 2006-12-15 at 21:39 +, Keir Fraser wrote: > On 15/12/06 20:41, "Hollis Blanchard" <[EMAIL PROTECTED]> wrote: > > > It's an issue with any architecture with a large number of registers > > which aren't automatically saved by hardware (and a C ABI that makes > > some of them non-volatile). > > > > x86 has a small number of registers. ia64 automatically saves them (from > > what I understand). So of the currently-supported architectures, yes, > > that leaves PowerPC. > > I see. It sounds like returning from context_switch() is perhaps the right > thing for powerpc. That would be easier if you have per-cpu stacks (like > ia64). Yup, we have per-cpu stacks. > If not there are issues in saving register state later (and hence > delaying your call to context_saved()) as there are calls to do_softirq() > outside your asm code (well, not many, but there is one in domain.c for > example) where you won't end up executing your do_softirq() wrapper. In > general we'd like to reserve the right to include voluntary yield points, > and that won't mix well with lazy register saves and per-physical-cpu > stacks. Oh, we have per-physical-cpu stacks. We can do that because there's no such thing as a "hypervisor thread" which could block in hypervisor space and need to be restored later. Are you saying in the future you want to have hypervisor threads, and so we'll need per-VIRTUAL-cpu stacks? -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Re: [Xen-devel] schedule() vs softirqs
On 15/12/06 20:41, "Hollis Blanchard" <[EMAIL PROTECTED]> wrote: > It's an issue with any architecture with a large number of registers > which aren't automatically saved by hardware (and a C ABI that makes > some of them non-volatile). > > x86 has a small number of registers. ia64 automatically saves them (from > what I understand). So of the currently-supported architectures, yes, > that leaves PowerPC. I see. It sounds like returning from context_switch() is perhaps the right thing for powerpc. That would be easier if you have per-cpu stacks (like ia64). If not there are issues in saving register state later (and hence delaying your call to context_saved()) as there are calls to do_softirq() outside your asm code (well, not many, but there is one in domain.c for example) where you won't end up executing your do_softirq() wrapper. In general we'd like to reserve the right to include voluntary yield points, and that won't mix well with lazy register saves and per-physical-cpu stacks. -- Keir ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Re: [Xen-devel] schedule() vs softirqs
On Fri, 2006-12-15 at 20:00 +, Keir Fraser wrote: > > > Also, that doesn't solve the lazy register saving problem. > > I assume this is a PPC-specific issue? It's an issue with any architecture with a large number of registers which aren't automatically saved by hardware (and a C ABI that makes some of them non-volatile). x86 has a small number of registers. ia64 automatically saves them (from what I understand). So of the currently-supported architectures, yes, that leaves PowerPC. -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Re: [Xen-devel] schedule() vs softirqs
On 15/12/06 19:09, "Hollis Blanchard" <[EMAIL PROTECTED]> wrote: > Ah OK, I see now how x86 is doing that. I don't think that code flow > really makes sense: why would you jump out of do_softirq() into assembly > just to call do_softirq() again? Well, that's the way it works out on x86. It is a bit odd, but it works and is unlikely to affect performance. I think returning from schedule() would have its own problems (e.g., context switch from idle domain to guest would return to the idle loop, which we'd need explicit code to bail from, etc). > Also, that doesn't solve the lazy register saving problem. I assume this is a PPC-specific issue? > However, I think I see how we can implement our desired context_switch() > scheme in arch-specific code. The context_switch() call in schedule() > will return, so please don't add a BUG() after that. :) We already support this mode of operation for IA64 which always returns from schedule(). -- Keir ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Re: [Xen-devel] schedule() vs softirqs
On Fri, 2006-12-15 at 17:36 +, Keir Fraser wrote: > On 15/12/06 17:27, "Hollis Blanchard" <[EMAIL PROTECTED]> wrote: > > > We recently uncovered a bug on PowerPC where if a timer tick arrives > > just inside schedule() while interrupts are still enabled, the > > decrementer is never reprogrammed to that appropriate value. This is > > because once inside schedule(), we never handle any subsequent softirqs: > > we call context_switch() and resume the guest. > > Easily fixed. You need to handle softirqs in the exit path to guest context. > You need to do this final check with interrupts disabled to avoid races. Ah OK, I see now how x86 is doing that. I don't think that code flow really makes sense: why would you jump out of do_softirq() into assembly just to call do_softirq() again? Also, that doesn't solve the lazy register saving problem. However, I think I see how we can implement our desired context_switch() scheme in arch-specific code. The context_switch() call in schedule() will return, so please don't add a BUG() after that. :) -- Hollis Blanchard IBM Linux Technology Center ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
[XenPPC] Re: [Xen-devel] schedule() vs softirqs
On 15/12/06 17:27, "Hollis Blanchard" <[EMAIL PROTECTED]> wrote: > We recently uncovered a bug on PowerPC where if a timer tick arrives > just inside schedule() while interrupts are still enabled, the > decrementer is never reprogrammed to that appropriate value. This is > because once inside schedule(), we never handle any subsequent softirqs: > we call context_switch() and resume the guest. Easily fixed. You need to handle softirqs in the exit path to guest context. You need to do this final check with interrupts disabled to avoid races. -- Keir ___ Xen-ppc-devel mailing list Xen-ppc-devel@lists.xensource.com http://lists.xensource.com/xen-ppc-devel
