Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
"Paul E. McKenney"writes: > On Fri, Apr 21, 2017 at 09:27:59AM +0200, Paolo Bonzini wrote: >> On 21/04/2017 06:17, Paul E. McKenney wrote: >> >> Thanks, this looks perfect to me, and if you're happy to put it on top >> >> of your tree that would limit the breakage to a smaller history window, >> >> so that would be ideal. >> >> >> >> Tested-by: Michael Ellerman >> > Thank you, Michael! >> > >> > Paolo, does this look good to you? My upstream maintainer will be much >> > more likely to take this with your ack. ;-) >> >> Sure, >> >> Acked-by: Paolo Bonzini >> >> since I don't expect conflicts between whoever applies this patch and >> the KVM tree. > > Applied, thank you! Thanks all. cheers
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
"Paul E. McKenney" writes: > On Fri, Apr 21, 2017 at 09:27:59AM +0200, Paolo Bonzini wrote: >> On 21/04/2017 06:17, Paul E. McKenney wrote: >> >> Thanks, this looks perfect to me, and if you're happy to put it on top >> >> of your tree that would limit the breakage to a smaller history window, >> >> so that would be ideal. >> >> >> >> Tested-by: Michael Ellerman >> > Thank you, Michael! >> > >> > Paolo, does this look good to you? My upstream maintainer will be much >> > more likely to take this with your ack. ;-) >> >> Sure, >> >> Acked-by: Paolo Bonzini >> >> since I don't expect conflicts between whoever applies this patch and >> the KVM tree. > > Applied, thank you! Thanks all. cheers
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Fri, Apr 21, 2017 at 09:27:59AM +0200, Paolo Bonzini wrote: > > > On 21/04/2017 06:17, Paul E. McKenney wrote: > >> Thanks, this looks perfect to me, and if you're happy to put it on top > >> of your tree that would limit the breakage to a smaller history window, > >> so that would be ideal. > >> > >> Tested-by: Michael Ellerman> > Thank you, Michael! > > > > Paolo, does this look good to you? My upstream maintainer will be much > > more likely to take this with your ack. ;-) > > Sure, > > Acked-by: Paolo Bonzini > > since I don't expect conflicts between whoever applies this patch and > the KVM tree. Applied, thank you! Thanx, Paul
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Fri, Apr 21, 2017 at 09:27:59AM +0200, Paolo Bonzini wrote: > > > On 21/04/2017 06:17, Paul E. McKenney wrote: > >> Thanks, this looks perfect to me, and if you're happy to put it on top > >> of your tree that would limit the breakage to a smaller history window, > >> so that would be ideal. > >> > >> Tested-by: Michael Ellerman > > Thank you, Michael! > > > > Paolo, does this look good to you? My upstream maintainer will be much > > more likely to take this with your ack. ;-) > > Sure, > > Acked-by: Paolo Bonzini > > since I don't expect conflicts between whoever applies this patch and > the KVM tree. Applied, thank you! Thanx, Paul
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On 21/04/2017 06:17, Paul E. McKenney wrote: >> Thanks, this looks perfect to me, and if you're happy to put it on top >> of your tree that would limit the breakage to a smaller history window, >> so that would be ideal. >> >> Tested-by: Michael Ellerman> Thank you, Michael! > > Paolo, does this look good to you? My upstream maintainer will be much > more likely to take this with your ack. ;-) Sure, Acked-by: Paolo Bonzini since I don't expect conflicts between whoever applies this patch and the KVM tree. Paolo
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On 21/04/2017 06:17, Paul E. McKenney wrote: >> Thanks, this looks perfect to me, and if you're happy to put it on top >> of your tree that would limit the breakage to a smaller history window, >> so that would be ideal. >> >> Tested-by: Michael Ellerman > Thank you, Michael! > > Paolo, does this look good to you? My upstream maintainer will be much > more likely to take this with your ack. ;-) Sure, Acked-by: Paolo Bonzini since I don't expect conflicts between whoever applies this patch and the KVM tree. Paolo
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Fri, Apr 21, 2017 at 11:42:01AM +1000, Michael Ellerman wrote: > "Paul E. McKenney"writes: > > On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote: > >> On 20/04/2017 05:40, Michael Ellerman wrote: > >> > "Paul E. McKenney" writes: > >> > > >> > This change seems to have had the non-obvious effect of breaking the > >> > powerpc KVM build. > ... > >> > >> Michael, if you want to move the two srcu structs at the end of struct > >> kvm, that would be fine by me. Please send a patch yourself so you can > >> test it on PPC. Thanks, > > > > On the off-chance that it is at all helpful, I have added the commit > > shown below to -rcu. > > As shown below :) > > > If it helps, I am happy to push this, but am just as happy to drop it > > in favor of some other fix. If at all possible, I would like to get > > this into the upcoming merge window. > > Thanks, this looks perfect to me, and if you're happy to put it on top > of your tree that would limit the breakage to a smaller history window, > so that would be ideal. > > Tested-by: Michael Ellerman Thank you, Michael! Paolo, does this look good to you? My upstream maintainer will be much more likely to take this with your ack. ;-) Thanx, Paul > cheers > > > >From a19a6617e2817e485ccc2f7cc5a97bd7ff769b87 Mon Sep 17 00:00:00 2001 > From: "Paul E. McKenney" > Date: Thu, 20 Apr 2017 17:30:06 -0700 > Subject: kvm: Move srcu_struct fields to end of struct kvm > > Parallelizing SRCU callback handling increased the size of srcu_struct, > which moved the kvm_arch field within the kvm struct out of reach of > powerpc's current assembly code, resulting in the following sort of > build error: > > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > (0xb328 is not between 0x8000 and 0x7fff) > > This commit moves the srcu_struct fields in the kvm structure to follow > the kvm_arch field, which again allows powerpc's assembly code to > reach it. > > Reported-by: Stephen Rothwell > Reported-by: Michael Ellerman > Reported-by: kbuild test robot > Suggested-by: Paolo Bonzini > Signed-off-by: Paul E. McKenney > --- > include/linux/kvm_host.h | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > index 2c14ad9..96c8e29 100644 > --- a/include/linux/kvm_host.h > +++ b/include/linux/kvm_host.h > @@ -375,8 +375,6 @@ struct kvm { > struct mutex slots_lock; > struct mm_struct *mm; /* userspace tied to this vm */ > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > - struct srcu_struct srcu; > - struct srcu_struct irq_srcu; > struct kvm_vcpu *vcpus[KVM_MAX_VCPUS]; > > /* > @@ -429,6 +427,8 @@ struct kvm { > struct list_head devices; > struct dentry *debugfs_dentry; > struct kvm_stat_data **debugfs_stat_data; > + struct srcu_struct srcu; > + struct srcu_struct irq_srcu; > }; > > #define kvm_err(fmt, ...) \ > -- > cgit v1.1
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Fri, Apr 21, 2017 at 11:42:01AM +1000, Michael Ellerman wrote: > "Paul E. McKenney" writes: > > On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote: > >> On 20/04/2017 05:40, Michael Ellerman wrote: > >> > "Paul E. McKenney" writes: > >> > > >> > This change seems to have had the non-obvious effect of breaking the > >> > powerpc KVM build. > ... > >> > >> Michael, if you want to move the two srcu structs at the end of struct > >> kvm, that would be fine by me. Please send a patch yourself so you can > >> test it on PPC. Thanks, > > > > On the off-chance that it is at all helpful, I have added the commit > > shown below to -rcu. > > As shown below :) > > > If it helps, I am happy to push this, but am just as happy to drop it > > in favor of some other fix. If at all possible, I would like to get > > this into the upcoming merge window. > > Thanks, this looks perfect to me, and if you're happy to put it on top > of your tree that would limit the breakage to a smaller history window, > so that would be ideal. > > Tested-by: Michael Ellerman Thank you, Michael! Paolo, does this look good to you? My upstream maintainer will be much more likely to take this with your ack. ;-) Thanx, Paul > cheers > > > >From a19a6617e2817e485ccc2f7cc5a97bd7ff769b87 Mon Sep 17 00:00:00 2001 > From: "Paul E. McKenney" > Date: Thu, 20 Apr 2017 17:30:06 -0700 > Subject: kvm: Move srcu_struct fields to end of struct kvm > > Parallelizing SRCU callback handling increased the size of srcu_struct, > which moved the kvm_arch field within the kvm struct out of reach of > powerpc's current assembly code, resulting in the following sort of > build error: > > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > (0xb328 is not between 0x8000 and 0x7fff) > > This commit moves the srcu_struct fields in the kvm structure to follow > the kvm_arch field, which again allows powerpc's assembly code to > reach it. > > Reported-by: Stephen Rothwell > Reported-by: Michael Ellerman > Reported-by: kbuild test robot > Suggested-by: Paolo Bonzini > Signed-off-by: Paul E. McKenney > --- > include/linux/kvm_host.h | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) > > diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h > index 2c14ad9..96c8e29 100644 > --- a/include/linux/kvm_host.h > +++ b/include/linux/kvm_host.h > @@ -375,8 +375,6 @@ struct kvm { > struct mutex slots_lock; > struct mm_struct *mm; /* userspace tied to this vm */ > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > - struct srcu_struct srcu; > - struct srcu_struct irq_srcu; > struct kvm_vcpu *vcpus[KVM_MAX_VCPUS]; > > /* > @@ -429,6 +427,8 @@ struct kvm { > struct list_head devices; > struct dentry *debugfs_dentry; > struct kvm_stat_data **debugfs_stat_data; > + struct srcu_struct srcu; > + struct srcu_struct irq_srcu; > }; > > #define kvm_err(fmt, ...) \ > -- > cgit v1.1
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
Paolo Bonziniwrites: > Michael, if you want to move the two srcu structs at the end of struct > kvm, that would be fine by me. Please send a patch yourself so you can > test it on PPC. Thanks, Paul beat me to it :) I've confirmed it fixes the build break and the resulting kernel can boot a KVM guest happily. cheers
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
Paolo Bonzini writes: > Michael, if you want to move the two srcu structs at the end of struct > kvm, that would be fine by me. Please send a patch yourself so you can > test it on PPC. Thanks, Paul beat me to it :) I've confirmed it fixes the build break and the resulting kernel can boot a KVM guest happily. cheers
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
"Paul E. McKenney"writes: > On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote: >> On 20/04/2017 05:40, Michael Ellerman wrote: >> > "Paul E. McKenney" writes: >> > >> > This change seems to have had the non-obvious effect of breaking the >> > powerpc KVM build. ... >> >> Michael, if you want to move the two srcu structs at the end of struct >> kvm, that would be fine by me. Please send a patch yourself so you can >> test it on PPC. Thanks, > > On the off-chance that it is at all helpful, I have added the commit > shown below to -rcu. As shown below :) > If it helps, I am happy to push this, but am just as happy to drop it > in favor of some other fix. If at all possible, I would like to get > this into the upcoming merge window. Thanks, this looks perfect to me, and if you're happy to put it on top of your tree that would limit the breakage to a smaller history window, so that would be ideal. Tested-by: Michael Ellerman cheers >From a19a6617e2817e485ccc2f7cc5a97bd7ff769b87 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Thu, 20 Apr 2017 17:30:06 -0700 Subject: kvm: Move srcu_struct fields to end of struct kvm Parallelizing SRCU callback handling increased the size of srcu_struct, which moved the kvm_arch field within the kvm struct out of reach of powerpc's current assembly code, resulting in the following sort of build error: arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0xb328 is not between 0x8000 and 0x7fff) This commit moves the srcu_struct fields in the kvm structure to follow the kvm_arch field, which again allows powerpc's assembly code to reach it. Reported-by: Stephen Rothwell Reported-by: Michael Ellerman Reported-by: kbuild test robot Suggested-by: Paolo Bonzini Signed-off-by: Paul E. McKenney --- include/linux/kvm_host.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2c14ad9..96c8e29 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -375,8 +375,6 @@ struct kvm { struct mutex slots_lock; struct mm_struct *mm; /* userspace tied to this vm */ struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; - struct srcu_struct srcu; - struct srcu_struct irq_srcu; struct kvm_vcpu *vcpus[KVM_MAX_VCPUS]; /* @@ -429,6 +427,8 @@ struct kvm { struct list_head devices; struct dentry *debugfs_dentry; struct kvm_stat_data **debugfs_stat_data; + struct srcu_struct srcu; + struct srcu_struct irq_srcu; }; #define kvm_err(fmt, ...) \ -- cgit v1.1
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
"Paul E. McKenney" writes: > On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote: >> On 20/04/2017 05:40, Michael Ellerman wrote: >> > "Paul E. McKenney" writes: >> > >> > This change seems to have had the non-obvious effect of breaking the >> > powerpc KVM build. ... >> >> Michael, if you want to move the two srcu structs at the end of struct >> kvm, that would be fine by me. Please send a patch yourself so you can >> test it on PPC. Thanks, > > On the off-chance that it is at all helpful, I have added the commit > shown below to -rcu. As shown below :) > If it helps, I am happy to push this, but am just as happy to drop it > in favor of some other fix. If at all possible, I would like to get > this into the upcoming merge window. Thanks, this looks perfect to me, and if you're happy to put it on top of your tree that would limit the breakage to a smaller history window, so that would be ideal. Tested-by: Michael Ellerman cheers >From a19a6617e2817e485ccc2f7cc5a97bd7ff769b87 Mon Sep 17 00:00:00 2001 From: "Paul E. McKenney" Date: Thu, 20 Apr 2017 17:30:06 -0700 Subject: kvm: Move srcu_struct fields to end of struct kvm Parallelizing SRCU callback handling increased the size of srcu_struct, which moved the kvm_arch field within the kvm struct out of reach of powerpc's current assembly code, resulting in the following sort of build error: arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0xb328 is not between 0x8000 and 0x7fff) This commit moves the srcu_struct fields in the kvm structure to follow the kvm_arch field, which again allows powerpc's assembly code to reach it. Reported-by: Stephen Rothwell Reported-by: Michael Ellerman Reported-by: kbuild test robot Suggested-by: Paolo Bonzini Signed-off-by: Paul E. McKenney --- include/linux/kvm_host.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 2c14ad9..96c8e29 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -375,8 +375,6 @@ struct kvm { struct mutex slots_lock; struct mm_struct *mm; /* userspace tied to this vm */ struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; - struct srcu_struct srcu; - struct srcu_struct irq_srcu; struct kvm_vcpu *vcpus[KVM_MAX_VCPUS]; /* @@ -429,6 +427,8 @@ struct kvm { struct list_head devices; struct dentry *debugfs_dentry; struct kvm_stat_data **debugfs_stat_data; + struct srcu_struct srcu; + struct srcu_struct irq_srcu; }; #define kvm_err(fmt, ...) \ -- cgit v1.1
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote: > > > On 20/04/2017 05:40, Michael Ellerman wrote: > > "Paul E. McKenney"writes: > > > >> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > >> index f2b3bd6c6bc2..0400e211aa44 100644 > >> --- a/include/linux/srcutree.h > >> +++ b/include/linux/srcutree.h > >> @@ -24,25 +24,75 @@ > > ... > >> +/* > >> + * Per-SRCU-domain structure, similar in function to rcu_state. > >> + */ > >> struct srcu_struct { > >> - unsigned long completed; > >> - unsigned long srcu_gp_seq; > >> - atomic_t srcu_exp_cnt; > >> - struct srcu_array __percpu *per_cpu_ref; > >> - spinlock_t queue_lock; /* protect ->srcu_cblist */ > >> - struct rcu_segcblist srcu_cblist; > >> + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > >> + struct srcu_node *level[RCU_NUM_LVLS + 1]; > >> + /* First node at each level. */ > >> + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > >> + spinlock_t gp_lock; /* protect ->srcu_cblist */ > >> + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > >> + unsigned int srcu_idx; /* Current rdr array element. */ > >> + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > >> + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > >> + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > >> + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > >> + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > >> + struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ > >> + struct completion srcu_barrier_completion; > >> + /* Awaken barrier rq at end. */ > >> + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > >> + /* callback for the barrier */ > >> + /* operation. */ > > > > This change seems to have had the non-obvious effect of breaking the > > powerpc KVM build. > > > > Because struct kvm contains two srcu_structs which are located > > before the kvm_arch struct, the increase in size of srcu_struct has > > caused the offset from the start of struct kvm to kvm_arch to be too big > > for some of our asm. > > > > struct kvm { > > spinlock_t mmu_lock; > > struct mutex slots_lock; > > struct mm_struct *mm; /* userspace tied to this vm */ > > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > > struct srcu_struct srcu; > > struct srcu_struct irq_srcu; > > ... > > struct kvm_arch arch; > > > > > > Example error: > > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > > (0xb328 is not between 0x8000 and > > 0x7fff) > > > > Where line 617 is: > > lwz r7,KVM_LPID(r9) > > > > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs > > new is: > > > > -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid) # > > +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid) # > > > > > > We can probably just fix it by changing the asm to keep the address of > > kvm_arch in a register, and then offset from that. > > Michael, if you want to move the two srcu structs at the end of struct > kvm, that would be fine by me. Please send a patch yourself so you can > test it on PPC. Thanks, On the off-chance that it is at all helpful, I have added the commit shown below to -rcu. If it helps, I am happy to push this, but am just as happy to drop it in favor of some other fix. If at all possible, I would like to get this into the upcoming merge window. Thanx, Paul
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Thu, Apr 20, 2017 at 05:28:32PM +0200, Paolo Bonzini wrote: > > > On 20/04/2017 05:40, Michael Ellerman wrote: > > "Paul E. McKenney" writes: > > > >> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > >> index f2b3bd6c6bc2..0400e211aa44 100644 > >> --- a/include/linux/srcutree.h > >> +++ b/include/linux/srcutree.h > >> @@ -24,25 +24,75 @@ > > ... > >> +/* > >> + * Per-SRCU-domain structure, similar in function to rcu_state. > >> + */ > >> struct srcu_struct { > >> - unsigned long completed; > >> - unsigned long srcu_gp_seq; > >> - atomic_t srcu_exp_cnt; > >> - struct srcu_array __percpu *per_cpu_ref; > >> - spinlock_t queue_lock; /* protect ->srcu_cblist */ > >> - struct rcu_segcblist srcu_cblist; > >> + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > >> + struct srcu_node *level[RCU_NUM_LVLS + 1]; > >> + /* First node at each level. */ > >> + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > >> + spinlock_t gp_lock; /* protect ->srcu_cblist */ > >> + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > >> + unsigned int srcu_idx; /* Current rdr array element. */ > >> + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > >> + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > >> + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > >> + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > >> + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > >> + struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ > >> + struct completion srcu_barrier_completion; > >> + /* Awaken barrier rq at end. */ > >> + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > >> + /* callback for the barrier */ > >> + /* operation. */ > > > > This change seems to have had the non-obvious effect of breaking the > > powerpc KVM build. > > > > Because struct kvm contains two srcu_structs which are located > > before the kvm_arch struct, the increase in size of srcu_struct has > > caused the offset from the start of struct kvm to kvm_arch to be too big > > for some of our asm. > > > > struct kvm { > > spinlock_t mmu_lock; > > struct mutex slots_lock; > > struct mm_struct *mm; /* userspace tied to this vm */ > > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > > struct srcu_struct srcu; > > struct srcu_struct irq_srcu; > > ... > > struct kvm_arch arch; > > > > > > Example error: > > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > > (0xb328 is not between 0x8000 and > > 0x7fff) > > > > Where line 617 is: > > lwz r7,KVM_LPID(r9) > > > > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs > > new is: > > > > -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid) # > > +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid) # > > > > > > We can probably just fix it by changing the asm to keep the address of > > kvm_arch in a register, and then offset from that. > > Michael, if you want to move the two srcu structs at the end of struct > kvm, that would be fine by me. Please send a patch yourself so you can > test it on PPC. Thanks, On the off-chance that it is at all helpful, I have added the commit shown below to -rcu. If it helps, I am happy to push this, but am just as happy to drop it in favor of some other fix. If at all possible, I would like to get this into the upcoming merge window. Thanx, Paul
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On 20/04/2017 05:40, Michael Ellerman wrote: > "Paul E. McKenney"writes: > >> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h >> index f2b3bd6c6bc2..0400e211aa44 100644 >> --- a/include/linux/srcutree.h >> +++ b/include/linux/srcutree.h >> @@ -24,25 +24,75 @@ > ... >> +/* >> + * Per-SRCU-domain structure, similar in function to rcu_state. >> + */ >> struct srcu_struct { >> -unsigned long completed; >> -unsigned long srcu_gp_seq; >> -atomic_t srcu_exp_cnt; >> -struct srcu_array __percpu *per_cpu_ref; >> -spinlock_t queue_lock; /* protect ->srcu_cblist */ >> -struct rcu_segcblist srcu_cblist; >> +struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ >> +struct srcu_node *level[RCU_NUM_LVLS + 1]; >> +/* First node at each level. */ >> +struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ >> +spinlock_t gp_lock; /* protect ->srcu_cblist */ >> +struct mutex srcu_gp_mutex; /* Serialize GP work. */ >> +unsigned int srcu_idx; /* Current rdr array element. */ >> +unsigned long srcu_gp_seq; /* Grace-period seq #. */ >> +unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ >> +atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ >> +struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ >> +unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ >> +struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ >> +struct completion srcu_barrier_completion; >> +/* Awaken barrier rq at end. */ >> +atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ >> +/* callback for the barrier */ >> +/* operation. */ > > This change seems to have had the non-obvious effect of breaking the > powerpc KVM build. > > Because struct kvm contains two srcu_structs which are located > before the kvm_arch struct, the increase in size of srcu_struct has > caused the offset from the start of struct kvm to kvm_arch to be too big > for some of our asm. > > struct kvm { > spinlock_t mmu_lock; > struct mutex slots_lock; > struct mm_struct *mm; /* userspace tied to this vm */ > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > struct srcu_struct srcu; > struct srcu_struct irq_srcu; > ... > struct kvm_arch arch; > > > Example error: > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > (0xb328 is not between 0x8000 and 0x7fff) > > Where line 617 is: > lwz r7,KVM_LPID(r9) > > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs > new is: > > -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)# > +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)# > > > We can probably just fix it by changing the asm to keep the address of > kvm_arch in a register, and then offset from that. Michael, if you want to move the two srcu structs at the end of struct kvm, that would be fine by me. Please send a patch yourself so you can test it on PPC. Thanks, Paolo
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On 20/04/2017 05:40, Michael Ellerman wrote: > "Paul E. McKenney" writes: > >> diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h >> index f2b3bd6c6bc2..0400e211aa44 100644 >> --- a/include/linux/srcutree.h >> +++ b/include/linux/srcutree.h >> @@ -24,25 +24,75 @@ > ... >> +/* >> + * Per-SRCU-domain structure, similar in function to rcu_state. >> + */ >> struct srcu_struct { >> -unsigned long completed; >> -unsigned long srcu_gp_seq; >> -atomic_t srcu_exp_cnt; >> -struct srcu_array __percpu *per_cpu_ref; >> -spinlock_t queue_lock; /* protect ->srcu_cblist */ >> -struct rcu_segcblist srcu_cblist; >> +struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ >> +struct srcu_node *level[RCU_NUM_LVLS + 1]; >> +/* First node at each level. */ >> +struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ >> +spinlock_t gp_lock; /* protect ->srcu_cblist */ >> +struct mutex srcu_gp_mutex; /* Serialize GP work. */ >> +unsigned int srcu_idx; /* Current rdr array element. */ >> +unsigned long srcu_gp_seq; /* Grace-period seq #. */ >> +unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ >> +atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ >> +struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ >> +unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ >> +struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ >> +struct completion srcu_barrier_completion; >> +/* Awaken barrier rq at end. */ >> +atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ >> +/* callback for the barrier */ >> +/* operation. */ > > This change seems to have had the non-obvious effect of breaking the > powerpc KVM build. > > Because struct kvm contains two srcu_structs which are located > before the kvm_arch struct, the increase in size of srcu_struct has > caused the offset from the start of struct kvm to kvm_arch to be too big > for some of our asm. > > struct kvm { > spinlock_t mmu_lock; > struct mutex slots_lock; > struct mm_struct *mm; /* userspace tied to this vm */ > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > struct srcu_struct srcu; > struct srcu_struct irq_srcu; > ... > struct kvm_arch arch; > > > Example error: > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > (0xb328 is not between 0x8000 and 0x7fff) > > Where line 617 is: > lwz r7,KVM_LPID(r9) > > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs > new is: > > -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)# > +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)# > > > We can probably just fix it by changing the asm to keep the address of > kvm_arch in a register, and then offset from that. Michael, if you want to move the two srcu structs at the end of struct kvm, that would be fine by me. Please send a patch yourself so you can test it on PPC. Thanks, Paolo
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Thu, Apr 20, 2017 at 01:40:13PM +1000, Michael Ellerman wrote: > "Paul E. McKenney"writes: > > > diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > > index f2b3bd6c6bc2..0400e211aa44 100644 > > --- a/include/linux/srcutree.h > > +++ b/include/linux/srcutree.h > > @@ -24,25 +24,75 @@ > ... > > +/* > > + * Per-SRCU-domain structure, similar in function to rcu_state. > > + */ > > struct srcu_struct { > > - unsigned long completed; > > - unsigned long srcu_gp_seq; > > - atomic_t srcu_exp_cnt; > > - struct srcu_array __percpu *per_cpu_ref; > > - spinlock_t queue_lock; /* protect ->srcu_cblist */ > > - struct rcu_segcblist srcu_cblist; > > + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > > + struct srcu_node *level[RCU_NUM_LVLS + 1]; > > + /* First node at each level. */ > > + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > > + spinlock_t gp_lock; /* protect ->srcu_cblist */ > > + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > > + unsigned int srcu_idx; /* Current rdr array element. */ > > + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > > + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > > + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > > + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > > + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > > + struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ > > + struct completion srcu_barrier_completion; > > + /* Awaken barrier rq at end. */ > > + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > > + /* callback for the barrier */ > > + /* operation. */ > > This change seems to have had the non-obvious effect of breaking the > powerpc KVM build. > > Because struct kvm contains two srcu_structs which are located > before the kvm_arch struct, the increase in size of srcu_struct has > caused the offset from the start of struct kvm to kvm_arch to be too big > for some of our asm. > > struct kvm { > spinlock_t mmu_lock; > struct mutex slots_lock; > struct mm_struct *mm; /* userspace tied to this vm */ > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > struct srcu_struct srcu; > struct srcu_struct irq_srcu; > ... > struct kvm_arch arch; > > > Example error: > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > (0xb328 is not between 0x8000 and 0x7fff) > > Where line 617 is: > lwz r7,KVM_LPID(r9) > > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs > new is: > > -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)# > +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)# > > > We can probably just fix it by changing the asm to keep the address of > kvm_arch in a register, and then offset from that. Ouch! What do you need from me? Thanx, Paul
Re: powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
On Thu, Apr 20, 2017 at 01:40:13PM +1000, Michael Ellerman wrote: > "Paul E. McKenney" writes: > > > diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > > index f2b3bd6c6bc2..0400e211aa44 100644 > > --- a/include/linux/srcutree.h > > +++ b/include/linux/srcutree.h > > @@ -24,25 +24,75 @@ > ... > > +/* > > + * Per-SRCU-domain structure, similar in function to rcu_state. > > + */ > > struct srcu_struct { > > - unsigned long completed; > > - unsigned long srcu_gp_seq; > > - atomic_t srcu_exp_cnt; > > - struct srcu_array __percpu *per_cpu_ref; > > - spinlock_t queue_lock; /* protect ->srcu_cblist */ > > - struct rcu_segcblist srcu_cblist; > > + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > > + struct srcu_node *level[RCU_NUM_LVLS + 1]; > > + /* First node at each level. */ > > + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > > + spinlock_t gp_lock; /* protect ->srcu_cblist */ > > + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > > + unsigned int srcu_idx; /* Current rdr array element. */ > > + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > > + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > > + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > > + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > > + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > > + struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ > > + struct completion srcu_barrier_completion; > > + /* Awaken barrier rq at end. */ > > + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > > + /* callback for the barrier */ > > + /* operation. */ > > This change seems to have had the non-obvious effect of breaking the > powerpc KVM build. > > Because struct kvm contains two srcu_structs which are located > before the kvm_arch struct, the increase in size of srcu_struct has > caused the offset from the start of struct kvm to kvm_arch to be too big > for some of our asm. > > struct kvm { > spinlock_t mmu_lock; > struct mutex slots_lock; > struct mm_struct *mm; /* userspace tied to this vm */ > struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; > struct srcu_struct srcu; > struct srcu_struct irq_srcu; > ... > struct kvm_arch arch; > > > Example error: > arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range > (0xb328 is not between 0x8000 and 0x7fff) > > Where line 617 is: > lwz r7,KVM_LPID(r9) > > And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs > new is: > > -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid)# > +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid)# > > > We can probably just fix it by changing the asm to keep the address of > kvm_arch in a register, and then offset from that. Ouch! What do you need from me? Thanx, Paul
powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
"Paul E. McKenney"writes: > diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > index f2b3bd6c6bc2..0400e211aa44 100644 > --- a/include/linux/srcutree.h > +++ b/include/linux/srcutree.h > @@ -24,25 +24,75 @@ ... > +/* > + * Per-SRCU-domain structure, similar in function to rcu_state. > + */ > struct srcu_struct { > - unsigned long completed; > - unsigned long srcu_gp_seq; > - atomic_t srcu_exp_cnt; > - struct srcu_array __percpu *per_cpu_ref; > - spinlock_t queue_lock; /* protect ->srcu_cblist */ > - struct rcu_segcblist srcu_cblist; > + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > + struct srcu_node *level[RCU_NUM_LVLS + 1]; > + /* First node at each level. */ > + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > + spinlock_t gp_lock; /* protect ->srcu_cblist */ > + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > + unsigned int srcu_idx; /* Current rdr array element. */ > + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > + struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ > + struct completion srcu_barrier_completion; > + /* Awaken barrier rq at end. */ > + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > + /* callback for the barrier */ > + /* operation. */ This change seems to have had the non-obvious effect of breaking the powerpc KVM build. Because struct kvm contains two srcu_structs which are located before the kvm_arch struct, the increase in size of srcu_struct has caused the offset from the start of struct kvm to kvm_arch to be too big for some of our asm. struct kvm { spinlock_t mmu_lock; struct mutex slots_lock; struct mm_struct *mm; /* userspace tied to this vm */ struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; struct srcu_struct srcu; struct srcu_struct irq_srcu; ... struct kvm_arch arch; Example error: arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0xb328 is not between 0x8000 and 0x7fff) Where line 617 is: lwz r7,KVM_LPID(r9) And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs new is: -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid) # +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid) # We can probably just fix it by changing the asm to keep the address of kvm_arch in a register, and then offset from that. cheers
powerpc KVM build break in linux-next (was Re: [PATCH tip/core/rcu 40/40] srcu: Parallelize callback handling)
"Paul E. McKenney" writes: > diff --git a/include/linux/srcutree.h b/include/linux/srcutree.h > index f2b3bd6c6bc2..0400e211aa44 100644 > --- a/include/linux/srcutree.h > +++ b/include/linux/srcutree.h > @@ -24,25 +24,75 @@ ... > +/* > + * Per-SRCU-domain structure, similar in function to rcu_state. > + */ > struct srcu_struct { > - unsigned long completed; > - unsigned long srcu_gp_seq; > - atomic_t srcu_exp_cnt; > - struct srcu_array __percpu *per_cpu_ref; > - spinlock_t queue_lock; /* protect ->srcu_cblist */ > - struct rcu_segcblist srcu_cblist; > + struct srcu_node node[NUM_RCU_NODES]; /* Combining tree. */ > + struct srcu_node *level[RCU_NUM_LVLS + 1]; > + /* First node at each level. */ > + struct mutex srcu_cb_mutex; /* Serialize CB preparation. */ > + spinlock_t gp_lock; /* protect ->srcu_cblist */ > + struct mutex srcu_gp_mutex; /* Serialize GP work. */ > + unsigned int srcu_idx; /* Current rdr array element. */ > + unsigned long srcu_gp_seq; /* Grace-period seq #. */ > + unsigned long srcu_gp_seq_needed; /* Latest gp_seq needed. */ > + atomic_t srcu_exp_cnt; /* # ongoing expedited GPs. */ > + struct srcu_data __percpu *sda; /* Per-CPU srcu_data array. */ > + unsigned long srcu_barrier_seq; /* srcu_barrier seq #. */ > + struct mutex srcu_barrier_mutex;/* Serialize barrier ops. */ > + struct completion srcu_barrier_completion; > + /* Awaken barrier rq at end. */ > + atomic_t srcu_barrier_cpu_cnt; /* # CPUs not yet posting a */ > + /* callback for the barrier */ > + /* operation. */ This change seems to have had the non-obvious effect of breaking the powerpc KVM build. Because struct kvm contains two srcu_structs which are located before the kvm_arch struct, the increase in size of srcu_struct has caused the offset from the start of struct kvm to kvm_arch to be too big for some of our asm. struct kvm { spinlock_t mmu_lock; struct mutex slots_lock; struct mm_struct *mm; /* userspace tied to this vm */ struct kvm_memslots *memslots[KVM_ADDRESS_SPACE_NUM]; struct srcu_struct srcu; struct srcu_struct irq_srcu; ... struct kvm_arch arch; Example error: arch/powerpc/kvm/book3s_hv_rmhandlers.S:617: Error: operand out of range (0xb328 is not between 0x8000 and 0x7fff) Where line 617 is: lwz r7,KVM_LPID(r9) And the KVM_LPID constant comes from asm-offsets.s. The diff of old vs new is: -->KVM_LPID 17752 offsetof(struct kvm, arch.lpid) # +->KVM_LPID 45864 offsetof(struct kvm, arch.lpid) # We can probably just fix it by changing the asm to keep the address of kvm_arch in a register, and then offset from that. cheers