On Wed, Aug 26, 2015 at 06:12:10PM +0200, Mark Kettenis wrote: > > Date: Wed, 26 Aug 2015 17:30:14 +0200 > > From: Alexandr Nedvedicky <alexandr.nedvedi...@oracle.com> > > > > Hello, > > > > I'm not sure I got everything right in Calgary. So this patch should > > roughly illustrates how I think we should start moving forward to > > make PF MULTIPROCESSOR friendly. It's quite likely my proposal/way > > is completely off, I'll be happy if you put me back to ground. > > > > The brief summary of what patch is trying to achieve is as follows: > > > > patch trades all splsoftnet() for KERNEL_LOCK() when it gets compiled > > with MULTIPROCESSOR option on. > > > > if MULTIPROCESSOR option is off, the compiler produces PF, which uses > > splsoftnet. > > > > To achieve this the patch introduces macros PF_LOCK()/PF_UNLOCK(), > > which expand to KERNEL_LOCK()/KERNEL_UNLOCK(), when MULTIPROCESSOR is > > on. > > On the other hand if MULTIPROCESSOR is off the PF_*LOCK() macros become > > splsoftnet()/splx() > > I don't think this will work. Even on MULTIPROCESSOR kernels you'll > need to raise the spl to prevent soft interrupts from running on the > same CPU. KERNEL_LOCK() will not prevent this from happening as it is > a recursive lock. This is why OpenBSD's mutexes (spinning locks) > raise the spl. > > So I think you'll have to define PF_LOCK()/PF_UNLOCK() to do the spl > stuff even for MULTIPROCESSOR kernels.
Thanks for putting my feet to ground. I'm trying to make some sense from that. So if I understand splsotfnet() right in MULTIPROCESSOR kernel, it prevents CPU from handling another network interrupt until the first one is handled. So that splsoftnet should be raised by network driver, before packet reaches PF (pf_test() function). the pf_test() must protect consistency of firewall data, which are shared between CPUs (?can we say kernel threads?). So for the first phase, we opt for KERNEL_LOCK() being grabbed by pf_test() as soon as it gets called. It makes PF to still process single packet at time. All other CPUs will have to wait on KERNEL_LOCK(), holding their splsoftnets(). Do I still follow the concept, or am I wrong again? Assuming the locking in MULTIPROCESSOR goes like: interrupt grabs splsoftnet -> ip_input -> PF grabs KERNEL_LOCK() We need to take care of ioctl() call path and purge thread. Those need to get synchronize with packets using KERNEL_LOCK(). They should not to mess with splsoftnet() any more in MULTIPROCESSOR version. Does that idea sound right? If not what piece I'm still missing in puzzle? thanks and regards sasha