On Sun, Mar 22, 2020 at 12:40:00PM -0700, Jason Thorpe wrote: > > On Mar 22, 2020, at 12:34 PM, Andrew Doran <a...@netbsd.org> wrote: > > > > This works well for me on amd64, but I haven't tried it on other machines. > > From code inspection I can see that some architectures need more work so > > this is disabled on them until I can make the needed changes: hppa, mips, > > powerpc and riscv. It's enabled everywhere else. > > Can you provide a summary of what's needed to fix these other pmap > modules? Or, at least, what the issue is that you're avoiding with the > switch?
Sure, I'll try to give an outline. Before this the pmap module could work with the assumption that for a given page "pg", at all times there was an exclusive lock held by the VM system on the owning object, and so there would never be any concurrent pmap operations on "pg". The pmap module could also assume that within a single pmap, there would never be concurrent operations on the same set of VAs. Both of those things have changed now. For example you can have one process doing pmap_enter() on a page while a different process does pmap_remove() for the same page. You can also have N threads in a multi-threaded process racing to do pmap_enter() on the exactly the same page and same VA because they faulted on it at the same time and both think it's unentered (those threads will never try to do something conflicting though, i.e. the new mapping will always have the same attributes). In the pmap module there are situations that can be excluded by making assumptions about what locks are held in the VM system, but I don't think the middle ground between "always strongly locked" and "never locked" is a happy place to be - so for this concurrent handling of faults I think the pmap should be able to handle anything thrown at it. The only assumption I think is reasonable to make is that a page won't change identity while being worked on in the pmap module, and won't change identity between a call to e.g. pmap_remove() -> pmap_update(). With that in mind for each pmap it's a matter of code inspection while thinking about the "what if" scenarios. For example the races above, or like what if pmap_is_modified(pg) is called right in the middle of another thread doing pmap_remove() for the same - will the right answer be produced. I think the approach to solving it depends on the machine and how much effort we can put into testing / maintaining it. For vax I've gone with a global pmap_lock because it's a good compromise there. Not taxing on the machine or the programmer at all. Obviously that wouldn't make the grade for something modern with many CPUs. The other two basic approaches we have to solving it are firstly per-pmap + per-page locking like x86 or aarch64, and secondly the alpha (old x86) approach of a global RW lock + per pmap locks (benchmarked extensively back in 2007 - it works for sure, but is worse than a global mutex from a performance POV). Andrew