(Sorry about the previous HTML email, accidentally used the wrong email client)
Hey All, >On 19/04/2024 08:43, Ryan Roberts wrote: >> Hi All, >> >> This series adds uffd write-protect and soft-dirty tracking support for >> arm64. I >> consider the soft-dirty support (patches 3 and 4) as RFC - see rationale >> below. >> >> Previous attempts to add these features have failed because of a perceived >> lack >> of available PTE SW bits. However it actually turns out that there are 2 >> available but they are hidden. PTE_PROT_NONE was previously occupying a SW >> bit, >> but it only applies when PTE_VALID is clear, so this is moved to overlay >> PTE_UXN >> in patch 1, freeing up the SW bit. Bit 63 is marked as "IGNORED" in the Arm >> ARM, >> but it does not currently indicate "reserved for SW use" like it does for the >> other SW bits. I've confirmed with the spec owner that this is an oversight; >> the >> bit is intended to be reserved for SW use and the spec will clarify this in a >> future update. >> >> So we have our two bits; patch 2 enables uffd-wp, patch 3 enables soft-dirty >> and >> patches 4 and 5 sort out the selftests so that the soft-dirty tests are >> compiled >> for, and run on arm64. >> >> That said, these are the last 2 SW bits and we may want to keep 1 bit in >> reserve >> for future use. soft-dirty is only used for CRIU to my knowledge, and it is >> thought that their use case could be solved with the more generic uffd-wp. So >> unless somebody makes a clear case for the inclusion of soft-dirty support, >> we >> are probably better off dropping patches 3 and 4 and keeping bit 63 for >> future >> use. Although note that the most recent attempt to add soft-dirty for arm64 >> was >> last month [1] so I'd like to give Shivansh Vij the opportunity to make the >> case. > > Ugh, forgot to mention that this applies on top of v6.9-rc3, and all the > uffd-wp > and soft-dirty tests in the mm selftests suite run and pass. And no > regressions > are observed in any of the other selftests. Appreciate the opportunity to provide input here. I personally don't know of any applications other than CRIU that make heavy use of soft-dirty, and my use case is specifically focused on adding live-migration support to CRIU on ARM. Cloud providers like AWS have pretty massive discounts for ARM-based spot instances (90% last time I checked), and having live-migration in CRIU would allow more applications to take advantage of that. As Ryan mentioned, there are two ways to achieve this - add dirty tracking to ARM (Patch 3/4), or tear out the existing dirty tracking code in CRIU and replace it with uffd-wp. I picked option one (dirty tracking in arm) because it seems to be the simplest way to move forward, whereas it would be a relatively heavy effort to add uffd-wp support to CRIU. >From a performance perspective I am also a little worried that uffd will be >slower than just tracking the dirty bits asynchronously with sw dirty, but >maybe that's not as much of a concern with the addition of uffd-wp async. With all this being said, I'll defer to the wisdom of the crowd about which approach makes more sense - after all, with this patch we should get uffd-wp support on arm so at least there will be _a_ way forward for CRIU (albeit one requiring slightly more work). Thanks, Shivansh
