John Groves wrote: > On 26/03/26 05:46PM, Ira Weiny wrote: > > John Groves wrote: > > > On 26/03/25 11:04AM, Ira Weiny wrote: > > > > John Groves wrote: > > > > > On 26/03/24 02:39PM, Jonathan Cameron wrote: > > > > > > On Tue, 24 Mar 2026 00:38:31 +0000 > > > > > > John Groves <[email protected]> wrote: > > > > > > > > > > > > > From: John Groves <[email protected]> > > > > > > > > > > > > > > The new fsdev driver provides pages/folios initialized compatibly > > > > > > > with > > > > > > > fsdax - normal rather than devdax-style refcounting, and starting > > > > > > > out > > > > > > > with order-0 folios. > > > > > > > > > > > > > > When fsdev binds to a daxdev, it is usually (always?) switching > > > > > > > from the > > > > > > > devdax mode (device.c), which pre-initializes compound folios > > > > > > > according > > > > > > > to its alignment. Fsdev uses fsdev_clear_folio_state() to switch > > > > > > > the > > > > > > > folios into a fsdax-compatible state. > > > > > > > > > > > > > > A side effect of this is that raw mmap doesn't (can't?) work on > > > > > > > an fsdev > > > > > > > dax instance. Accordingly, The fsdev driver does not provide raw > > > > > > > mmap - > > > > > > > devices must be put in 'devdax' mode (drivers/dax/device.c) to > > > > > > > get raw > > > > > > > mmap capability. > > > > > > > > > > > > > > In this commit is just the framework, which remaps pages/folios > > > > > > > compatibly > > > > > > > with fsdax. > > > > > > > > > > > > > > Enabling dax changes: > > > > > > > > > > > > > > - bus.h: add DAXDRV_FSDEV_TYPE driver type > > > > > > > - bus.c: allow DAXDRV_FSDEV_TYPE drivers to bind to daxdevs > > > > > > > - dax.h: prototype inode_dax(), which fsdev needs > > > > > > > > > > > > > > Suggested-by: Dan Williams <[email protected]> > > > > > > > Suggested-by: Gregory Price <[email protected]> > > > > > > > Signed-off-by: John Groves <[email protected]> > > > > > > > > > > > > I was kind of thinking you'd go with a hidden KCONFIG option with > > > > > > default > > > > > > magic to do the same build condition to you had in the Makefil, but > > > > > > one the > > > > > > user can opt in or out for is also fine. > > > > > > > > > > > > Comments on that below. Meh, I think this is better anyway :) > > > > > > > > > > > > Reviewed-by: Jonathan Cameron <[email protected]> > > > > > > > > > > > > > > > > > > > > > > > > > diff --git a/drivers/dax/Kconfig b/drivers/dax/Kconfig > > > > > > > index d656e4c0eb84..7051b70980d5 100644 > > > > > > > --- a/drivers/dax/Kconfig > > > > > > > +++ b/drivers/dax/Kconfig > > > > > > > @@ -61,6 +61,17 @@ config DEV_DAX_HMEM_DEVICES > > > > > > > depends on DEV_DAX_HMEM && DAX > > > > > > > def_bool y > > > > > > > > > > > > > > +config DEV_DAX_FSDEV > > > > > > > + tristate "FSDEV DAX: fs-dax compatible devdax driver" > > > > > > > + depends on DEV_DAX && FS_DAX > > > > > > > + help > > > > > > > + Support fs-dax access to DAX devices via a character device > > > > > > > + interface. Unlike device_dax (which pre-initializes compound > > > > > > > folios > > > > > > > + based on device alignment), this driver leaves folios at > > > > > > > order-0 so > > > > > > > + that fs-dax filesystems can manage folio order dynamically. > > > > > > > + > > > > > > > + Say M if unsure. > > > > > > Fine like this, but if you wanted to hide it in interests of not > > > > > > confusing users... > > > > > > > > > > > > config DEV_DAX_FSDEV > > > > > > tristate > > > > > > depends on DEV_DAX && FS_DAX > > > > > > default DEV_DAX > > > > > > > > > > I like this better. I see no reason not to default to including fsdev. > > > > > It does nothing other than frustrating famfs users if it's off - since > > > > > building it still has no effect unless you put a daxdev in famfs mode. > > > > > > > > > > Ira, it's kinda in your hands at the moment. Do you feel like making > > > > > this > > > > > change? > > > > > > > > I don't mind making this change. But we have to deal with the breakage > > > > to > > > > current device dax users. > > > > > > > > https://lore.kernel.org/all/[email protected]/ > > > > > > > > What am I missing? > > > > > > > > Ira > > > > > > OK, I can reproduce that failure with kernel 7.0.0-rc5 and > > > straight ndctl v84. So it's not famfs. > > > > No it is the fsdev_dax driver which causes the issue. > > > > I can reload the driver and effectively change the order the drivers are > > searched. > > > > I can prove this with a simple print. With my test system (where > > fsdev_dax _happens_ to be the first driver searched) the failure happens. > > > > [ 526.564232] IKW searching drv type 0 ; type 1 > > [ 526.564515] IKW searching drv type 2 ; type 1 > > > > If I remove your driver (modprobe -r fsdev_dax) prior to running the test > > I get. > > > > [ 59.748171] IKW searching drv type 0 ; type 1 > > [ 59.749127] IKW searching drv type 1 ; type 1 > > > > And it passes. I can continue by loading fsdev_dax back and it will > > continue to work. If you are getting this to pass it must be because in > > your system that driver gets loaded first... not sure how. > > > > This is with the same exact kernel just with your module removed at run > > time. > > > > dax_match_type() needs some other way of matching when the fsdev_dax > > driver should be used. > > I think the correct answer is that fsdev/famfs should never automatically > match and bind. Weird that I haven't seen it do that (or maybe it did but > I didn't notice?)
Agreed. > > If one does a mkfs.famfs or 'famfs mount', the famfs tools already try to > bind fsdev/famfs mode if necessary and fail if they can't. Yep. > > > > > I'm not seeing a clear path ATM. > > I do, but I need to test it out. If it works I'll send a v10 patch set > in a day or two. > > Also, I am definitely seeing ndctl/dax test failures from the device-dax > and dm.sh tests at rc5 with no famfs code (dax or otherwise) at all; I'm > puzzled that you don't see any ndctl test failures in that situation. If > I understood Allison correctly, she saw something similar to what I saw). > But no worries, we'll get it sorted. :-/ Ok I can get dm.sh to fail with rc5. But it is intermittent. I'll investigate that. FWIW I'm not saying device-dax does not ever fail on rc5. I've not seen it though. But I can definitely get it to fail with the procedure above. If there is another failure it would be good to send your log with a report and I'll look at that separately. > > If my strategy works, the next version won't ever automatically bind fsdev, > but it will be explicitly bindable via daxctl or famfs tools. Famfs does not > need fsdev to ever be automatically bound do dax mem... > > > > > > > > > I also studied the verbose logs trying to figure out if famfs > > > could cause it (while running a famfs kernel and ndctl), but > > > I don't see it. > > > > > > Then I tried non-famfs kernel and ndctl and it's the same with > > > or without famfs kernel and famfs ndctl. > > > > :-/ I'm not seeing any failures with rc5. > > > > Also I'm not running with famfs. Just the dax changes. > > Right - if fsdev ever gets automatically bound instead of > drivers/dax/device.c, that's my bad. Weird that I haven't seen that happen, > but that's why we review and test :D Sounds good! Thanks, Ira
