Re: [PATCH] fs/dax: Fix "don't skip locked entries when scanning entries"
On Fri, 23 May 2025 14:37:49 +1000, Alistair Popple wrote:
> Commit 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> entries") introduced a new function, wait_entry_unlocked_exclusive(),
> which waits for the current entry to become unlocked without advancing
> the XArray iterator state.
>
> Waiting for the entry to become unlocked requires dropping the XArray
> lock. This requires calling xas_pause() prior to dropping the lock
> which leaves the xas in a suitable state for the next iteration. However
> this has the side-effect of advancing the xas state to the next index.
> Normally this isn't an issue because xas_for_each() contains code to
> detect this state and thus avoid advancing the index a second time on
> the next loop iteration.
>
> [...]
Applied to the vfs.fixes branch of the vfs/vfs.git tree.
Patches in the vfs.fixes branch should appear in linux-next soon.
Please report any outstanding bugs that were missed during review in a
new review to the original patch series allowing us to drop it.
It's encouraged to provide Acked-bys and Reviewed-bys even though the
patch has now been applied. If possible patch trailers will be updated.
Note that commit hashes shown below are subject to change due to rebase,
trailer updates or similar. If in doubt, please check the listed branch.
tree: https://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs.git
branch: vfs.fixes
[1/1] fs/dax: Fix "don't skip locked entries when scanning entries"
https://git.kernel.org/vfs/vfs/c/dd59137bfe70
Re: [PATCH] fs/dax: Fix "don't skip locked entries when scanning entries"
On Tue, May 27, 2025 at 02:46:28PM -0700, Dan Williams wrote:
> Alistair Popple wrote:
> > Commit 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> > entries") introduced a new function, wait_entry_unlocked_exclusive(),
> > which waits for the current entry to become unlocked without advancing
> > the XArray iterator state.
> >
> > Waiting for the entry to become unlocked requires dropping the XArray
> > lock. This requires calling xas_pause() prior to dropping the lock
> > which leaves the xas in a suitable state for the next iteration. However
> > this has the side-effect of advancing the xas state to the next index.
> > Normally this isn't an issue because xas_for_each() contains code to
> > detect this state and thus avoid advancing the index a second time on
> > the next loop iteration.
> >
> > However both callers of and wait_entry_unlocked_exclusive() itself
> > subsequently use the xas state to reload the entry. As xas_pause()
> > updated the state to the next index this will cause the current entry
> > which is being waited on to be skipped. This caused the following
> > warning to fire intermittently when running xftest generic/068 on an XFS
> > filesystem with FS DAX enabled:
> >
> > [ 35.067397] [ cut here ]
> > [ 35.068229] WARNING: CPU: 21 PID: 1640 at mm/truncate.c:89
> > truncate_folio_batch_exceptionals+0xd8/0x1e0
> [..]
> >
> > Fix this by using xas_reset() instead, which is equivalent in
> > implementation to xas_pause() but does not advance the XArray state.
> >
> > Fixes: 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> > entries")
> > Signed-off-by: Alistair Popple
> [..]
> >
> > Hi Andrew,
> >
> > Apologies for finding this so late in the cycle. This is a very
> > intermittent issue for me, and it seems it was only exposed by a recent
> > upgrade to my test machine/setup. The user visible impact is the same
> > as for the original commit this fixes. That is possible file data
> > corruption if a device has a FS DAX page pinned for DMA.
> >
> > So in other words it means my original fix was not 100% effective.
> > The issue that commit fixed has existed for a long time without being
> > reported, so not sure if this is worth trying to get into v6.15 or not.
> >
> > Either way I figured it would be best to send this ASAP, which means I
> > am still waiting for a complete xfstest run to complete (although the
> > failing test does now pass cleanly).
> > ---
> > fs/dax.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/fs/dax.c b/fs/dax.c
> > index 676303419e9e..f8d8b1afd232 100644
> > --- a/fs/dax.c
> > +++ b/fs/dax.c
> > @@ -257,7 +257,7 @@ static void *wait_entry_unlocked_exclusive(struct
> > xa_state *xas, void *entry)
> > wq = dax_entry_waitqueue(xas, entry, &ewait.key);
> > prepare_to_wait_exclusive(wq, &ewait.wait,
> > TASK_UNINTERRUPTIBLE);
> > - xas_pause(xas);
> > + xas_reset(xas);
> > xas_unlock_irq(xas);
> > schedule();
> > finish_wait(wq, &ewait.wait);
>
> This looks super-subtle, but so did the original fix commit 6be3e21d25ca
> ("fs/dax: don't skip locked entries when scanning entries"). The
> resolution is the same to make sure the xarray state does not mistakenly
> advance when the lock is dropped.
Yes, the XArray iterator code is super subtle. The fact it took at least two
engineers three attempts which were also reviewed to get this (hopefully!) right
suggests it's certainly not obvious.
That said I don't have any good suggestions for making it better other than
renaming functions which will create a lot of churn.
> You can add:
>
> Reviewed-by: Dan Williams
Thanks!
Re: [PATCH] fs/dax: Fix "don't skip locked entries when scanning entries"
Alistair Popple wrote:
> Commit 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> entries") introduced a new function, wait_entry_unlocked_exclusive(),
> which waits for the current entry to become unlocked without advancing
> the XArray iterator state.
>
> Waiting for the entry to become unlocked requires dropping the XArray
> lock. This requires calling xas_pause() prior to dropping the lock
> which leaves the xas in a suitable state for the next iteration. However
> this has the side-effect of advancing the xas state to the next index.
> Normally this isn't an issue because xas_for_each() contains code to
> detect this state and thus avoid advancing the index a second time on
> the next loop iteration.
>
> However both callers of and wait_entry_unlocked_exclusive() itself
> subsequently use the xas state to reload the entry. As xas_pause()
> updated the state to the next index this will cause the current entry
> which is being waited on to be skipped. This caused the following
> warning to fire intermittently when running xftest generic/068 on an XFS
> filesystem with FS DAX enabled:
>
> [ 35.067397] [ cut here ]
> [ 35.068229] WARNING: CPU: 21 PID: 1640 at mm/truncate.c:89
> truncate_folio_batch_exceptionals+0xd8/0x1e0
[..]
>
> Fix this by using xas_reset() instead, which is equivalent in
> implementation to xas_pause() but does not advance the XArray state.
>
> Fixes: 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> entries")
> Signed-off-by: Alistair Popple
[..]
>
> Hi Andrew,
>
> Apologies for finding this so late in the cycle. This is a very
> intermittent issue for me, and it seems it was only exposed by a recent
> upgrade to my test machine/setup. The user visible impact is the same
> as for the original commit this fixes. That is possible file data
> corruption if a device has a FS DAX page pinned for DMA.
>
> So in other words it means my original fix was not 100% effective.
> The issue that commit fixed has existed for a long time without being
> reported, so not sure if this is worth trying to get into v6.15 or not.
>
> Either way I figured it would be best to send this ASAP, which means I
> am still waiting for a complete xfstest run to complete (although the
> failing test does now pass cleanly).
> ---
> fs/dax.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/fs/dax.c b/fs/dax.c
> index 676303419e9e..f8d8b1afd232 100644
> --- a/fs/dax.c
> +++ b/fs/dax.c
> @@ -257,7 +257,7 @@ static void *wait_entry_unlocked_exclusive(struct
> xa_state *xas, void *entry)
> wq = dax_entry_waitqueue(xas, entry, &ewait.key);
> prepare_to_wait_exclusive(wq, &ewait.wait,
> TASK_UNINTERRUPTIBLE);
> - xas_pause(xas);
> + xas_reset(xas);
> xas_unlock_irq(xas);
> schedule();
> finish_wait(wq, &ewait.wait);
This looks super-subtle, but so did the original fix commit 6be3e21d25ca
("fs/dax: don't skip locked entries when scanning entries"). The
resolution is the same to make sure the xarray state does not mistakenly
advance when the lock is dropped.
You can add:
Reviewed-by: Dan Williams
Re: [PATCH] fs/dax: Fix "don't skip locked entries when scanning entries"
On Fri 23-05-25 14:37:49, Alistair Popple wrote:
> Commit 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> entries") introduced a new function, wait_entry_unlocked_exclusive(),
> which waits for the current entry to become unlocked without advancing
> the XArray iterator state.
>
> Waiting for the entry to become unlocked requires dropping the XArray
> lock. This requires calling xas_pause() prior to dropping the lock
> which leaves the xas in a suitable state for the next iteration. However
> this has the side-effect of advancing the xas state to the next index.
> Normally this isn't an issue because xas_for_each() contains code to
> detect this state and thus avoid advancing the index a second time on
> the next loop iteration.
>
> However both callers of and wait_entry_unlocked_exclusive() itself
> subsequently use the xas state to reload the entry. As xas_pause()
> updated the state to the next index this will cause the current entry
> which is being waited on to be skipped. This caused the following
> warning to fire intermittently when running xftest generic/068 on an XFS
> filesystem with FS DAX enabled:
>
> [ 35.067397] [ cut here ]
> [ 35.068229] WARNING: CPU: 21 PID: 1640 at mm/truncate.c:89
> truncate_folio_batch_exceptionals+0xd8/0x1e0
> [ 35.069717] Modules linked in: nd_pmem dax_pmem nd_btt nd_e820 libnvdimm
> [ 35.071006] CPU: 21 UID: 0 PID: 1640 Comm: fstest Not tainted 6.15.0-rc7+
> #77 PREEMPT(voluntary)
> [ 35.072613] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
> rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/204
> [ 35.074845] RIP: 0010:truncate_folio_batch_exceptionals+0xd8/0x1e0
> [ 35.075962] Code: a1 00 00 00 f6 47 0d 20 0f 84 97 00 00 00 4c 63 e8 41 39
> c4 7f 0b eb 61 49 83 c5 01 45 39 ec 7e 58 42 f68
> [ 35.079522] RSP: 0018:b04e426c7850 EFLAGS: 00010202
> [ 35.080359] RAX: RBX: 9d21e3481908 RCX:
> b04e426c77f4
> [ 35.081477] RDX: b04e426c79e8 RSI: b04e426c79e0 RDI:
> 9d21e34816e8
> [ 35.082590] RBP: b04e426c79e0 R08: 0001 R09:
> 0003
> [ 35.083733] R10: R11: 822b53c0f7a49868 R12:
> 001f
> [ 35.084850] R13: R14: b04e426c78e8 R15:
> fffe
> [ 35.085953] FS: 7f9134c87740() GS:9d22abba()
> knlGS:
> [ 35.087346] CS: 0010 DS: ES: CR0: 80050033
> [ 35.088244] CR2: 7f9134c86000 CR3: 00040afff000 CR4:
> 06f0
> [ 35.089354] Call Trace:
> [ 35.089749]
> [ 35.090168] truncate_inode_pages_range+0xfc/0x4d0
> [ 35.091078] truncate_pagecache+0x47/0x60
> [ 35.091735] xfs_setattr_size+0xc7/0x3e0
> [ 35.092648] xfs_vn_setattr+0x1ea/0x270
> [ 35.093437] notify_change+0x1f4/0x510
> [ 35.094219] ? do_truncate+0x97/0xe0
> [ 35.094879] do_truncate+0x97/0xe0
> [ 35.095640] path_openat+0xabd/0xca0
> [ 35.096278] do_filp_open+0xd7/0x190
> [ 35.096860] do_sys_openat2+0x8a/0xe0
> [ 35.097459] __x64_sys_openat+0x6d/0xa0
> [ 35.098076] do_syscall_64+0xbb/0x1d0
> [ 35.098647] entry_SYSCALL_64_after_hwframe+0x77/0x7f
> [ 35.099444] RIP: 0033:0x7f9134d81fc1
> [ 35.100033] Code: 75 57 89 f0 25 00 00 41 00 3d 00 00 41 00 74 49 80 3d 2a
> 26 0e 00 00 74 6d 89 da 48 89 ee bf 9c ff ff ff5
> [ 35.102993] RSP: 002b:7ffcd41e0d10 EFLAGS: 0202 ORIG_RAX:
> 0101
> [ 35.104263] RAX: ffda RBX: 0242 RCX:
> 7f9134d81fc1
> [ 35.105452] RDX: 0242 RSI: 7ffcd41e1200 RDI:
> ff9c
> [ 35.106663] RBP: 7ffcd41e1200 R08: R09:
> 0064
> [ 35.107923] R10: 01a4 R11: 0202 R12:
> 0066
> [ 35.109112] R13: 0010 R14: 0010 R15:
> 0400
> [ 35.110357]
> [ 35.110769] irq event stamp: 8415587
> [ 35.111486] hardirqs last enabled at (8415599): []
> __up_console_sem+0x52/0x60
> [ 35.113067] hardirqs last disabled at (8415610): []
> __up_console_sem+0x37/0x60
> [ 35.114575] softirqs last enabled at (8415300): []
> handle_softirqs+0x315/0x3f0
> [ 35.115933] softirqs last disabled at (8415291): []
> __irq_exit_rcu+0xa1/0xc0
> [ 35.117316] ---[ end trace ]---
>
> Fix this by using xas_reset() instead, which is equivalent in
> implementation to xas_pause() but does not advance the XArray state.
>
> Fixes: 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
> entries")
> Signed-off-by: Alistair Popple
> Cc: Dan Williams
> Cc: Alison Schofield
> Cc: Matthew Wilcow (Oracle)
> Cc: Balbir Singh
> Cc: "Darrick J. Wong"
> Cc: Dave Chinner
> Cc: David Hildenbrand
> Cc: Jan Kara
> Cc: John Hubbard
> Cc: Ted Ts'o
> Cc: Alexander Viro
> Cc: Christian Brauner
Looks good to me. Feel free to add:
Reviewed-by: Jan Kara
[PATCH] fs/dax: Fix "don't skip locked entries when scanning entries"
Commit 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning
entries") introduced a new function, wait_entry_unlocked_exclusive(),
which waits for the current entry to become unlocked without advancing
the XArray iterator state.
Waiting for the entry to become unlocked requires dropping the XArray
lock. This requires calling xas_pause() prior to dropping the lock
which leaves the xas in a suitable state for the next iteration. However
this has the side-effect of advancing the xas state to the next index.
Normally this isn't an issue because xas_for_each() contains code to
detect this state and thus avoid advancing the index a second time on
the next loop iteration.
However both callers of and wait_entry_unlocked_exclusive() itself
subsequently use the xas state to reload the entry. As xas_pause()
updated the state to the next index this will cause the current entry
which is being waited on to be skipped. This caused the following
warning to fire intermittently when running xftest generic/068 on an XFS
filesystem with FS DAX enabled:
[ 35.067397] [ cut here ]
[ 35.068229] WARNING: CPU: 21 PID: 1640 at mm/truncate.c:89
truncate_folio_batch_exceptionals+0xd8/0x1e0
[ 35.069717] Modules linked in: nd_pmem dax_pmem nd_btt nd_e820 libnvdimm
[ 35.071006] CPU: 21 UID: 0 PID: 1640 Comm: fstest Not tainted 6.15.0-rc7+
#77 PREEMPT(voluntary)
[ 35.072613] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/204
[ 35.074845] RIP: 0010:truncate_folio_batch_exceptionals+0xd8/0x1e0
[ 35.075962] Code: a1 00 00 00 f6 47 0d 20 0f 84 97 00 00 00 4c 63 e8 41 39
c4 7f 0b eb 61 49 83 c5 01 45 39 ec 7e 58 42 f68
[ 35.079522] RSP: 0018:b04e426c7850 EFLAGS: 00010202
[ 35.080359] RAX: RBX: 9d21e3481908 RCX: b04e426c77f4
[ 35.081477] RDX: b04e426c79e8 RSI: b04e426c79e0 RDI: 9d21e34816e8
[ 35.082590] RBP: b04e426c79e0 R08: 0001 R09: 0003
[ 35.083733] R10: R11: 822b53c0f7a49868 R12: 001f
[ 35.084850] R13: R14: b04e426c78e8 R15: fffe
[ 35.085953] FS: 7f9134c87740() GS:9d22abba()
knlGS:
[ 35.087346] CS: 0010 DS: ES: CR0: 80050033
[ 35.088244] CR2: 7f9134c86000 CR3: 00040afff000 CR4: 06f0
[ 35.089354] Call Trace:
[ 35.089749]
[ 35.090168] truncate_inode_pages_range+0xfc/0x4d0
[ 35.091078] truncate_pagecache+0x47/0x60
[ 35.091735] xfs_setattr_size+0xc7/0x3e0
[ 35.092648] xfs_vn_setattr+0x1ea/0x270
[ 35.093437] notify_change+0x1f4/0x510
[ 35.094219] ? do_truncate+0x97/0xe0
[ 35.094879] do_truncate+0x97/0xe0
[ 35.095640] path_openat+0xabd/0xca0
[ 35.096278] do_filp_open+0xd7/0x190
[ 35.096860] do_sys_openat2+0x8a/0xe0
[ 35.097459] __x64_sys_openat+0x6d/0xa0
[ 35.098076] do_syscall_64+0xbb/0x1d0
[ 35.098647] entry_SYSCALL_64_after_hwframe+0x77/0x7f
[ 35.099444] RIP: 0033:0x7f9134d81fc1
[ 35.100033] Code: 75 57 89 f0 25 00 00 41 00 3d 00 00 41 00 74 49 80 3d 2a
26 0e 00 00 74 6d 89 da 48 89 ee bf 9c ff ff ff5
[ 35.102993] RSP: 002b:7ffcd41e0d10 EFLAGS: 0202 ORIG_RAX:
0101
[ 35.104263] RAX: ffda RBX: 0242 RCX: 7f9134d81fc1
[ 35.105452] RDX: 0242 RSI: 7ffcd41e1200 RDI: ff9c
[ 35.106663] RBP: 7ffcd41e1200 R08: R09: 0064
[ 35.107923] R10: 01a4 R11: 0202 R12: 0066
[ 35.109112] R13: 0010 R14: 0010 R15: 0400
[ 35.110357]
[ 35.110769] irq event stamp: 8415587
[ 35.111486] hardirqs last enabled at (8415599): []
__up_console_sem+0x52/0x60
[ 35.113067] hardirqs last disabled at (8415610): []
__up_console_sem+0x37/0x60
[ 35.114575] softirqs last enabled at (8415300): []
handle_softirqs+0x315/0x3f0
[ 35.115933] softirqs last disabled at (8415291): []
__irq_exit_rcu+0xa1/0xc0
[ 35.117316] ---[ end trace ]---
Fix this by using xas_reset() instead, which is equivalent in
implementation to xas_pause() but does not advance the XArray state.
Fixes: 6be3e21d25ca ("fs/dax: don't skip locked entries when scanning entries")
Signed-off-by: Alistair Popple
Cc: Dan Williams
Cc: Alison Schofield
Cc: Matthew Wilcow (Oracle)
Cc: Balbir Singh
Cc: "Darrick J. Wong"
Cc: Dave Chinner
Cc: David Hildenbrand
Cc: Jan Kara
Cc: John Hubbard
Cc: Ted Ts'o
Cc: Alexander Viro
Cc: Christian Brauner
---
Hi Andrew,
Apologies for finding this so late in the cycle. This is a very
intermittent issue for me, and it seems it was only exposed by a recent
upgrade to my test machine/setup. The user visible impact is the same
as for the original commit this fixes. That is possible file data
corruption if a device has a FS DAX page pinned for DMA.
So in other words it means
