Re: [PATCH v3 0/4] kernfs: proposed locking and concurrency improvement

2021-04-19 Thread Ian Kent
On Mon, 2021-04-19 at 15:56 +0800, Fox Chen wrote: > On Fri, Apr 9, 2021 at 9:14 AM Ian Kent wrote: > > There have been a few instances of contention on the kernfs_mutex > > during > > path walks, a case on very large IBM systems seen by myself, a > > report by >

Re: [PATCH v3 2/4] kernfs: use VFS negative dentry caching

2021-04-09 Thread Ian Kent
On Fri, 2021-04-09 at 16:26 +0800, Ian Kent wrote: > On Fri, 2021-04-09 at 01:35 +, Al Viro wrote: > > On Fri, Apr 09, 2021 at 09:15:06AM +0800, Ian Kent wrote: > > > + parent = kernfs_dentry_node(dentry->d_parent); > > > + if (parent) { > >

Re: [PATCH v3 2/4] kernfs: use VFS negative dentry caching

2021-04-09 Thread Ian Kent
On Fri, 2021-04-09 at 01:35 +, Al Viro wrote: > On Fri, Apr 09, 2021 at 09:15:06AM +0800, Ian Kent wrote: > > + parent = kernfs_dentry_node(dentry->d_parent); > > + if (parent) { > > +

[PATCH v3 4/4] kernfs: use i_lock to protect concurrent inode updates

2021-04-08 Thread Ian Kent
processes concurrently updating the inode attributes and link count in kernfs_refresh_inode(). The inode i_lock seems like the sensible thing to use to protect these inode attribute updates so use it in kernfs_refresh_inode(). Signed-off-by: Ian Kent --- fs/kernfs/inode.c | 10 ++ fs

[PATCH v3 3/4] kernfs: switch kernfs to use an rwsem

2021-04-08 Thread Ian Kent
The kernfs global lock restricts the ability to perform kernfs node lookup operations in parallel during path walks. Change the kernfs mutex to an rwsem so that, when opportunity arises, node searches can be done in parallel with path walk lookups. Signed-off-by: Ian Kent --- fs/kernfs/dir.c

[PATCH v3 2/4] kernfs: use VFS negative dentry caching

2021-04-08 Thread Ian Kent
these lookups. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 55 +-- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index 4c69e2af82dac..edfeee1bf38ec 100644 --- a/fs/kernfs/dir.c +++ b/fs/kernfs

[PATCH v3 1/4] kernfs: move revalidate to be near lookup

2021-04-08 Thread Ian Kent
ths it will need to call kernfs_find_ns(). So, to avoid a forward declaration, move it to be near kernfs_iop_lookup(). There's no functional change from this patch. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 86 --- 1 file changed, 43 insertions(+),

[PATCH v3 0/4] kernfs: proposed locking and concurrency improvement

2021-04-08 Thread Ian Kent
in rcu-walk mode. - drop the use a revision to identify if a directory has changed patch. Changes since v1: - fix locking in .permission() and .getattr() by re-factoring the attribute handling code. --- Ian Kent (4): kernfs: move revalidate to be near lookup kernfs: use VFS negative

Re: [RFC PATCH] autofs: find_autofs_mount overmounted parent support

2021-03-23 Thread Ian Kent
On Tue, 2021-03-09 at 13:43 +0300, Alexander Mikhalitsyn wrote: > On Sat, 06 Mar 2021 17:13:32 +0800 > Ian Kent wrote: > > > On Fri, 2021-03-05 at 14:55 +0300, Alexander Mikhalitsyn wrote: > > > On Fri, 05 Mar 2021 18:10:02 +0800 > > > Ian Kent wrote: > >

Re: [RFC PATCH] autofs: find_autofs_mount overmounted parent support

2021-03-06 Thread Ian Kent
On Fri, 2021-03-05 at 14:55 +0300, Alexander Mikhalitsyn wrote: > On Fri, 05 Mar 2021 18:10:02 +0800 > Ian Kent wrote: > > > On Thu, 2021-03-04 at 13:11 +0300, Alexander Mikhalitsyn wrote: > > > On Thu, 04 Mar 2021 14:54:11 +0800 > > > Ian Kent wrote: > >

Re: [RFC PATCH] autofs: find_autofs_mount overmounted parent support

2021-03-05 Thread Ian Kent
On Thu, 2021-03-04 at 13:11 +0300, Alexander Mikhalitsyn wrote: > On Thu, 04 Mar 2021 14:54:11 +0800 > Ian Kent wrote: > > > On Wed, 2021-03-03 at 18:28 +0300, Alexander Mikhalitsyn wrote: > > > It was discovered that find_autofs_mount() function > > > in aut

Re: [RFC PATCH] autofs: find_autofs_mount overmounted parent support

2021-03-03 Thread Ian Kent
On Wed, 2021-03-03 at 18:28 +0300, Alexander Mikhalitsyn wrote: > It was discovered that find_autofs_mount() function > in autofs not support cases when autofs mount > parent is overmounted. In this case this function will > always return -ENOENT. Ok, I get this shouldn't happen. > > Real-life

[ANNOUNCE] autofs 5.1.7 release

2021-01-26 Thread Ian Kent
Hi all, It's time for a release, autofs-5.1.7. As with autofs-5.1.6 work to resolve difficulties using very large large direct mount maps has continued but there have been some difficulties. Trying to get back to the situation that existed before symlinking of the mount table can't be done in

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-21 Thread Ian Kent
On Sat, 2020-12-19 at 15:47 +0800, Fox Chen wrote: > On Sat, Dec 19, 2020 at 8:53 AM Ian Kent wrote: > > On Fri, 2020-12-18 at 21:20 +0800, Fox Chen wrote: > > > On Fri, Dec 18, 2020 at 7:21 PM Ian Kent > > > wrote: > > > > On Fri, 2020-12-18 at 16:01 +0800

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-19 Thread Ian Kent
On Sun, 2020-12-20 at 07:52 +0800, Ian Kent wrote: > On Sat, 2020-12-19 at 11:23 -0500, Tejun Heo wrote: > > Hello, > > > > On Sat, Dec 19, 2020 at 03:08:13PM +0800, Ian Kent wrote: > > > And looking further I see there's a race that kernfs can't do >

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-19 Thread Ian Kent
On Sat, 2020-12-19 at 11:23 -0500, Tejun Heo wrote: > Hello, > > On Sat, Dec 19, 2020 at 03:08:13PM +0800, Ian Kent wrote: > > And looking further I see there's a race that kernfs can't do > > anything > > about between kernfs_refresh_inode() and fs/inode.c:update_ti

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-18 Thread Ian Kent
On Fri, 2020-12-18 at 09:59 -0500, Tejun Heo wrote: > Hello, > > On Fri, Dec 18, 2020 at 03:36:21PM +0800, Ian Kent wrote: > > Sounds like your saying it would be ok to add a lock to the > > attrs structure, am I correct? > > Yeah, adding a lock to attrs is a lot les

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-18 Thread Ian Kent
On Fri, 2020-12-18 at 21:20 +0800, Fox Chen wrote: > On Fri, Dec 18, 2020 at 7:21 PM Ian Kent wrote: > > On Fri, 2020-12-18 at 16:01 +0800, Fox Chen wrote: > > > On Fri, Dec 18, 2020 at 3:36 PM Ian Kent > > > wrote: > > > > On Thu, 2020-12-17 at 10:14

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-18 Thread Ian Kent
On Fri, 2020-12-18 at 16:01 +0800, Fox Chen wrote: > On Fri, Dec 18, 2020 at 3:36 PM Ian Kent wrote: > > On Thu, 2020-12-17 at 10:14 -0500, Tejun Heo wrote: > > > Hello, > > > > > > On Thu, Dec 17, 2020 at 07:48:49PM +0800, Ian Kent wrote: > > > >

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-17 Thread Ian Kent
On Thu, 2020-12-17 at 10:14 -0500, Tejun Heo wrote: > Hello, > > On Thu, Dec 17, 2020 at 07:48:49PM +0800, Ian Kent wrote: > > > What could be done is to make the kernfs node attr_mutex > > > a pointer and dynamically allocate it but even that is too > > >

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-17 Thread Ian Kent
On Thu, 2020-12-17 at 19:09 +0800, Ian Kent wrote: > On Thu, 2020-12-17 at 18:09 +0800, Ian Kent wrote: > > On Thu, 2020-12-17 at 16:54 +0800, Fox Chen wrote: > > > On Thu, Dec 17, 2020 at 12:46 PM Ian Kent > > > wrote: > > > > On Tue, 2020-12-15 at 20:59

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-17 Thread Ian Kent
On Thu, 2020-12-17 at 18:09 +0800, Ian Kent wrote: > On Thu, 2020-12-17 at 16:54 +0800, Fox Chen wrote: > > On Thu, Dec 17, 2020 at 12:46 PM Ian Kent wrote: > > > On Tue, 2020-12-15 at 20:59 +0800, Ian Kent wrote: > > > > On Tue, 2020-12-15 at 16:33 +0800, Fox Chen w

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-17 Thread Ian Kent
On Thu, 2020-12-17 at 16:54 +0800, Fox Chen wrote: > On Thu, Dec 17, 2020 at 12:46 PM Ian Kent wrote: > > On Tue, 2020-12-15 at 20:59 +0800, Ian Kent wrote: > > > On Tue, 2020-12-15 at 16:33 +0800, Fox Chen wrote: > > > > On Mon, Dec 14, 2020 at 9:30 PM Ian Kent &

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-16 Thread Ian Kent
On Tue, 2020-12-15 at 20:59 +0800, Ian Kent wrote: > On Tue, 2020-12-15 at 16:33 +0800, Fox Chen wrote: > > On Mon, Dec 14, 2020 at 9:30 PM Ian Kent wrote: > > > On Mon, 2020-12-14 at 14:14 +0800, Fox Chen wrote: > > > > On Sun, Dec 13, 2020 at 11:46 AM Ian Kent &

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-15 Thread Ian Kent
On Tue, 2020-12-15 at 16:33 +0800, Fox Chen wrote: > On Mon, Dec 14, 2020 at 9:30 PM Ian Kent wrote: > > On Mon, 2020-12-14 at 14:14 +0800, Fox Chen wrote: > > > On Sun, Dec 13, 2020 at 11:46 AM Ian Kent > > > wrote: > > > > On Fri, 2020-12-11 at 10:17

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-14 Thread Ian Kent
On Mon, 2020-12-14 at 14:14 +0800, Fox Chen wrote: > On Sun, Dec 13, 2020 at 11:46 AM Ian Kent wrote: > > On Fri, 2020-12-11 at 10:17 +0800, Ian Kent wrote: > > > On Fri, 2020-12-11 at 10:01 +0800, Ian Kent wrote: > > > > > For the patches, there

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-12 Thread Ian Kent
On Fri, 2020-12-11 at 10:17 +0800, Ian Kent wrote: > On Fri, 2020-12-11 at 10:01 +0800, Ian Kent wrote: > > > For the patches, there is a mutex_lock in kn->attr_mutex, as > > > Tejun > > > mentioned here > > > (https://lore.kernel.org/lkml/x8fe0cmu+aq1g.

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-10 Thread Ian Kent
On Fri, 2020-12-11 at 10:01 +0800, Ian Kent wrote: > > > For the patches, there is a mutex_lock in kn->attr_mutex, as Tejun > > mentioned here > > (https://lore.kernel.org/lkml/x8fe0cmu+aq1g...@mtj.duckdns.org/), > > maybe a global > > rwsem for kn->ia

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-12-10 Thread Ian Kent
On Thu, 2020-12-10 at 16:44 +, Fox Chen wrote: > Hi, > > I found this series of patches solves exact the problem I am trying > to solve. > https://lore.kernel.org/lkml/20201202145837.48040-1-foxhlc...@gmail.com/ Right. > > The problem is reported by Brice Goglin on thread: > Re: [PATCH

Re: 5.9.0-next-20201015: autofs oops in update-binfmts

2020-10-16 Thread Ian Kent
sion > 15.32.2.9 > [ 59.809322] traps: clock-applet[3636] trap int3 ip:b724ffc0 > sp:bf879b90 error:0 in libglib-2.0.so.0.5000.3[b7203000+12a000] > [ 59.812036] traps: mateweather-app[3638] trap int3 ip:b7283fc0 > sp:bfb65760 error:0 in libglib-2.0.so.0.5000.3[b7237000+12a000] > [ 64.628401] wlan0: authenticate with 5c:f4:ab:10:d2:bb > -- Ian Kent

Re: [PATCH] Harden autofs ioctl table

2020-08-19 Thread Ian Kent
; > Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Ian Kent Thanks Matthew, I certainly want to know about changes to autofs made by others so thanks for sending this to me. At the same time I need to send my patches via someone else as Linus asked me to do ages ago now. So, once again Andrew,

Re: file metadata via fs API

2020-08-12 Thread Ian Kent
On Wed, 2020-08-12 at 12:50 -0700, Linus Torvalds wrote: > On Wed, Aug 12, 2020 at 12:34 PM Steven Whitehouse < > swhit...@redhat.com> wrote: > > The point of this is to give us the ability to monitor mounts from > > userspace. > > We haven't had that before, I don't see why it's suddenly such a

Re: file metadata via fs API

2020-08-12 Thread Ian Kent
On Wed, 2020-08-12 at 14:06 +0100, David Howells wrote: > Miklos Szeredi wrote: > > > That presumably means the mount ID <-> mount path mapping already > > exists, which means it's just possible to use the open(mount_path, > > O_PATH) to obtain the base fd. > > No, you can't. A path more

Re: file metadata via fs API (was: [GIT PULL] Filesystem Information)

2020-08-11 Thread Ian Kent
On Tue, 2020-08-11 at 21:39 +0200, Christian Brauner wrote: > On Tue, Aug 11, 2020 at 09:05:22AM -0700, Linus Torvalds wrote: > > On Tue, Aug 11, 2020 at 8:30 AM Miklos Szeredi > > wrote: > > > What's the disadvantage of doing it with a single lookup WITH an > > > enabling flag? > > > > > > It's

Re: [PATCH] fs: autofs: delete repeated words in comments

2020-08-11 Thread Ian Kent
On Tue, 2020-08-11 at 07:42 -0700, Randy Dunlap wrote: > On 8/11/20 1:36 AM, Ian Kent wrote: > > On Mon, 2020-08-10 at 19:18 -0700, Randy Dunlap wrote: > > > Drop duplicated words {the, at} in comments. > > > > > > Signed-off-by: Randy Dunlap > > > C

Re: [PATCH] fs: autofs: delete repeated words in comments

2020-08-11 Thread Ian Kent
On Mon, 2020-08-10 at 19:18 -0700, Randy Dunlap wrote: > Drop duplicated words {the, at} in comments. > > Signed-off-by: Randy Dunlap > Cc: Ian Kent > Cc: aut...@vger.kernel.org Acked-by: Ian Kent > --- > fs/autofs/dev-ioctl.c |4 ++-- > 1 file changed, 2 in

Re: [PATCH 06/18] fsinfo: Add a uniquifier ID to struct mount [ver #21]

2020-08-05 Thread Ian Kent
On Wed, 2020-08-05 at 20:33 +0100, Matthew Wilcox wrote: > On Wed, Aug 05, 2020 at 04:30:10PM +0100, David Howells wrote: > > Miklos Szeredi wrote: > > > > > idr_alloc_cyclic() seems to be a good template for doing the > > > lower > > > 32bit allocation, and we can add code to increment the high

Re: [PATCH 10/18] fsinfo: Provide notification overrun handling support [ver #21]

2020-08-05 Thread Ian Kent
On Wed, 2020-08-05 at 13:27 +0200, Miklos Szeredi wrote: > On Wed, Aug 5, 2020 at 1:23 PM Ian Kent wrote: > > On Wed, 2020-08-05 at 09:45 +0200, Miklos Szeredi wrote: > > > Hmm, what's the other possibility for lost notifications? > > > > In user space th

Re: [PATCH 10/18] fsinfo: Provide notification overrun handling support [ver #21]

2020-08-05 Thread Ian Kent
On Wed, 2020-08-05 at 09:45 +0200, Miklos Szeredi wrote: > On Wed, Aug 5, 2020 at 4:46 AM Ian Kent wrote: > > Coming back to an actual use case. > > > > What I said above is one aspect but, since I'm looking at this > > right > > now with systemd, and I do

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-08-05 Thread Ian Kent
On Wed, 2020-08-05 at 09:43 +0200, Miklos Szeredi wrote: > On Wed, Aug 5, 2020 at 3:54 AM Ian Kent wrote: > > > > It's way more useful to have these in the notification than > > > > obtainable > > > > via fsinfo() IMHO. > > > > > > Wha

Re: [GIT PULL] Filesystem Information

2020-08-05 Thread Ian Kent
On Wed, 2020-08-05 at 10:00 +0200, Miklos Szeredi wrote: > On Wed, Aug 5, 2020 at 3:33 AM Ian Kent wrote: > > On Tue, 2020-08-04 at 16:36 +0200, Miklos Szeredi wrote: > > > And notice how similar the above interface is to getxattr(), or > > > the > >

Re: [PATCH 10/18] fsinfo: Provide notification overrun handling support [ver #21]

2020-08-04 Thread Ian Kent
On Wed, 2020-08-05 at 10:05 +0800, Ian Kent wrote: > On Tue, 2020-08-04 at 15:56 +0200, Miklos Szeredi wrote: > > On Mon, Aug 03, 2020 at 02:37:50PM +0100, David Howells wrote: > > > Provide support for the handling of an overrun in a watch > > > queue. In the > &g

Re: [PATCH 10/18] fsinfo: Provide notification overrun handling support [ver #21]

2020-08-04 Thread Ian Kent
On Tue, 2020-08-04 at 15:56 +0200, Miklos Szeredi wrote: > On Mon, Aug 03, 2020 at 02:37:50PM +0100, David Howells wrote: > > Provide support for the handling of an overrun in a watch > > queue. In the > > event that an overrun occurs, the watcher needs to be able to find > > out what > > it was

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-08-04 Thread Ian Kent
On Tue, 2020-08-04 at 15:19 +0200, Miklos Szeredi wrote: > On Tue, Aug 4, 2020 at 1:39 PM Ian Kent wrote: > > On Mon, 2020-08-03 at 11:29 +0200, Miklos Szeredi wrote: > > > On Thu, Jul 23, 2020 at 12:48 PM David Howells < > > > dh

Re: [GIT PULL] Filesystem Information

2020-08-04 Thread Ian Kent
On Tue, 2020-08-04 at 16:36 +0200, Miklos Szeredi wrote: > On Tue, Aug 4, 2020 at 4:15 AM Ian Kent wrote: > > On Mon, 2020-08-03 at 18:42 +0200, Miklos Szeredi wrote: > > > On Mon, Aug 3, 2020 at 5:50 PM David Howells > > > > > > wrote: > > > > Hi

Re: [PATCH 15/18] fsinfo: Add an attribute that lists all the visible mounts in a namespace [ver #21]

2020-08-04 Thread Ian Kent
On Tue, 2020-08-04 at 16:05 +0200, Miklos Szeredi wrote: > On Mon, Aug 03, 2020 at 02:38:34PM +0100, David Howells wrote: > > Add a filesystem attribute that exports a list of all the visible > > mounts in > > a namespace, given the caller's chroot setting. The returned list > > is an > > array

Re: [PATCH 06/18] fsinfo: Add a uniquifier ID to struct mount [ver #21]

2020-08-04 Thread Ian Kent
On Tue, 2020-08-04 at 12:41 +0200, Miklos Szeredi wrote: > On Mon, Aug 03, 2020 at 02:37:16PM +0100, David Howells wrote: > > Add a uniquifier ID to struct mount that is effectively unique over > > the > > kernel lifetime to deal around mnt_id values being reused. This > > can then > > be

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-08-04 Thread Ian Kent
On Mon, 2020-08-03 at 11:29 +0200, Miklos Szeredi wrote: > On Thu, Jul 23, 2020 at 12:48 PM David Howells > wrote: > > > > > __u32 topology_changes; > > > > __u32 attr_changes; > > > > __u32 aux_topology_changes; > > > > > > Being 32bit this

Re: [GIT PULL] Filesystem Information

2020-08-03 Thread Ian Kent
one. > > The counters got made atomic_long_t inside the kernel and __u64 in > > the > > UAPI. The aggregate changes can be assessed by comparing pre- > > change tag, > > fsinfo-core-20200724 to the requested pull tag. > > > > Karel Zak has created preliminar

Re: [GIT PULL] Mount notifications

2020-08-03 Thread Ian Kent
n whether or not a mount > watch > may be set on a particular path. > > This series is intended to be taken in conjunction with the fsinfo > series > which I'll post a pull request for shortly and which is dependent on > it. > > Karel Zak[*] has created preliminary patches tha

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-08-03 Thread Ian Kent
On Mon, 2020-08-03 at 13:31 +0100, David Howells wrote: > Ian Kent wrote: > > > > I'm changing it so that the fields are 64-bit, but initialised > > > with the > > > existing mount ID in the notifications set. The fsinfo set > > > changes that

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-08-03 Thread Ian Kent
On Mon, 2020-08-03 at 12:49 +0100, David Howells wrote: > Miklos Szeredi wrote: > > > OTOH mount notification is way smaller and IMO a more mature > > interface. So just picking the unique ID patch into this set might > > make sense. > > But userspace can't retrieve the unique ID without

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-07-24 Thread Ian Kent
On Fri, 2020-07-24 at 11:19 +0100, David Howells wrote: > David Howells wrote: > > > > What guarantees that mount_id is going to remain a 32bit entity? > > > > You think it likely we'd have >4 billion concurrent mounts on a > > system? That > > would require >1.2TiB of RAM just for the struct

Re: [PATCH] autofs: fix doubled word

2020-07-15 Thread Ian Kent
On Wed, 2020-07-15 at 18:28 -0700, Randy Dunlap wrote: > From: Randy Dunlap > > Change doubled word "is" to "it is". > > Signed-off-by: Randy Dunlap > Cc: Ian Kent > Cc: aut...@vger.kernel.org > Cc: Andrew Morton Acked-by: Ian Kent > --

Re: [PATCH 01/10] Documentation: filesystems: autofs-mount-control: drop doubled words

2020-07-05 Thread Ian Kent
On Fri, 2020-07-03 at 14:43 -0700, Randy Dunlap wrote: > Drop the doubled words "the" and "and". > > Signed-off-by: Randy Dunlap > Cc: Jonathan Corbet > Cc: linux-...@vger.kernel.org > Cc: Ian Kent Acked-by: Ian Kent > Cc: aut...@vger.kernel.org

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-25 Thread Ian Kent
On Thu, 2020-06-25 at 11:43 +0200, Greg Kroah-Hartman wrote: > On Thu, Jun 25, 2020 at 04:15:19PM +0800, Ian Kent wrote: > > On Tue, 2020-06-23 at 19:13 -0400, Tejun Heo wrote: > > > Hello, Rick. > > > > > > On Mon, Jun 22, 2020 at 02:22:34PM -0700, Rick Li

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-25 Thread Ian Kent
On Tue, 2020-06-23 at 19:13 -0400, Tejun Heo wrote: > Hello, Rick. > > On Mon, Jun 22, 2020 at 02:22:34PM -0700, Rick Lindsley wrote: > > > I don't know. The above highlights the absurdity of the approach > > > itself to > > > me. You seem to be aware of it too in writing: 250,000 "devices". > >

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-23 Thread Ian Kent
On Tue, 2020-06-23 at 02:33 -0700, Rick Lindsley wrote: > On 6/22/20 11:02 PM, Greg Kroah-Hartman wrote: > > > First off, this is not my platform, and not my problem, so it's > > funny > > you ask me :) > > Wlll, not your platform perhaps but MAINTAINERS does list you > first and Tejun

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-23 Thread Ian Kent
On Tue, 2020-06-23 at 16:01 +0800, Ian Kent wrote: > On Tue, 2020-06-23 at 08:02 +0200, Greg Kroah-Hartman wrote: > > On Tue, Jun 23, 2020 at 01:09:08PM +0800, Ian Kent wrote: > > > On Mon, 2020-06-22 at 20:03 +0200, Greg Kroah-Hartman wrote: > > > > On Mon, Jun 22, 2

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-23 Thread Ian Kent
On Tue, 2020-06-23 at 08:02 +0200, Greg Kroah-Hartman wrote: > On Tue, Jun 23, 2020 at 01:09:08PM +0800, Ian Kent wrote: > > On Mon, 2020-06-22 at 20:03 +0200, Greg Kroah-Hartman wrote: > > > On Mon, Jun 22, 2020 at 01:48:45PM -0400, Tejun Heo wrote: > > > > Hello

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-22 Thread Ian Kent
On Mon, 2020-06-22 at 20:03 +0200, Greg Kroah-Hartman wrote: > On Mon, Jun 22, 2020 at 01:48:45PM -0400, Tejun Heo wrote: > > Hello, Ian. > > > > On Sun, Jun 21, 2020 at 12:55:33PM +0800, Ian Kent wrote: > > > > > They are used for hotplugging and par

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-20 Thread Ian Kent
On Fri, 2020-06-19 at 18:23 -0400, Tejun Heo wrote: > On Fri, Jun 19, 2020 at 01:41:39PM -0700, Rick Lindsley wrote: > > On 6/19/20 8:38 AM, Tejun Heo wrote: > > > > > I don't have strong objections to the series but the rationales > > > don't seem > > > particularly strong. It's solving a

Re: [PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-20 Thread Ian Kent
On Fri, 2020-06-19 at 11:38 -0400, Tejun Heo wrote: > Hello, Ian. > > On Wed, Jun 17, 2020 at 03:37:43PM +0800, Ian Kent wrote: > > The series here tries to reduce the locking needed during path > > walks > > based on the assumption that there are many path walks

[PATCH v2 4/6] kernfs: use revision to identify directory node changes

2020-06-17 Thread Ian Kent
If a kernfs directory node hasn't changed there's no need to search for an added (or removed) child dentry. Add a revision counter to kernfs directory nodes so it can be used to detect if a directory node has changed. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 17

[PATCH v2 2/6] kernfs: move revalidate to be near lookup

2020-06-17 Thread Ian Kent
eed for a forward declaration. There's no functional change from this patch. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 86 --- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index d8213fc65

[PATCH v2 6/6] kernfs: make attr_mutex a local kernfs node lock

2020-06-17 Thread Ian Kent
lead to contention when the number of concurrent path walks is high. This mutex is used for kernfs node objects only so make it local to the kernfs node to reduce the impact of this type of contention. Signed-off-by: Ian Kent --- fs/kernfs/dir.c|1 + fs/kernfs/inode.c | 12

[PATCH v2 1/6] kernfs: switch kernfs to use an rwsem

2020-06-17 Thread Ian Kent
The kernfs global lock restricts the ability to perform kernfs node lookup operations in parallel. Change the kernfs mutex to an rwsem so that, when oppertunity arises, node searches can be done in parallel. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 119

[PATCH v2 3/6] kernfs: improve kernfs path resolution

2020-06-17 Thread Ian Kent
where possible and use the dentry cache handling of negative hashed dentries to avoid allocating (and freeing shortly after) new dentries on every negative lookup. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 87 ++- 1 file changed, 72

[PATCH v2 5/6] kernfs: refactor attr locking

2020-06-17 Thread Ian Kent
-off-by: Ian Kent --- fs/kernfs/inode.c | 50 -- 1 file changed, 28 insertions(+), 22 deletions(-) diff --git a/fs/kernfs/inode.c b/fs/kernfs/inode.c index 23a7996d06a9..5c3fac356ce0 100644 --- a/fs/kernfs/inode.c +++ b/fs/kernfs/inode.c @@ -17,6

Re: [PATCH v2 1/6] kernfs: switch kernfs to use an rwsem

2020-06-17 Thread Ian Kent
On Wed, 2020-06-17 at 15:30 +0800, Ian Kent wrote: > The kernfs global lock restricts the ability to perform kernfs node > lookup operations in parallel. > > Change the kernfs mutex to an rwsem so that, when oppertunity arises, > node searches can be done in parallel. Please igno

[PATCH v2 0/6] kernfs: proposed locking and concurrency improvement

2020-06-17 Thread Ian Kent
the attribute handling code. --- Ian Kent (6): kernfs: switch kernfs to use an rwsem kernfs: move revalidate to be near lookup kernfs: improve kernfs path resolution kernfs: use revision to identify directory node changes kernfs: refactor attr locking kernfs: make

[PATCH v2 1/6] kernfs: switch kernfs to use an rwsem

2020-06-17 Thread Ian Kent
The kernfs global lock restricts the ability to perform kernfs node lookup operations in parallel. Change the kernfs mutex to an rwsem so that, when oppertunity arises, node searches can be done in parallel. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 119

Re: [PATCH 13/17] watch_queue: Implement mount topology and attribute change notifications [ver #5]

2020-06-13 Thread Ian Kent
On Thu, 2020-04-02 at 17:19 +0200, Miklos Szeredi wrote: > > > Firstly, a watch queue needs to be created: > > > > pipe2(fds, O_NOTIFICATION_PIPE); > > ioctl(fds[1], IOC_WATCH_QUEUE_SET_SIZE, 256); > > > > then a notification can be set up to report notifications via that > >

Re: [kernfs] ea7c5fc39a: stress-ng.stream.ops_per_sec 11827.2% improvement

2020-06-10 Thread Ian Kent
On Thu, 2020-06-11 at 10:06 +0800, kernel test robot wrote: > On Sun, Jun 07, 2020 at 09:13:08AM +0800, Ian Kent wrote: > > On Sat, 2020-06-06 at 20:18 +0200, Greg Kroah-Hartman wrote: > > > On Sat, Jun 06, 2020 at 11:52:16PM +0800, kernel test robot > > &

Re: [PATCH 1/4] kernfs: switch kernfs to use an rwsem

2020-06-08 Thread Ian Kent
On Sun, 2020-06-07 at 16:40 +0800, Ian Kent wrote: > Hi Greg, > > On Mon, 2020-05-25 at 13:47 +0800, Ian Kent wrote: > > @@ -189,9 +189,9 @@ int kernfs_iop_getattr(const struct path *path, > > struct kstat *stat, > > struct inode *inode = d_inode(path->dentry);

Re: [GIT PULL] General notification queue and key notifications

2020-06-07 Thread Ian Kent
On Wed, 2020-06-03 at 10:15 +0800, Ian Kent wrote: > On Tue, 2020-06-02 at 16:55 +0100, David Howells wrote: > > [[ With regard to the mount/sb notifications and fsinfo(), Karel > > Zak > > and > >Ian Kent have been working on making libmount use them, > &

Re: [PATCH 1/4] kernfs: switch kernfs to use an rwsem

2020-06-07 Thread Ian Kent
Hi Greg, On Mon, 2020-05-25 at 13:47 +0800, Ian Kent wrote: > @@ -189,9 +189,9 @@ int kernfs_iop_getattr(const struct path *path, > struct kstat *stat, > struct inode *inode = d_inode(path->dentry); > struct kernfs_node *kn = inode->i_private; > >

Re: [kernfs] ea7c5fc39a: stress-ng.stream.ops_per_sec 11827.2% improvement

2020-06-06 Thread Ian Kent
ommit: > > > > > > commit: ea7c5fc39ab005b501e0c7666c29db36321e4f74 ("[PATCH 1/4] > > kernfs: switch kernfs to use an rwsem") > > url: > > https://github.com/0day-ci/linux/commits/Ian-Kent/kernfs-proposed-locking-and-concurrency-improvement/20200525-134849 > > > > Seriou

Re: [GIT PULL] General notification queue and key notifications

2020-06-02 Thread Ian Kent
On Tue, 2020-06-02 at 16:55 +0100, David Howells wrote: > > [[ With regard to the mount/sb notifications and fsinfo(), Karel Zak > and >Ian Kent have been working on making libmount use them, > preparatory to >working on systemd: > > https://github.com/kar

Re: [PATCH 0/4] kernfs: proposed locking and concurrency improvement

2020-05-25 Thread Ian Kent
On Mon, 2020-05-25 at 08:16 +0200, Greg Kroah-Hartman wrote: > On Mon, May 25, 2020 at 01:46:59PM +0800, Ian Kent wrote: > > For very large systems with hundreds of CPUs and TBs of RAM booting > > can > > take a very long time. > > > > Initial reports sh

[PATCH 0/4] kernfs: proposed locking and concurrency improvement

2020-05-24 Thread Ian Kent
linux-next for broader exposure and hopefully have them merged into mainline. --- Ian Kent (4): kernfs: switch kernfs to use an rwsem kernfs: move revalidate to be near lookup kernfs: improve kernfs path resolution kernfs: use revision to identify directory node changes fs

[PATCH 1/4] kernfs: switch kernfs to use an rwsem

2020-05-24 Thread Ian Kent
The kernfs global lock restricts the ability to perform kernfs node lookup operations in parallel. Change the kernfs mutex to an rwsem so that, when oppertunity arises, node searches can be done in parallel. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 119

[PATCH 4/4] kernfs: use revision to identify directory node changes

2020-05-24 Thread Ian Kent
If a kernfs directory node hasn't changed there's no need to search for an added (or removed) child dentry. Add a revision counter to kernfs directory nodes so it can be used to detect if a directory node has changed. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 17

[PATCH 3/4] kernfs: improve kernfs path resolution

2020-05-24 Thread Ian Kent
where possible and use the dentry cache handling of negative hashed dentries to avoid allocating (and freeing shortly after) new dentries on every negative lookup. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 87 ++- 1 file changed, 72

[PATCH 2/4] kernfs: move revalidate to be near lookup

2020-05-24 Thread Ian Kent
eed for a forward declaration. There's no functional change from this patch. Signed-off-by: Ian Kent --- fs/kernfs/dir.c | 86 --- 1 file changed, 43 insertions(+), 43 deletions(-) diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c index d8213fc65

Re: [PATCH 02/14] autofs: switch to kernel_write

2020-05-14 Thread Ian Kent
should be fine AFAICS. Acked-by: Ian Kent > --- > fs/autofs/waitq.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/autofs/waitq.c b/fs/autofs/waitq.c > index b04c528b19d34..74c886f7c51cb 100644 > --- a/fs/autofs/waitq.c > +++ b/fs/autofs/waitq.

Re: mount on tmpfs failing to parse context option

2019-10-08 Thread Ian Kent
On Tue, 2019-10-08 at 20:38 +0800, Ian Kent wrote: > On Mon, 2019-10-07 at 17:50 -0700, Hugh Dickins wrote: > > On Mon, 7 Oct 2019, Laura Abbott wrote: > > > On 9/30/19 12:07 PM, Laura Abbott wrote: > > > > Hi, > > > > > > > > Fedora got a bu

Re: mount on tmpfs failing to parse context option

2019-10-08 Thread Ian Kent
On Mon, 2019-10-07 at 17:50 -0700, Hugh Dickins wrote: > On Mon, 7 Oct 2019, Laura Abbott wrote: > > On 9/30/19 12:07 PM, Laura Abbott wrote: > > > Hi, > > > > > > Fedora got a bug report > https://bugzilla.redhat.com/show_bug.cgi?id=1757104 > > > of a failure to parse options with the context

[ANNOUNCE] autofs 5.1.6 release

2019-10-06 Thread Ian Kent
Hi all, It's time for a release, autofs-5.1.6. This is an important release because it marks the beginning of work to be done torward resolving the very long standing problem of using very large direct mount maps in autofs, along with the needed mitigatigation of the effects of those large mount

Re: [PATCH 4/6] vfs: Allow mount information to be queried by fsinfo() [ver #15]

2019-07-02 Thread Ian Kent
On Wed, 2019-07-03 at 09:24 +0800, Ian Kent wrote: > On Wed, 2019-07-03 at 09:09 +0800, Ian Kent wrote: > > Hi Christian, > > > > About the propagation attributes you mentioned ... > > Umm ... how did you work out if a mount is unbindable from proc > mountinfo?

Re: [PATCH 4/6] vfs: Allow mount information to be queried by fsinfo() [ver #15]

2019-07-02 Thread Ian Kent
On Wed, 2019-07-03 at 09:09 +0800, Ian Kent wrote: > Hi Christian, > > About the propagation attributes you mentioned ... Umm ... how did you work out if a mount is unbindable from proc mountinfo? I didn't notice anything that could be used for that when I was looking at this. &g

Re: [PATCH 4/6] vfs: Allow mount information to be queried by fsinfo() [ver #15]

2019-07-02 Thread Ian Kent
Hi Christian, About the propagation attributes you mentioned ... On Fri, 2019-06-28 at 16:47 +0100, David Howells wrote: snip ... > + > +#ifdef CONFIG_FSINFO > +int fsinfo_generic_mount_info(struct path *path, struct fsinfo_kparams > *params) > +{ > + struct fsinfo_mount_info *p =

Re: [PATCH 00/25] VFS: Introduce filesystem information query syscall [ver #14]

2019-06-26 Thread Ian Kent
On Wed, 2019-06-26 at 12:47 +0200, Christian Brauner wrote: > On Wed, Jun 26, 2019 at 06:42:51PM +0800, Ian Kent wrote: > > On Wed, 2019-06-26 at 12:05 +0200, Christian Brauner wrote: > > > On Mon, Jun 24, 2019 at 03:08:45PM +0100, David Howells wrote: > > > >

Re: [PATCH 00/25] VFS: Introduce filesystem information query syscall [ver #14]

2019-06-26 Thread Ian Kent
On Wed, 2019-06-26 at 12:05 +0200, Christian Brauner wrote: > On Mon, Jun 24, 2019 at 03:08:45PM +0100, David Howells wrote: > > Hi Al, > > > > Here are a set of patches that adds a syscall, fsinfo(), that allows > > attributes of a filesystem/superblock to be queried. Attribute values are > >

Re: [PATCH 05/13] vfs: don't parse "silent" option

2019-06-19 Thread Ian Kent
On Wed, 2019-06-19 at 14:30 +0200, Miklos Szeredi wrote: > While this is a standard option as documented in mount(8), it is ignored by > most filesystems. So reject, unless filesystem explicitly wants to handle > it. > > The exception is unconverted filesystems, where it is unknown if the >

[PATCH] vfs: update d_make_root() description

2019-04-10 Thread Ian Kent
Clearify d_make_root() usage, error handling and cleanup requirements. Signed-off-by: Ian Kent --- Documentation/filesystems/porting | 15 +-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/Documentation/filesystems/porting b/Documentation/filesystems/porting

Re: kernel BUG at fs/inode.c:LINE!

2019-04-10 Thread Ian Kent
On Wed, 2019-04-10 at 14:41 +0200, Dmitry Vyukov wrote: > On Wed, Apr 10, 2019 at 2:12 PM Al Viro wrote: > > > > On Wed, Apr 10, 2019 at 08:07:15PM +0800, Ian Kent wrote: > > > > > > I'm unable to find a branch matching the line numbers. > &g

Re: kernel BUG at fs/inode.c:LINE!

2019-04-10 Thread Ian Kent
On Wed, 2019-04-10 at 19:57 +0800, Ian Kent wrote: > On Wed, 2019-04-10 at 13:40 +0200, Dmitry Vyukov wrote: > > On Wed, Apr 10, 2019 at 12:35 PM Ian Kent wrote: > > > > > > On Wed, 2019-04-10 at 10:27 +0200, Dmitry Vyukov wrote: > > > > On Wed,

Re: kernel BUG at fs/inode.c:LINE!

2019-04-10 Thread Ian Kent
On Wed, 2019-04-10 at 13:40 +0200, Dmitry Vyukov wrote: > On Wed, Apr 10, 2019 at 12:35 PM Ian Kent wrote: > > > > On Wed, 2019-04-10 at 10:27 +0200, Dmitry Vyukov wrote: > > > On Wed, Apr 10, 2019 at 2:26 AM Al Viro wrote: > > > > > > > > On Tue

Re: kernel BUG at fs/inode.c:LINE!

2019-04-10 Thread Ian Kent
On Wed, 2019-04-10 at 10:27 +0200, Dmitry Vyukov wrote: > On Wed, Apr 10, 2019 at 2:26 AM Al Viro wrote: > > > > On Tue, Apr 09, 2019 at 07:36:00AM -0700, syzbot wrote: > > > Bisection is inconclusive: the first bad commit could be any of: > > > > [snip the useless pile] > > > > > bisection

  1   2   3   4   5   6   7   8   9   10   >