Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Thu, 2016-10-27 at 14:50 +0800, Ian Kent wrote: > On Thu, 2016-10-27 at 10:47 +0800, Ian Kent wrote: > > > > On Thu, 2016-10-27 at 03:11 +0100, Al Viro wrote: > > > > > > > > > > > > How much testing did it get? I've several test setups involving > > > autofs, but they are nowhere near exhaustive and I don't have good > > > enough feel of the codebase to slap together something with decent > > > coverage... > > It got my standard testing. > > > > For that I use a modified version of the autofs Connectathon system. > > > > It's more about testing a wide variety of syntax and map setups and so > > exercises > > a large number of different types of autofs mounts. > > > > It's meant to check normal operation but not so much stress testing even > > though > > it does perform quite a few mounts (around 250-300, not to mention the > > autofs > > mounts themselves). > > > > I have another standard test I call the submount-test and it was originally > > done > > to stress test the most common problem I see, concurrent expire to mount. > > > > I didn't see any problems I couldn't explain in these but I might need to > > re- > > visit the submount-test to see if it is still doing what I want. > > > > OTOH, the pattern of mount and umount I see when the submount-test is run > > does > > look like it is doing what I want but it might not be getting all the way to > > the > > top of the tree of mounts enough times over the course of the test. > > > > So I'm happy with my testing, just not as happy as I could be. > Well, almost happy with my testing. > > Naturally I also tested the specific case this series is meant to fix. > > Basically: > ls /mnt/foo# do the initial automount > unshare -m sleep 10 & # hold the automount in a new namespace > umount /mnt/foo# pretend the mount timed out > ls /mnt/foo# try to access it again > ls: cannot open directory '/mnt/foo': Too many levels of symbolic links > > as seen on the autofs mailing list. My specific test was a little different > but > verified this was resolved. > > Now that Al seems reasonably OK with the series, with some changes, I'll test > some other use cases, mainly to verify the expire still functions as required. > That might need more work. I have done some further tests, specifically for (what I believe are) the two most common use cases. First, using automount(8) entirely within a container, as expected works fine. But the second case, one where automount(8) is run in the root namespace and has automount directories bound into a container does have a problem. The problem is due to may_umount_tree() only considering mounts in the root namespace and leads to expire attempts on mounts even if they are in use in another namespace. It's not a serious problem as the umount attempt fails because the mount is busy but it would be good to avoid the call back overhead. Unfortunately it looks like transforming may_umount_tree() to use a similar check to may_umount() introduces a race (picked up by my submount-test) which I'm struggling to understand, I'll continue to work on it. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Thu, 2016-10-27 at 14:50 +0800, Ian Kent wrote: > On Thu, 2016-10-27 at 10:47 +0800, Ian Kent wrote: > > > > On Thu, 2016-10-27 at 03:11 +0100, Al Viro wrote: > > > > > > > > > > > > How much testing did it get? I've several test setups involving > > > autofs, but they are nowhere near exhaustive and I don't have good > > > enough feel of the codebase to slap together something with decent > > > coverage... > > It got my standard testing. > > > > For that I use a modified version of the autofs Connectathon system. > > > > It's more about testing a wide variety of syntax and map setups and so > > exercises > > a large number of different types of autofs mounts. > > > > It's meant to check normal operation but not so much stress testing even > > though > > it does perform quite a few mounts (around 250-300, not to mention the > > autofs > > mounts themselves). > > > > I have another standard test I call the submount-test and it was originally > > done > > to stress test the most common problem I see, concurrent expire to mount. > > > > I didn't see any problems I couldn't explain in these but I might need to > > re- > > visit the submount-test to see if it is still doing what I want. > > > > OTOH, the pattern of mount and umount I see when the submount-test is run > > does > > look like it is doing what I want but it might not be getting all the way to > > the > > top of the tree of mounts enough times over the course of the test. > > > > So I'm happy with my testing, just not as happy as I could be. > Well, almost happy with my testing. > > Naturally I also tested the specific case this series is meant to fix. > > Basically: > ls /mnt/foo# do the initial automount > unshare -m sleep 10 & # hold the automount in a new namespace > umount /mnt/foo# pretend the mount timed out > ls /mnt/foo# try to access it again > ls: cannot open directory '/mnt/foo': Too many levels of symbolic links > > as seen on the autofs mailing list. My specific test was a little different > but > verified this was resolved. > > Now that Al seems reasonably OK with the series, with some changes, I'll test > some other use cases, mainly to verify the expire still functions as required. > That might need more work. I have done some further tests, specifically for (what I believe are) the two most common use cases. First, using automount(8) entirely within a container, as expected works fine. But the second case, one where automount(8) is run in the root namespace and has automount directories bound into a container does have a problem. The problem is due to may_umount_tree() only considering mounts in the root namespace and leads to expire attempts on mounts even if they are in use in another namespace. It's not a serious problem as the umount attempt fails because the mount is busy but it would be good to avoid the call back overhead. Unfortunately it looks like transforming may_umount_tree() to use a similar check to may_umount() introduces a race (picked up by my submount-test) which I'm struggling to understand, I'll continue to work on it. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Thu, 2016-10-27 at 10:47 +0800, Ian Kent wrote: > On Thu, 2016-10-27 at 03:11 +0100, Al Viro wrote: > > > > > > How much testing did it get? I've several test setups involving > > autofs, but they are nowhere near exhaustive and I don't have good > > enough feel of the codebase to slap together something with decent > > coverage... > It got my standard testing. > > For that I use a modified version of the autofs Connectathon system. > > It's more about testing a wide variety of syntax and map setups and so > exercises > a large number of different types of autofs mounts. > > It's meant to check normal operation but not so much stress testing even > though > it does perform quite a few mounts (around 250-300, not to mention the autofs > mounts themselves). > > I have another standard test I call the submount-test and it was originally > done > to stress test the most common problem I see, concurrent expire to mount. > > I didn't see any problems I couldn't explain in these but I might need to re- > visit the submount-test to see if it is still doing what I want. > > OTOH, the pattern of mount and umount I see when the submount-test is run does > look like it is doing what I want but it might not be getting all the way to > the > top of the tree of mounts enough times over the course of the test. > > So I'm happy with my testing, just not as happy as I could be. Well, almost happy with my testing. Naturally I also tested the specific case this series is meant to fix. Basically: ls /mnt/foo# do the initial automount unshare -m sleep 10 & # hold the automount in a new namespace umount /mnt/foo# pretend the mount timed out ls /mnt/foo# try to access it again ls: cannot open directory '/mnt/foo': Too many levels of symbolic links as seen on the autofs mailing list. My specific test was a little different but verified this was resolved. Now that Al seems reasonably OK with the series, with some changes, I'll test some other use cases, mainly to verify the expire still functions as required. That might need more work. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Thu, 2016-10-27 at 10:47 +0800, Ian Kent wrote: > On Thu, 2016-10-27 at 03:11 +0100, Al Viro wrote: > > > > > > How much testing did it get? I've several test setups involving > > autofs, but they are nowhere near exhaustive and I don't have good > > enough feel of the codebase to slap together something with decent > > coverage... > It got my standard testing. > > For that I use a modified version of the autofs Connectathon system. > > It's more about testing a wide variety of syntax and map setups and so > exercises > a large number of different types of autofs mounts. > > It's meant to check normal operation but not so much stress testing even > though > it does perform quite a few mounts (around 250-300, not to mention the autofs > mounts themselves). > > I have another standard test I call the submount-test and it was originally > done > to stress test the most common problem I see, concurrent expire to mount. > > I didn't see any problems I couldn't explain in these but I might need to re- > visit the submount-test to see if it is still doing what I want. > > OTOH, the pattern of mount and umount I see when the submount-test is run does > look like it is doing what I want but it might not be getting all the way to > the > top of the tree of mounts enough times over the course of the test. > > So I'm happy with my testing, just not as happy as I could be. Well, almost happy with my testing. Naturally I also tested the specific case this series is meant to fix. Basically: ls /mnt/foo# do the initial automount unshare -m sleep 10 & # hold the automount in a new namespace umount /mnt/foo# pretend the mount timed out ls /mnt/foo# try to access it again ls: cannot open directory '/mnt/foo': Too many levels of symbolic links as seen on the autofs mailing list. My specific test was a little different but verified this was resolved. Now that Al seems reasonably OK with the series, with some changes, I'll test some other use cases, mainly to verify the expire still functions as required. That might need more work. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Thu, 2016-10-27 at 03:11 +0100, Al Viro wrote: > On Fri, Oct 21, 2016 at 07:39:36AM +0800, Ian Kent wrote: > > > > > Maybe Al has been too busy to comment, he has been on the Cc from the start. > That's... a very mild version of what's been going on. Let's just say that > the last few weeks had been really interesting. Not that the shit has > settled, but there was some slackening in the shitstorm last few days. > Unlikely to last, I'm afraid, but... > > > > > Hopefully this email will prompt a review, Al? > Aside of the Eric's note re constifying struct path (strongly seconded), > I'm not sure if expiration-related side of that is correct. OTOH, > since the expiration happens from userland... Sure, I have a follow up series to do the constifying as recommended by Eric and now yourself. > > How much testing did it get? I've several test setups involving > autofs, but they are nowhere near exhaustive and I don't have good > enough feel of the codebase to slap together something with decent > coverage... It got my standard testing. For that I use a modified version of the autofs Connectathon system. It's more about testing a wide variety of syntax and map setups and so exercises a large number of different types of autofs mounts. It's meant to check normal operation but not so much stress testing even though it does perform quite a few mounts (around 250-300, not to mention the autofs mounts themselves). I have another standard test I call the submount-test and it was originally done to stress test the most common problem I see, concurrent expire to mount. I didn't see any problems I couldn't explain in these but I might need to re- visit the submount-test to see if it is still doing what I want. OTOH, the pattern of mount and umount I see when the submount-test is run does look like it is doing what I want but it might not be getting all the way to the top of the tree of mounts enough times over the course of the test. So I'm happy with my testing, just not as happy as I could be. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Thu, 2016-10-27 at 03:11 +0100, Al Viro wrote: > On Fri, Oct 21, 2016 at 07:39:36AM +0800, Ian Kent wrote: > > > > > Maybe Al has been too busy to comment, he has been on the Cc from the start. > That's... a very mild version of what's been going on. Let's just say that > the last few weeks had been really interesting. Not that the shit has > settled, but there was some slackening in the shitstorm last few days. > Unlikely to last, I'm afraid, but... > > > > > Hopefully this email will prompt a review, Al? > Aside of the Eric's note re constifying struct path (strongly seconded), > I'm not sure if expiration-related side of that is correct. OTOH, > since the expiration happens from userland... Sure, I have a follow up series to do the constifying as recommended by Eric and now yourself. > > How much testing did it get? I've several test setups involving > autofs, but they are nowhere near exhaustive and I don't have good > enough feel of the codebase to slap together something with decent > coverage... It got my standard testing. For that I use a modified version of the autofs Connectathon system. It's more about testing a wide variety of syntax and map setups and so exercises a large number of different types of autofs mounts. It's meant to check normal operation but not so much stress testing even though it does perform quite a few mounts (around 250-300, not to mention the autofs mounts themselves). I have another standard test I call the submount-test and it was originally done to stress test the most common problem I see, concurrent expire to mount. I didn't see any problems I couldn't explain in these but I might need to re- visit the submount-test to see if it is still doing what I want. OTOH, the pattern of mount and umount I see when the submount-test is run does look like it is doing what I want but it might not be getting all the way to the top of the tree of mounts enough times over the course of the test. So I'm happy with my testing, just not as happy as I could be. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Fri, Oct 21, 2016 at 07:39:36AM +0800, Ian Kent wrote: > Maybe Al has been too busy to comment, he has been on the Cc from the start. That's... a very mild version of what's been going on. Let's just say that the last few weeks had been really interesting. Not that the shit has settled, but there was some slackening in the shitstorm last few days. Unlikely to last, I'm afraid, but... > Hopefully this email will prompt a review, Al? Aside of the Eric's note re constifying struct path (strongly seconded), I'm not sure if expiration-related side of that is correct. OTOH, since the expiration happens from userland... How much testing did it get? I've several test setups involving autofs, but they are nowhere near exhaustive and I don't have good enough feel of the codebase to slap together something with decent coverage...
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Fri, Oct 21, 2016 at 07:39:36AM +0800, Ian Kent wrote: > Maybe Al has been too busy to comment, he has been on the Cc from the start. That's... a very mild version of what's been going on. Let's just say that the last few weeks had been really interesting. Not that the shit has settled, but there was some slackening in the shitstorm last few days. Unlikely to last, I'm afraid, but... > Hopefully this email will prompt a review, Al? Aside of the Eric's note re constifying struct path (strongly seconded), I'm not sure if expiration-related side of that is correct. OTOH, since the expiration happens from userland... How much testing did it get? I've several test setups involving autofs, but they are nowhere near exhaustive and I don't have good enough feel of the codebase to slap together something with decent coverage...
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Wed, 2016-10-19 at 12:40 -0700, Andrew Morton wrote: > On Tue, 11 Oct 2016 13:33:52 +0800 Ian Kentwrote: > > > For the autofs module to be able to reliably check if a dentry is a > > mountpoint in a multiple namespace environment the ->d_manage() dentry > > operation will need to take a path argument instead of a dentry. > > This patchset contains lots of ViroStuff. I'll queue it up for some > testing and will go into wait-and-see mode. Thanks Andrew. Maybe Al has been too busy to comment, he has been on the Cc from the start. Hopefully this email will prompt a review, Al? > > Some patches had an explicit From: Ian Kent and some > did not. I assumed that this was intended for all patches. So all > patches now have different From: and Signed-off-by: email addresses. > Unclear if this was your intent ;) Umm ... sorry about that, I didn't pay enough attention to the From of the patches when I resurrected them from an earlier attempt at this. Since the time I did this with an earlier patch series I have elected to not change my git config and set these in the local tree config so it shouldn't happen for new work. I'll pay special attention to this if I need to resurrect any other older patches too. I know this looks confusing and isn't the best but both email addresses have been present on my gpg key for a long time so it's verifiable the patches are from me. Once again sorry about that. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Wed, 2016-10-19 at 12:40 -0700, Andrew Morton wrote: > On Tue, 11 Oct 2016 13:33:52 +0800 Ian Kent wrote: > > > For the autofs module to be able to reliably check if a dentry is a > > mountpoint in a multiple namespace environment the ->d_manage() dentry > > operation will need to take a path argument instead of a dentry. > > This patchset contains lots of ViroStuff. I'll queue it up for some > testing and will go into wait-and-see mode. Thanks Andrew. Maybe Al has been too busy to comment, he has been on the Cc from the start. Hopefully this email will prompt a review, Al? > > Some patches had an explicit From: Ian Kent and some > did not. I assumed that this was intended for all patches. So all > patches now have different From: and Signed-off-by: email addresses. > Unclear if this was your intent ;) Umm ... sorry about that, I didn't pay enough attention to the From of the patches when I resurrected them from an earlier attempt at this. Since the time I did this with an earlier patch series I have elected to not change my git config and set these in the local tree config so it shouldn't happen for new work. I'll pay special attention to this if I need to resurrect any other older patches too. I know this looks confusing and isn't the best but both email addresses have been present on my gpg key for a long time so it's verifiable the patches are from me. Once again sorry about that. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Tue, 11 Oct 2016 13:33:52 +0800 Ian Kentwrote: > For the autofs module to be able to reliably check if a dentry is a > mountpoint in a multiple namespace environment the ->d_manage() dentry > operation will need to take a path argument instead of a dentry. This patchset contains lots of ViroStuff. I'll queue it up for some testing and will go into wait-and-see mode. Some patches had an explicit From: Ian Kent and some did not. I assumed that this was intended for all patches. So all patches now have different From: and Signed-off-by: email addresses. Unclear if this was your intent ;)
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Tue, 11 Oct 2016 13:33:52 +0800 Ian Kent wrote: > For the autofs module to be able to reliably check if a dentry is a > mountpoint in a multiple namespace environment the ->d_manage() dentry > operation will need to take a path argument instead of a dentry. This patchset contains lots of ViroStuff. I'll queue it up for some testing and will go into wait-and-see mode. Some patches had an explicit From: Ian Kent and some did not. I assumed that this was intended for all patches. So all patches now have different From: and Signed-off-by: email addresses. Unclear if this was your intent ;)
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Tue, 2016-10-11 at 11:04 -0500, Eric W. Biederman wrote: > Ian Kentwrites: > > > For the autofs module to be able to reliably check if a dentry is a > > mountpoint in a multiple namespace environment the ->d_manage() dentry > > operation will need to take a path argument instead of a dentry. > > Taking a quick look overall I see no issues with this series. Overall > it seems straight forward. > > On the nit side I expect saying const struct path * in the functions > that now take a struct path would be useful. > > I suspect it would also be useful to say > const struct path *path; > path = >f_path; > > In the one part of the code where you do that. Instead of copying the > path out of the struct file. > > Overall I expect that will keep down bugs at no reduction in usability. > Just a statement that the struct path won't change when it is passed > to various functions. Thanks Eric, that's a good suggestion for a follow up patch, will do. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
On Tue, 2016-10-11 at 11:04 -0500, Eric W. Biederman wrote: > Ian Kent writes: > > > For the autofs module to be able to reliably check if a dentry is a > > mountpoint in a multiple namespace environment the ->d_manage() dentry > > operation will need to take a path argument instead of a dentry. > > Taking a quick look overall I see no issues with this series. Overall > it seems straight forward. > > On the nit side I expect saying const struct path * in the functions > that now take a struct path would be useful. > > I suspect it would also be useful to say > const struct path *path; > path = >f_path; > > In the one part of the code where you do that. Instead of copying the > path out of the struct file. > > Overall I expect that will keep down bugs at no reduction in usability. > Just a statement that the struct path won't change when it is passed > to various functions. Thanks Eric, that's a good suggestion for a follow up patch, will do. Ian
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
Ian Kentwrites: > For the autofs module to be able to reliably check if a dentry is a > mountpoint in a multiple namespace environment the ->d_manage() dentry > operation will need to take a path argument instead of a dentry. Taking a quick look overall I see no issues with this series. Overall it seems straight forward. On the nit side I expect saying const struct path * in the functions that now take a struct path would be useful. I suspect it would also be useful to say const struct path *path; path = >f_path; In the one part of the code where you do that. Instead of copying the path out of the struct file. Overall I expect that will keep down bugs at no reduction in usability. Just a statement that the struct path won't change when it is passed to various functions. Eric
Re: [PATCH 1/8] vfs - change d_manage() to take a struct path
Ian Kent writes: > For the autofs module to be able to reliably check if a dentry is a > mountpoint in a multiple namespace environment the ->d_manage() dentry > operation will need to take a path argument instead of a dentry. Taking a quick look overall I see no issues with this series. Overall it seems straight forward. On the nit side I expect saying const struct path * in the functions that now take a struct path would be useful. I suspect it would also be useful to say const struct path *path; path = >f_path; In the one part of the code where you do that. Instead of copying the path out of the struct file. Overall I expect that will keep down bugs at no reduction in usability. Just a statement that the struct path won't change when it is passed to various functions. Eric