Re: [autofs] Returning issues.
On 06/08/2011 06:55 AM, Ian Kent wrote: There's a lot of information her, which probably isn't specific to the issue being discussed so snip ... Ah very nice to know the project I'm working on for a long time now is being appreciated I do not appreciate it when one is just snipping the text I've written about my project, to inform others about what the project is about. Don't know, can't see anything relevant in the kernel source. Maybe you should look harder: http://www.h-online.com/open/features/Kernel-Log-Coming-in-2-6-39-Part-2-Storage-and-file-systems-1232317.html?page=2 and http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=1abf0c718f15a56a0a435588d1b104c7a37dc9bd Stef ___ autofs mailing list autofs@linux.kernel.org http://linux.kernel.org/mailman/listinfo/autofs
Re: [autofs] Returning issues.
On 06/07/2011 07:23 AM, Ian Kent wrote: On Sun, 2011-06-05 at 19:01 +0200, Stef Bon wrote: hi, I see that issues that automounter doing unnecessary mounting like: http://linux.kernel.org/pipermail/autofs/2011-May/006568.html I think you misunderstood the report here. Neither of the points below were related to it. and here http://linux.kernel.org/pipermail/autofs/2011-April/006542.html some more documentation here would prevent users asking the same questions. This behaviour is mostly caused by: A. ls is a alias (in de shell) to ls --color=auto which will follow targets, and this makes the automounter mount. This mostly isn't a problem these days as most user space utilities behave better with automount these days. We will need to wait and see how the user space utilities behave with the new vfs automount kernel infrastructure and work out what needs to be done as we go. I hope you're right. How do apps know a directory is a autofs managed mountpoint? B. extended attributes called by the environment, or an app. I see here a sollution is proposed to fix this in coreutils and the acl package. Hmm, is this right? Yes, this was related to inconsistent use of system calls within the user space utility, an lgetxattr(2) call was being made instead of a getxattr(2) call (inconsistent with respect to previous calls in the same utility code path), which was causing a mount to occur on the follow, as required by the lgetxattr(2) call. So, in this case, it needed to be fixed in the utility. You're mixing things here I think. A lgetxattr will normally not follow a symlink, as the call getxattr does. Isn't it the other way around? (compare stat and lstat: lstat looks at the symlink self, stat follows the symlink and takes the target) Anyhow, when a userspace utility does something like that, it's a serious error. Utilities should do exactly what is requested, and never anything else. | autofs directories don't have extended attributes. You mean only the autofs managed mount points? The contents of the share can have Xattr right? Can't an autofs managed directory have Xattr?? That does not sound right. No, the autofs fs doesn't support extended attributes. Do you really think we need extended attributes? If you do then a patch, which includes some reasoning of why we need them, would be welcomed. No, it suprise me only, since the contents of an autofs managed mountpoint can have Xatttr (see above) and I consider the autofs managed mountpoint belonging to the mounted share. Maybe simular like the stat call, where the trailing slash makes the difference, a trailing slash to the end of the path when doing getxattr/setxattr is doing a call to the . dir on the remote share, and without it it's going to the autofs managed dir on the localhost, resulting in a EOPNOTSUPP as expected. In my construction I'm blocking the xattr calls to autofs managed dirs, unless it's already mounted, and I consider this as an ugly hack. | How? This is possible in the (special) construction I'm working on, fuse-workspace and mount.md5key. In short, it's like GoboLinux, providing a special environment to the user. GoboLinux needed a special kernel patch to achieve this, I'm using a FUSE fs to do the same. The environment looks like: /Computer /Home /Internet Services /Mounts /Network /Shared /System This the the root of the filesystem. Quite different he?!? Now you may think like what the hell you're doing to the Linux system, but the normal directories like /bin, /etc, /lib, etcetera are present but hidden, the readdir - part of the FUSE fs - call has the ability to hide entries when configured. You can do a stat on /bin, and enter it normally . Well that's required to make the system run. First step is to mount the FUSE fs at a certain directory: /var/lib/workspace/$USER/bind Then create the root system directory: mkdir /var/lib/workspace/$USER/bind/bin then make it hidden: setfattr --name=system.workspace_entry_hide --no-dereference --value=1 -- /var/lib/workspace/bind/$USER/bind/bin This command changes the value of the Xattr (system.workspace_entry_hide) of the entry managed by the FUSE fs, as the FUSE fs has a administration of entries and inodes of it's own, and has it's own set of Xattr as you can see. Now the entry will not show when doing a readdir call anymore. But it does exist, and can be used to mount: mount --bind /bin /var/lib/workspace/$USER/bind/bin Every syscall to something in this directory will not go through the FUSE fs, but directly will lead to /bin, which is a very good thing, the extra layer will slow things down. The VFS takes care of that. Now do this for every root system map, and you have a complete directory you can chroot to, where you can do things with the FUSE fs, and not have any loss of performance. Now chroot to this environment using pam_chroot when the session begins, and you'r done. It's possible to
Re: [autofs] Returning issues.
On Tue, 2011-06-07 at 11:15 +0200, Stef Bon wrote: On 06/07/2011 07:23 AM, Ian Kent wrote: On Sun, 2011-06-05 at 19:01 +0200, Stef Bon wrote: hi, I see that issues that automounter doing unnecessary mounting like: http://linux.kernel.org/pipermail/autofs/2011-May/006568.html I think you misunderstood the report here. Neither of the points below were related to it. and here http://linux.kernel.org/pipermail/autofs/2011-April/006542.html some more documentation here would prevent users asking the same questions. This behaviour is mostly caused by: A. ls is a alias (in de shell) to ls --color=auto which will follow targets, and this makes the automounter mount. This mostly isn't a problem these days as most user space utilities behave better with automount these days. We will need to wait and see how the user space utilities behave with the new vfs automount kernel infrastructure and work out what needs to be done as we go. I hope you're right. How do apps know a directory is a autofs managed mountpoint? B. extended attributes called by the environment, or an app. I see here a sollution is proposed to fix this in coreutils and the acl package. Hmm, is this right? Yes, this was related to inconsistent use of system calls within the user space utility, an lgetxattr(2) call was being made instead of a getxattr(2) call (inconsistent with respect to previous calls in the same utility code path), which was causing a mount to occur on the follow, as required by the lgetxattr(2) call. So, in this case, it needed to be fixed in the utility. You're mixing things here I think. A lgetxattr will normally not follow a symlink, as the call getxattr does. Isn't it the other way around? (compare stat and lstat: lstat looks at the symlink self, stat follows the symlink and takes the target) Oh ... of course you are right and I think I'll need to re-visit the report for case B. Don't forget that we are talking about directories that have follow link semantics and that is the source of the semantic behavior. Also we are talking about automount directories themselves not the thing that may be mounted on top of them. Once mounted upon, the autofs directory should be followed when looking up a path unless it is in the process of expiring or has an in progress mount occurring. Lets try that again with the test in the kernel and referring to your problem with stat in the previous mail. if (!(flags LOOKUP_FOLLOW) !(flags (LOOKUP_CONTINUE | LOOKUP_DIRECTORY | LOOKUP_OPEN | LOOKUP_CREATE))) return -EISDIR; If we aren't using an l variant (such as stat(2)) LOOKUP_FOLLOW will be set so this test won't prevent a mount from being attempted. The EISDIR is a little misleading. It is used internally by the kernel automount code to say don't automount just use this directory. The comment: /* We want to mount if someone is trying to open/create a file of any * type under the mountpoint, wants to traverse through the mountpoint * or wants to open the mounted directory. snip ... * We don't want to mount if someone's just doing a stat and they've * set AT_SYMLINK_NOFOLLOW - unless they're stat'ing a directory and * appended a '/' to the name. essentially says, don't mount for l variants of system calls that are trying to get information unless a / is appended to the end of the path. That sounds like it is the opposite to what you observed and the opposite of what used to happen. Previously the autofs module did not distinguish between LOOKUP_FOLLOW being set or not and would always not perform a mount. So, once again we have the age old problem of what to do to prevent mount storms without compromising the semantics of system calls. But we don't always see the mount storms occurring nowadays so a lot has changed in user space, which is good. I did miss this during development though, *sigh*. Note that LOOKUP_CONTINUE is set if the path component is not the last component in the path so a mount will always be attempted in that case. Anyhow, when a userspace utility does something like that, it's a serious error. Utilities should do exactly what is requested, and never anything else. | autofs directories don't have extended attributes. You mean only the autofs managed mount points? The contents of the share can have Xattr right? Sure, yes, autofs must get out of the road once it has a mount on it. Can't an autofs managed directory have Xattr?? That does not sound right. No, the autofs fs doesn't support extended attributes. Do you really think we need extended attributes? If you do then a patch, which includes some reasoning of why we need them, would be welcomed. No, it suprise me only, since the contents of an autofs managed mountpoint can have Xatttr (see above) and I consider
Re: [autofs] Returning issues.
On Sun, 2011-06-05 at 19:01 +0200, Stef Bon wrote: hi, I see that issues that automounter doing unnecessary mounting like: http://linux.kernel.org/pipermail/autofs/2011-May/006568.html I think you misunderstood the report here. Neither of the points below were related to it. and here http://linux.kernel.org/pipermail/autofs/2011-April/006542.html some more documentation here would prevent users asking the same questions. This behaviour is mostly caused by: A. ls is a alias (in de shell) to ls --color=auto which will follow targets, and this makes the automounter mount. This mostly isn't a problem these days as most user space utilities behave better with automount these days. We will need to wait and see how the user space utilities behave with the new vfs automount kernel infrastructure and work out what needs to be done as we go. B. extended attributes called by the environment, or an app. I see here a sollution is proposed to fix this in coreutils and the acl package. Hmm, is this right? Yes, this was related to inconsistent use of system calls within the user space utility, an lgetxattr(2) call was being made instead of a getxattr(2) call (inconsistent with respect to previous calls in the same utility code path), which was causing a mount to occur on the follow, as required by the lgetxattr(2) call. So, in this case, it needed to be fixed in the utility. We may see more of these over the coming months, we will need to wait and see and work out what we need to do on a case by case basis. If an app wants to get the xattr of an autofs managed directory, what is the sollution proposed here then? autofs directories don't have extended attributes. Can't an autofs managed directory have Xattr?? That does not sound right. No, the autofs fs doesn't support extended attributes. Do you really think we need extended attributes? If you do then a patch, which includes some reasoning of why we need them, would be welcomed. In my construction I'm blocking the xattr calls to autofs managed dirs, unless it's already mounted, and I consider this as an ugly hack. How? Since the autofs fs inode operations do not define a getxattr operation these calls always return -EOPNOTSUPP. So they don't cause a callback to user space. User space should handle the EOPNOTSUPP return since extended attributes may not be available for a file system? OTOH the lgetxattr(2) call requires the kernel follow the mount point (by definition) and so a mount should be attempted and the call made upon the mounted file system if it succeeds. The bottom line is that the kernel has changed quite significantly underneath you, sorry. Although the behavior should be substantially the same it isn't exactly the same and we will need to work out if we need to change anything and what we need to change. Keep in mind that we need to try and adhere to normal system call semantics were we can, if at all possible. Ian ___ autofs mailing list autofs@linux.kernel.org http://linux.kernel.org/mailman/listinfo/autofs
