Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On 2023-08-26 11:40, Martin Wege via Cygwin wrote: On Sat, Aug 26, 2023 at 4:16 PM Corinna Vinschen via Cygwin wrote: On Aug 26 13:26, Cedric Blancher via Cygwin wrote: On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin wrote: On Aug 25 14:18, Corinna Vinschen via Cygwin wrote: On Aug 23 01:05, Roland Mainz via Cygwin wrote: 2. Check whether the filesystem for the fifos path is NFS (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with |ENOSYS| (not implemented) I like the idea. EPERM, btw. See https://man7.org/linux/man-pages/man2/mknod.2.html I am vehemently opposed to that idea. That will IMHO not only be the wrong solution, it will break actual things like autoconf on NFS - they create fifos for testing, but do not use them. If you just make this return EPERM, then you break stuff we have to rely on. https://cygwin.com/acronyms/#STC, please.# Real world failure for Cygwin on NFS: wget 'http://deb.debian.org/debian/pool/main/k/ksh93u+m/ksh93u+m_1.0.4.orig.tar.gz cd ksh-1.0.4 $PWD/bin/package make This fails with Cygwin 3.4.8-1: cat buildlog.log package: update /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/bin/execrate package: initialize the /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64 view package: update /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/probe/C/make/probe package: update /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/bin/mamake package: make with mamake package: make output captured in /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out mkfifo: cannot set permissions of '/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out.fifo': No such file or directory package: make done at Fri Aug 26 19:24:26 CEST 2023 in /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64 If you need more examples just let me now, we have MUCH more of that... No - the DLL developers need fewer examples, just *ONE* Simple Test Case, boiled down to the essentials, with notes on preconditions, if you can not recreate the required conditions in a program or script. Alternatively, Cygwin strace output with notes from a failing program, or a gdb script and log of the failing code with enough calling context and notes. Please read the problem reporting guidelines linked at the bottom of all posts and follow them. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Thu, Aug 24, 2023 at 6:45 PM Martin Wege wrote:
>
> On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
> wrote:
> >
> > On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> > via Cygwin wrote:
> > > > FIFOs which don't make *any* sense
> > > > ... FWIW, a remote NFS fileystem.
> > >
> > > I got an impression that the OP is trying to deploy something (maybe the
> > > entire Cygwin) onto an
> > > NFS share. So the named FIFO "file" is also created in there.
> >
> > I agree with that impression. This is basically what large sites
> > (universities etc) do with UNIX and Linux: The machines mount an
> > user's ${HOMR} directory via automounter over NFS, and users are
> > discouraged (e.g. grumpy admin visiting you in person, blocking all
> > escape routes... =:-) ) from using the machine's local filesystems (in
> > Cygwin's case that includes "C:"!).
> >
> > In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> > /usr/bin/mkfifo in their home directory, and don't expect that it does
> > not work.
> >
> > But that is what happens on Cygwin 3.4.8 right now, if someone tries
> > to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> > CITI NFSv4 clients):
> > |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
> >
> > Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> > snip
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ uname -a
> > CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> > x86_64 Cygwin
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ mount
> > C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> > C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> > C:/cygwin64 on / type ntfs (binary,auto)
> > C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> > H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 1
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> > cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat -n cygwin_mkfifo_on_nfs.c
> > 1 #include
> > 2 #include
> > 3 #include
> > 4 #include
> > 5 #include
> > 6 #include
> > 7
> > 8 int main(int ac, char *av[])
> > 9 {
> > 10 (void)puts("# start");
> > 11
> > 12 if
> > (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
> > 13 perror("mkfifo failed");
> > 14 (void)puts("# done.");
> > 15 return EXIT_SUCCESS;
> > 16 }
> > 17
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ gcc -g cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ./a
> > # start
> > # done.
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 68
> > -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> > cygwin_mkfifo_on_nfs.c
> > lrwxrwxrwx 1 Unix_User+0 Unix_Group+011 Aug 23 00:12 myfifo.fifo
> > -> ':\0:c4:1000'
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat > -bash: myfifo.fifo: No such file or directory
> > snip
> >
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
> >
> > Better algorithm for [1] might be to check whether the inode is a
> > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > assuming this is more performant.
>
> We would like to see option 1 implemented, maybe as an option in the
> CYGWIN environment variable
> (CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)
One problem I see is how we can set machine-wide defaults, as the
CYGWIN variable is only inherited from process to process, and
/usr/bin/env - myprog will start myprog with an empty environment.
So if there is no way to set machine-wide defaults, I would prefer to
have the nfs_emulate_dev_special_files_as_symlink enabled by default.
Thanks,
Martin
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Sat, Aug 26, 2023 at 4:16 PM Corinna Vinschen via Cygwin wrote: > > On Aug 26 13:26, Cedric Blancher via Cygwin wrote: > > On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin > > wrote: > > > > > > On Aug 25 14:18, Corinna Vinschen via Cygwin wrote: > > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > > > 2. Check whether the filesystem for the fifos path is NFS > > > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > > > > |ENOSYS| (not implemented) > > > > > > > > I like the idea. > > > > > > EPERM, btw. See https://man7.org/linux/man-pages/man2/mknod.2.html > > > > > > > I am vehemently opposed to that idea. That will IMHO not only be the > > wrong solution, it will break actual things like autoconf on > > NFS - they create fifos for testing, but do not use them. If you just > > make this return EPERM, then you break stuff we have to rely on. > > https://cygwin.com/acronyms/#STC, please.# Real world failure for Cygwin on NFS: wget 'http://deb.debian.org/debian/pool/main/k/ksh93u+m/ksh93u+m_1.0.4.orig.tar.gz cd ksh-1.0.4 $PWD/bin/package make This fails with Cygwin 3.4.8-1: cat buildlog.log package: update /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/bin/execrate package: initialize the /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64 view package: update /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/probe/C/make/probe package: update /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/bin/mamake package: make with mamake package: make output captured in /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out mkfifo: cannot set permissions of '/cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64/lib/package/gen/make.out.fifo': No such file or directory package: make done at Fri Aug 26 19:24:26 CEST 2023 in /cygdrive/z/arbeitdfn/windows_ksh/build_nfsv3/ksh-1.0.4/arch/cygwin.i386-64 If you need more examples just let me now, we have MUCH more of that... Thanks, Martin -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
RE: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
> You don't seem to understand the problem. I think I do, and that aligns with your explanation how Cygwin machinery works to fake the FIFOs. > If I can recognize a file as FIFO, I can use it as FIFO, regardless if it's a > native FIFO or a Cygwin FIFO. That's exactly what I meant! > Show me how to create a FIFO on NTFS or FAT by OS calls only, please. > Again, there is no API to create native FIFOs on a remote NFS filesystem. And because of the above, Cygwin can keep creating FIFOs as before, on any filesystem. Anton Lavrentiev Contractor NIH/NLM/NCBI -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Aug 25 23:21, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin wrote: > > it is not possible to diffentiate between Cygwin > > FIFOs and real FIFOs created from the remote side in `ls -l' > > output. > > Why would that be necessary? If it's a FIFO, it can be used as a > FIFO, regardless where and how it was created.. You don't seem to understand the problem. A Cygwin FIFO is not a real FIFO. We're running on Windows, right? Show me how to create a FIFO on NTFS or FAT by OS calls only, please. Cygwin's way of creating FIFOs is to create a Cygwin syumlink with special content which in turn is faked to be a FIFO when running Cygwin's stat(2) call. Usually that's a Windows shortcut (one of the .lnk thingies), but on NFS we create a real symlink with weird content *accidentally*, because that's how we create symlinks on NFS, and nobody ever cared to that anyway. Again, there is no API to create native FIFOs on a remote NFS filesystem. But, ultimately, you're still right. If I can recognize a file as FIFO, I can use it as FIFO, regardless if it's a native FIFO or a Cygwin FIFO. This just isn't done yet. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Aug 26 13:26, Cedric Blancher via Cygwin wrote: > On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin > wrote: > > > > On Aug 25 14:18, Corinna Vinschen via Cygwin wrote: > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > > 2. Check whether the filesystem for the fifos path is NFS > > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > > > |ENOSYS| (not implemented) > > > > > > I like the idea. > > > > EPERM, btw. See https://man7.org/linux/man-pages/man2/mknod.2.html > > > > I am vehemently opposed to that idea. That will IMHO not only be the > wrong solution, it will break actual things like autoconf on > NFS - they create fifos for testing, but do not use them. If you just > make this return EPERM, then you break stuff we have to rely on. https://cygwin.com/acronyms/#STC, please. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Fri, 25 Aug 2023 at 17:15, Roland Mainz via Cygwin wrote: > > On Fri, Aug 25, 2023 at 2:18 PM Corinna Vinschen via Cygwin > wrote: > > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO, > > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'". > > > > > > AFAIK there are (at least) these two options to fix the problems: > > > 1. Check whether the filesystem for the fifos path is NFS > > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it > > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created > > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|, > > > |open()| etc. should treat this inode as FIFO. > > > > The downside is that it is not possible to diffentiate between Cygwin > > FIFOs and real FIFOs created from the remote side in `ls -l' > > output. Note that Cygwin returns the NFS stat info verbatim, so > > a real FIFO is returned as a real FIFO: > > > > linux$ mkfifo bar > > cygwin$ ls -l bar > > prw-r--r-- 1 corinna vinschen 0 Aug 25 13:58 bar > > I know. > > > The idea was always to use NFS as exchange medium, but not as > > installation medium for the entire distro or to keep Cygwin home > > dirs on NFS. There were times where NFS was pretty unstable. > > I used NFS for quite some time to build Cygwin packages, but at one > > point I got trouble (performance problems with multiple concurrent > > processes accessing an NFS share, build errors out of the blue), > > so I switched to Samba shares, albeit grudgingly. I'm not yet > > sure if the problems are fixed. At least a recent OpenSSH build > > ran through without problems... > > I think most issues have been fixed for the Microsoft NFSv3 client, > and for the CITI NFSv4.1 client (technically it's called > "ms-nfsv41-client") the situation is even better since it's > opensource, and we can fix problems even faster there. > From what I see the ms-nfsv41-client is stable enough for daily > routine usage, and I know that other institutions like DFG and CERN > use it for daily work too. The only nasty part is the damn lack of > documentation, and that there are no signed binaries, so any kernel > with UEFI/SecureBook cannot use them. > > > Anyway. How would you like to make sure that a Cygwin application > > can differ between real FIFOs and Cygwin FIFOs? > > For now I can provide a migration script, and in the medium term we > should get Microsoft to provide some kind of |mknod()| API. see below. > > > > 2. Check whether the filesystem for the fifos path is NFS > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > > |ENOSYS| (not implemented) > > > > I like the idea. > > See below. > > > > Better algorithm for [1] might be to check whether the inode is a > > > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|, > > > assuming this is more performant. > > > > Even better would be if we could just create and use real FIFOs > > on NFS(*). But while NtQueryEaFile can be used to fetch real > > NFS file info, and while NtCreateFile can be used to create real > > synmlinks via NFS, I don't see an interface resembling mknod/mkfifo. > > Looking at the ms-nfs41-client source code, there is no API for that *YET*. > > So my plan would be like this: > 1. Cygwin: implements the proposed devnodes-as-symlink emulation code, > if the env var CYGWIN has "nfs_emulate_dev_special_files_as_symlink" > set > 2. Cygwin: By default |mkfifo()| will fail with |ENOSYS| (not |EPERM|, > as we intend to fix the issue!) on NFS filesystems, unless > CYGWIN=nfs_emulate_dev_special_files_as_symlink I think the emulation should be on by default, or you break things. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin wrote: > > On Aug 25 14:18, Corinna Vinschen via Cygwin wrote: > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > 2. Check whether the filesystem for the fifos path is NFS > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > > |ENOSYS| (not implemented) > > > > I like the idea. > > EPERM, btw. See https://man7.org/linux/man-pages/man2/mknod.2.html > I am vehemently opposed to that idea. That will IMHO not only be the wrong solution, it will break actual things like autoconf on NFS - they create fifos for testing, but do not use them. If you just make this return EPERM, then you break stuff we have to rely on. mkfifo should just work with Cygwin on NFS. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
RE: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
> it is not possible to diffentiate between Cygwin > FIFOs and real FIFOs created from the remote side in `ls -l' > output. Why would that be necessary? If it's a FIFO, it can be used as a FIFO, regardless where and how it was created.. Anton Lavrentiev Contractor NIH/NLM/NCBI -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Fri, Aug 25, 2023 at 2:18 PM Corinna Vinschen via Cygwin wrote: > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO, > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'". > > > > AFAIK there are (at least) these two options to fix the problems: > > 1. Check whether the filesystem for the fifos path is NFS > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|, > > |open()| etc. should treat this inode as FIFO. > > The downside is that it is not possible to diffentiate between Cygwin > FIFOs and real FIFOs created from the remote side in `ls -l' > output. Note that Cygwin returns the NFS stat info verbatim, so > a real FIFO is returned as a real FIFO: > > linux$ mkfifo bar > cygwin$ ls -l bar > prw-r--r-- 1 corinna vinschen 0 Aug 25 13:58 bar I know. > The idea was always to use NFS as exchange medium, but not as > installation medium for the entire distro or to keep Cygwin home > dirs on NFS. There were times where NFS was pretty unstable. > I used NFS for quite some time to build Cygwin packages, but at one > point I got trouble (performance problems with multiple concurrent > processes accessing an NFS share, build errors out of the blue), > so I switched to Samba shares, albeit grudgingly. I'm not yet > sure if the problems are fixed. At least a recent OpenSSH build > ran through without problems... I think most issues have been fixed for the Microsoft NFSv3 client, and for the CITI NFSv4.1 client (technically it's called "ms-nfsv41-client") the situation is even better since it's opensource, and we can fix problems even faster there. From what I see the ms-nfsv41-client is stable enough for daily routine usage, and I know that other institutions like DFG and CERN use it for daily work too. The only nasty part is the damn lack of documentation, and that there are no signed binaries, so any kernel with UEFI/SecureBook cannot use them. > Anyway. How would you like to make sure that a Cygwin application > can differ between real FIFOs and Cygwin FIFOs? For now I can provide a migration script, and in the medium term we should get Microsoft to provide some kind of |mknod()| API. see below. > > 2. Check whether the filesystem for the fifos path is NFS > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > |ENOSYS| (not implemented) > > I like the idea. See below. > > Better algorithm for [1] might be to check whether the inode is a > > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|, > > assuming this is more performant. > > Even better would be if we could just create and use real FIFOs > on NFS(*). But while NtQueryEaFile can be used to fetch real > NFS file info, and while NtCreateFile can be used to create real > synmlinks via NFS, I don't see an interface resembling mknod/mkfifo. Looking at the ms-nfs41-client source code, there is no API for that *YET*. So my plan would be like this: 1. Cygwin: implements the proposed devnodes-as-symlink emulation code, if the env var CYGWIN has "nfs_emulate_dev_special_files_as_symlink" set 2. Cygwin: By default |mkfifo()| will fail with |ENOSYS| (not |EPERM|, as we intend to fix the issue!) on NFS filesystems, unless CYGWIN=nfs_emulate_dev_special_files_as_symlink 3. (CITI) ms-nfsv41-client: adds a private API to do a |mknod()| 4. Cygwin: Implements the new API added in [3] as *prototype*, so |mknod()| will properly work in a portable manner 5. We ask Microsoft to review the private API created in [3], and ask them to implement it for the Microsoft NFSv3 client too (because we have a precedent with [4]!) Optional: 6. Cygwin: Like [1] add "afs_emulate_dev_special_files_as_symlink" to do the same for AFS 7. Cygwin: Like [2] |mkfifo()| should fail on AFS, unless [6] ... Issues: - [6] and [7] should be decided by the owner of the Microsoft AFS filesystem driver. - Should the new Windows NFS mknod-API be only available for accounts with administrator rights/permissions ? Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.ma...@nrubsig.org \__\/\/__/ MPEG specialist, C&&& programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Aug 25 14:18, Corinna Vinschen via Cygwin wrote: > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > 2. Check whether the filesystem for the fifos path is NFS > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > |ENOSYS| (not implemented) > > I like the idea. EPERM, btw. See https://man7.org/linux/man-pages/man2/mknod.2.html Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Aug 23 01:05, Roland Mainz via Cygwin wrote: > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO, > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'". > > AFAIK there are (at least) these two options to fix the problems: > 1. Check whether the filesystem for the fifos path is NFS > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|, > |open()| etc. should treat this inode as FIFO. The downside is that it is not possible to diffentiate between Cygwin FIFOs and real FIFOs created from the remote side in `ls -l' output. Note that Cygwin returns the NFS stat info verbatim, so a real FIFO is returned as a real FIFO: linux$ mkfifo bar cygwin$ ls -l bar prw-r--r-- 1 corinna vinschen 0 Aug 25 13:58 bar The idea was always to use NFS as exchange medium, but not as installation medium for the entire distro or to keep Cygwin home dirs on NFS. There were times where NFS was pretty unstable. I used NFS for quite some time to build Cygwin packages, but at one point I got trouble (performance problems with multiple concurrent processes accessing an NFS share, build errors out of the blue), so I switched to Samba shares, albeit grudgingly. I'm not yet sure if the problems are fixed. At least a recent OpenSSH build ran through without problems... Anyway. How would you like to make sure that a Cygwin application can differ between real FIFOs and Cygwin FIFOs? > 2. Check whether the filesystem for the fifos path is NFS > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > |ENOSYS| (not implemented) I like the idea. > Better algorithm for [1] might be to check whether the inode is a > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|, > assuming this is more performant. Even better would be if we could just create and use real FIFOs on NFS(*). But while NtQueryEaFile can be used to fetch real NFS file info, and while NtCreateFile can be used to create real synmlinks via NFS, I don't see an interface resembling mknod/mkfifo. (*) Not using them in the sense that Cygwin can actually communicate with the real remote FIFO, but in the sense that the fake is good enough so that our fhandler_fifo class can actually utilize the file just like FIFOs on NTFS. Corinna -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Thu, 24 Aug 2023 at 18:45, Martin Wege via Cygwin wrote:
>
> On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
> wrote:
> >
> > On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> > via Cygwin wrote:
> > > > FIFOs which don't make *any* sense
> > > > ... FWIW, a remote NFS fileystem.
> > >
> > > I got an impression that the OP is trying to deploy something (maybe the
> > > entire Cygwin) onto an
> > > NFS share. So the named FIFO "file" is also created in there.
> >
> > I agree with that impression. This is basically what large sites
> > (universities etc) do with UNIX and Linux: The machines mount an
> > user's ${HOMR} directory via automounter over NFS, and users are
> > discouraged (e.g. grumpy admin visiting you in person, blocking all
> > escape routes... =:-) ) from using the machine's local filesystems (in
> > Cygwin's case that includes "C:"!).
> >
> > In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> > /usr/bin/mkfifo in their home directory, and don't expect that it does
> > not work.
> >
> > But that is what happens on Cygwin 3.4.8 right now, if someone tries
> > to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> > CITI NFSv4 clients):
> > |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
> >
> > Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> > snip
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ uname -a
> > CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> > x86_64 Cygwin
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ mount
> > C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> > C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> > C:/cygwin64 on / type ntfs (binary,auto)
> > C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> > H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 1
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> > cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat -n cygwin_mkfifo_on_nfs.c
> > 1 #include
> > 2 #include
> > 3 #include
> > 4 #include
> > 5 #include
> > 6 #include
> > 7
> > 8 int main(int ac, char *av[])
> > 9 {
> > 10 (void)puts("# start");
> > 11
> > 12 if
> > (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
> > 13 perror("mkfifo failed");
> > 14 (void)puts("# done.");
> > 15 return EXIT_SUCCESS;
> > 16 }
> > 17
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ gcc -g cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ./a
> > # start
> > # done.
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 68
> > -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> > cygwin_mkfifo_on_nfs.c
> > lrwxrwxrwx 1 Unix_User+0 Unix_Group+011 Aug 23 00:12 myfifo.fifo
> > -> ':\0:c4:1000'
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat > -bash: myfifo.fifo: No such file or directory
> > snip
> >
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
> >
> > Better algorithm for [1] might be to check whether the inode is a
> > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > assuming this is more performant.
>
> We would like to see option 1 implemented,
+1
> maybe as an option in the
> CYGWIN environment variable
> (CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)
+1
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
wrote:
>
> On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> via Cygwin wrote:
> > > FIFOs which don't make *any* sense
> > > ... FWIW, a remote NFS fileystem.
> >
> > I got an impression that the OP is trying to deploy something (maybe the
> > entire Cygwin) onto an
> > NFS share. So the named FIFO "file" is also created in there.
>
> I agree with that impression. This is basically what large sites
> (universities etc) do with UNIX and Linux: The machines mount an
> user's ${HOMR} directory via automounter over NFS, and users are
> discouraged (e.g. grumpy admin visiting you in person, blocking all
> escape routes... =:-) ) from using the machine's local filesystems (in
> Cygwin's case that includes "C:"!).
>
> In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> /usr/bin/mkfifo in their home directory, and don't expect that it does
> not work.
>
> But that is what happens on Cygwin 3.4.8 right now, if someone tries
> to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> CITI NFSv4 clients):
> |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
>
> Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> snip
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ uname -a
> CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> x86_64 Cygwin
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ mount
> C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> C:/cygwin64 on / type ntfs (binary,auto)
> C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ ls -l
> total 1
> -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58 cygwin_mkfifo_on_nfs.c
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ cat -n cygwin_mkfifo_on_nfs.c
> 1 #include
> 2 #include
> 3 #include
> 4 #include
> 5 #include
> 6 #include
> 7
> 8 int main(int ac, char *av[])
> 9 {
> 10 (void)puts("# start");
> 11
> 12 if
> (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
> 13 perror("mkfifo failed");
> 14 (void)puts("# done.");
> 15 return EXIT_SUCCESS;
> 16 }
> 17
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ gcc -g cygwin_mkfifo_on_nfs.c
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ ./a
> # start
> # done.
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ ls -l
> total 68
> -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> cygwin_mkfifo_on_nfs.c
> lrwxrwxrwx 1 Unix_User+0 Unix_Group+011 Aug 23 00:12 myfifo.fifo
> -> ':\0:c4:1000'
>
> roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> $ cat -bash: myfifo.fifo: No such file or directory
> snip
>
> Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
>
> AFAIK there are (at least) these two options to fix the problems:
> 1. Check whether the filesystem for the fifos path is NFS
> (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> |open()| etc. should treat this inode as FIFO.
> 2. Check whether the filesystem for the fifos path is NFS
> (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> |ENOSYS| (not implemented)
>
> Better algorithm for [1] might be to check whether the inode is a
> symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> assuming this is more performant.
We would like to see option 1 implemented, maybe as an option in the
CYGWIN environment variable
(CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)
Thanks,
Martin
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
