Fwd: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-05-28 ...
Good evening! For your consideration - we need FEEDBACK, please! New is support running it as service (sc start ms-nfs41-client-service), setgid()/newgrp support (with a new winsg utility to run Windows applications with different primary group), 32bit kernel support, ACL+UNC path (cd //host@port/path1/path2 work in bash and ksh93)+chgrp+nfs:// support, and the NFS server no longer needs the "insecure" export switch. Ced -- Forwarded message - From: Roland Mainz Date: Tue, 28 May 2024 at 19:17 Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-05-28 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem client driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #0cb44281d376cd6aa0e43a402153405b7b32ddd8, git bundle in tarball), for testing and feedback (download URL in "Download" section below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. What is this ? NFSv4.1 client and filesystem driver for Windows 10/11 # 2. Features: - Full NFSv4.1 protocol support - idmapper (mapping usernames and uid/gid values between server and client) - Support for custom ports (NFSv4 defaults to TCP port 2049, this client can use different ports per mount) - Support for nfs://-URL * Why ? nfs://-URLs are crossplatform, portable and Character-Encoding independent descriptions of NFSv4 server resources (exports). - including custom ports and raw IPv6 addresses - nfs://-URL conversion utility (/usr/bin/nfsurlconv) to convert URLs, including non-ASCII/Unicode characters in mount path - Support ssh forwarding, e.g. mounting NFSv4 filesystems via ssh tunnel - Support for long paths (up to 4096 bytes), no Windows MAXPATH limit - Unicode support - UNC paths - IPv6 support in UNC paths - /sbin/nfs_mount prints UNC paths in Win32+Cygwin formats - Cygwin bash+ksh93 support UNC paths, e.g. cd //derfwnb4966@2049/nfs4/bigdisk/mysqldb4/ - IPv6 support - IPv6 address within '[', ']' (will be converted to *.ipv6-literal.net) - Windows ACLs - Win32 C:\Windows\system32\icacls.exe - Cygwin /usr/bin/setfacl+/usr/bin/getfacl - Windows Explorer ACL dialog - SFU/Cygwin support, including: - uid/gid - Cygwin symlinks - Custom primary group support - Supports primary group changes in the calling process/thread (via |SetTokenInformation(..., TokenPrimaryGroup,...)|), e.g. if the calling process/threads switches the primary group in its access token then the NFSv4.1 client will use that group as GID for file creation. - newgrp(1)/sg(1)-style "winsg" utilty to run cmd.exe with different primary group, e.g. $ winsg [-] -g group [-c command | /C command] # - Software compatibility: - Any NFSv4.1 server (Linux, Solaris, Illumos, FreeBSD, nfs4j, ...) - All tools from Cygwin/MinGW - Visual Studio - VMware Workstation (can use VMs hosted on NFSv4.1 filesystem) # 3. Requirements: - Windows 10 (32bit or 64bit) or Windows 11 - Cygwin: - Cygwin versions: - 64bit: >= 3.5.3 (or 3.6.x-devel) - 32bit: >= 3.3.6 - Packages (required): cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less libiconv libiconv2 pax pbzip2 procps-ng sed tar time util-linux wget - Packages (recommended): libnfs (for /usr/bin/nfs-ls) make git gcc-core gcc-g++ clang mingw64-i686-clang mingw64-x86_64-clang dos2unix unzip # 4. Download: $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240528_12h15m_git0cb4428.tar.bz2' $ openssl sha256 "msnfs41client_cygwin_binaries_20240528_12h15m_git0cb4428.tar.bz2" SHA2-256(msnfs41client_cygwin_binaries_20240528_12h15m_git0cb4428.tar.bz2)= e3d7adeef8b28161410bb7095036043aa587190488bf9247a00b881c19dbcc0d # 5. Installation (as "Administrator"): $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240528_12h15m_git0cb4428.tar.bz2 ) $ /sbin/msnfs41client install # 6. Deinstallation: $ (set -o xtrace ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240528_12h15m_git0cb4428.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) # 7. Usage: # Option a) # * Start NFSv4 client daemon as Windows service (requires # "Adminstrator" account): $ sc start ms-nfs41-client-service # * Notes: # - requires "Adminstrator" account, and one nfsd client daemon is # used
Re: Technical reason why 32bit Cygwin cannot be installed on 64bit Windows?
On Sat, 18 May 2024 at 13:22, Roland Mainz via Cygwin wrote: > > On Fri, May 17, 2024 at 9:48 AM Cedric Blancher via Cygwin > wrote: > > Is there a technical reason why 32bit Cygwin cannot be installed on > > 64bit Windows? We like to create a CI build pipeline, and want to > > create binaries for 32bit and 64bit Cygwin on the same machine, but > > setup.exe for 32bir Cygwin refuses to install > > Maybe the Cygwin 32bit setup wants to make sure only people who read > the instructions can use it ? :-) > > This should work from an Admin cmd.exe console. > snip > # Install Cygwin 32bit on Windows 10 32bit with packages required by > "ms-nfs41-client" (Windows NFSv4.1 client) > # get installer from https://www.cygwin.com/setup-x86.exe > setup-x86.exe --allow-unsupported-windows -q --no-verify --site > http://ctm.crouchingtigerhiddenfruitbat.org/pub/cygwin/circa/2022/11/23/063457 > cygwin,cygwin-devel,cygrunsrv,cygutils,cygutils-extra,bash,bzip2,coreutils,getent,gdb,grep,hostname,less,libiconv,libiconv2,pax,pbzip2,procps-ng,sed,tar,time,util-linux,wget,libnfs,make,git,dos2unix > snip That does only install the default packages for me, but not wget or cygrunserv. Is there a setup-x86.exe option just to add packages to an existing installation? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Technical reason why 32bit Cygwin cannot be installed on 64bit Windows?
On Fri, 17 May 2024 at 16:50, Brian Inglis via Cygwin wrote: > > On 2024-05-17 01:48, Cedric Blancher via Cygwin wrote: > > Is there a technical reason why 32bit Cygwin cannot be installed on > > 64bit Windows? We like to create a CI build pipeline, and want to > > create binaries for 32bit and 64bit Cygwin on the same machine, but > > setup.exe for 32bir Cygwin refuses to install > > Practical reason is 32 bit usage < 1% I would agree for commercial, well-funded enterprises. The situation is much different for funding-starved education, i.e. schools and universities, where Win10 32bit is squatting cheap computers in the *millions*. For example the schools in Paris alone have 22000 active Win10 32bit licenses in 2022 (last time this was counted). > and Cygwin is all volunteer, with > professionally and/or personally busy developers lacking time to do more. > You are on your own with 32 bit dropped, Does Cygwin 3.6 still compile on 32bit? > so ask questions on forums like SO. What is SO? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Technical reason why 32bit Cygwin cannot be installed on 64bit Windows?
Good morning! Is there a technical reason why 32bit Cygwin cannot be installed on 64bit Windows? We like to create a CI build pipeline, and want to create binaries for 32bit and 64bit Cygwin on the same machine, but setup.exe for 32bir Cygwin refuses to install Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/sg in Cygwin?
On Mon, 22 Apr 2024 at 01:20, Dan Shelton via Cygwin wrote: > > On Sat, 20 Apr 2024 at 07:39, Brian Inglis via Cygwin > wrote: > > > > On 2024-04-19 17:47, Dan Shelton via Cygwin wrote: > > > On Fri, 23 Feb 2024 at 22:25, Dan Shelton wrote: > > >> Is there a package which provides /usr/bin/sg (execute shell commands > > >> in a different group)? > > > > The POSIX standard command is newgrp - install cygwin-doc to see Cygwin (and > > newlib libc) man pages (and info, html, PDF docs), man-pages-posix to see > > POSIX > > man pages, and man-pages-linux if you want to see recent Linux release man > > pages. > > > > To see possibly relevant commands, run: > > > > $ apropos -s 1,1p group > > chgrp (1)- change group ownership > > chgrp (1p) - change the file group ownership > > chown (1)- change file owner and group > > g3topbm (1) - convert a Group 3 fax file into a PBM image > > groups (1) - print the groups a user is in > > id (1) - print real and effective user and group IDs > > make (1) - GNU Make utility to maintain groups of programs > > make (1p)- maintain, update, and regenerate groups of programs > > mkgroup (1) - Write /etc/group-like output to stdout > > newgrp (1) - change primary group for a command > > newgrp (1p) - change to a new group > > pbmtog3 (1) - convert a PBM image into a Group 3 MH fax file > > newgrp(1) is USELESS. It only opens an interactive shell, but does not > allow the user to execute a non-interactive script with the requested > group like bash -c does. Linux /usr/bin/sg source is in https://github.com/shadow-maint/shadow/blob/master/src/newgrp.c So this is just a packaging issue that whoever does the Cygwin newgrp package has to package /usr/bin/sg too Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/sg in Cygwin?
Good morning! On Sat, 20 Apr 2024 at 07:39, Brian Inglis via Cygwin wrote: > > On 2024-04-19 17:47, Dan Shelton via Cygwin wrote: > > On Fri, 23 Feb 2024 at 22:25, Dan Shelton wrote: > >> Is there a package which provides /usr/bin/sg (execute shell commands > >> in a different group)? > > The POSIX standard command is newgrp - install cygwin-doc to see Cygwin (and > newlib libc) man pages (and info, html, PDF docs), man-pages-posix to see > POSIX > man pages, and man-pages-linux if you want to see recent Linux release man > pages. Debian /usr/bin/sg (setgid analog to /usr/bin/su) comes from the package "login" (https://packages.debian.org/bookworm/login), same package as /usr/bin/newgrp. So maybe it just needs to be packaged by whoever owns the Cygwin package for /usr/bin/newgrp? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin file prefetch, block sizes?
On Wed, 3 Apr 2024 at 10:15, Corinna Vinschen via Cygwin wrote: > > On Apr 3 00:35, Martin Wege via Cygwin wrote: > > On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin > > wrote: > > > > > > On Apr 2 02:04, Martin Wege via Cygwin wrote: > > > > Hello, > > > > > > > > Is there any document which describes how Cygwin and Win32 file > > > > prefetch and readahead work, and which sizes are used (e.g. always > > > > read one full page even if only 16 bytes are requested?)? > > > > > > I'm not aware of any docs, but again, keep in mind that Cygwin is a > > > usersapce DLL. We basically do what Windows does for low-level file > > > access. > > > > > > > Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that > > > > mean the file's block size is really 64k? Is this info per filesystem, > > > > or hardcoded in Cygwin? > > > > > > Hardcoded in Cygwin since 2017, based on a discussion in terms of > > > file access performance, especially when using stdio.h functions: > > > > > > https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c > > > > OUCH. > > > > While I can understand the motivation, FAT32 on multi-GB-devices > > having 64k block size, and Win32 API on Win95/98/ME/Win7 being > > optimized to that insane block size, it is absolutely WRONG with > > today's NTFS and even more so with ReFS. This only works if you stream > > files, but as soon as you are doing random read/writes the performance > > is terrible due to cache thrashing. That could explain the many > > complaints about Cygwin's IO performance. > > The above patch *only* sets stat::st_blksize to 64K. Nothing else > happens! Yes, but applications use that information, and then make wrong choices. > > This usually means that stdio.h functions use this size for their buffer > and readahead. It doesn't affect direct calls to read(2)/write(2) and > fread(3)/fwrite(3) at all! > > > So, what can be done? I'm not a benchmarking guru, so I'd like to > > propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to > > No. > > We have two ways to handle this *iff* there's really a reason to > handle this. > > - Either we just lower PREFERRED_IO_BLKSIZE to 4K or 8K, but that's > kind of bad in terms of pipes, the clipboard, etc. I think the env variable was strictly meant for benchmarking **ONLY**, so someone with good benchmarking expertise can grab the Cygwin 3.6 alpha binaries and does benchmarking with them. > > - So we keep PREFERRED_IO_BLKSIZE at 64K but don't use it for disk > files. Rather, we read this info from the filesystem: > > > https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_fs_sector_size_information @Corinna Vinschen Could you please do me a favour and check whether SMB supports FILE_FS_SECTOR_SIZE_INFORMATION? > > If the filesystem is local and SSINFO_FLAGS_NO_SEEK_PENALTY is set, we > could stick to 64K. That still means that buffers will be very large, and associated memcpy(), read(), write() are large too. Also harmful is that such buffers might be allocated with alloca(), because the original authors didn't anticipate 64k buffer sizes... > > Otherwise the PhysicalBytesPerSectorForPerformance member might be > helpful I guess. Needs checking, of course. > > If this isn't any good, we can still fallback to > FILE_FS_FULL_SIZE_INFORMATION as in fhandler_base::fstatvfs_by_handle, > > https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fhandler/disk_file.cc#n661 You mean when FILE_FS_SECTOR_SIZE_INFORMATION is not supported you use FILE_FS_FULL_SIZE_INFORMATION instead? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin file prefetch, block sizes?
On Wed, 3 Apr 2024 at 03:10, Mark Geisert via Cygwin wrote: > > On 4/2/2024 3:35 PM, Martin Wege via Cygwin wrote: > > On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin > > wrote: > >> > >> On Apr 2 02:04, Martin Wege via Cygwin wrote: > >>> Hello, > >>> > >>> Is there any document which describes how Cygwin and Win32 file > >>> prefetch and readahead work, and which sizes are used (e.g. always > >>> read one full page even if only 16 bytes are requested?)? > >> > >> I'm not aware of any docs, but again, keep in mind that Cygwin is a > >> usersapce DLL. We basically do what Windows does for low-level file > >> access. > >> > >>> Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that > >>> mean the file's block size is really 64k? Is this info per filesystem, > >>> or hardcoded in Cygwin? > >> > >> Hardcoded in Cygwin since 2017, based on a discussion in terms of > >> file access performance, especially when using stdio.h functions: > >> > >>https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c > > > > OUCH. > > > > While I can understand the motivation, FAT32 on multi-GB-devices > > having 64k block size, and Win32 API on Win95/98/ME/Win7 being > > optimized to that insane block size, it is absolutely WRONG with > > today's NTFS and even more so with ReFS. This only works if you stream > > files, but as soon as you are doing random read/writes the performance > > is terrible due to cache thrashing. That could explain the many > > complaints about Cygwin's IO performance. > > No comment. > > > So, what can be done? I'm not a benchmarking guru, so I'd like to > > propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to > > the CYGWIN env variable (marked as "experimental"), so the > > benchmarking guys can do performance testing without recompiling > > everything, get perf results for Cygwin 3.6, and decide what to do for > > Cygwin 3.7. > > That kind of experiment is what folks who can build their own > cygwin1.dll might do. I doubt we'd want to make a run-time global disk > I/O strategy changer available like this, even temporarily. Realistically that would mean that Cygwin will forever be stuck with an insane IO block size. Building Cygwin.dll requires specialised knowledge and TIME, and no manager will waste the time of a performance engineer to produce custom binaries. Cygwin 3.6 is right now in development, so it would be better to add such a knob, so performance engineers can just grab those binaries and do benchmarking with them. BTW: A block size of 64k is CLEARLY harming performance. Have a look at https://www.zabkat.com/blog/buffered-disk-access.htm the sweet spot is somewhere between 16k and 32k, for SMB even below that. 64k is clearly on the backside of the curve, and actively harming performance, except for "linear reads". > > What could make sense is enhancing Cygwin's posix_fadvise() to support > POSIX_FADV_RANDOM getting mapped to Windows' FILE_RANDOM_ACCESS flag. > Something like this is currently done for POSIX_FADV_SEQUENTIAL -> > FILE_SEQUENTIAL_ONLY. These are per-filedescriptor adjustments and due > to Windows limitations would apply to a whole file rather than having > the POSIX behavior of being settable for a byte range within a file. Nope. Because we are talking about a sensible default for all applications, and a block size of 64k is HARMFUL, except on fat32 where the filesystem block size is already 64k for multi gigabyte disks. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin file prefetch, block sizes?
On Wed, 3 Apr 2024 at 00:36, Martin Wege via Cygwin wrote: > > On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin > wrote: > > > > On Apr 2 02:04, Martin Wege via Cygwin wrote: > > > Hello, > > > > > > Is there any document which describes how Cygwin and Win32 file > > > prefetch and readahead work, and which sizes are used (e.g. always > > > read one full page even if only 16 bytes are requested?)? > > > > I'm not aware of any docs, but again, keep in mind that Cygwin is a > > usersapce DLL. We basically do what Windows does for low-level file > > access. > > > > > Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that > > > mean the file's block size is really 64k? Is this info per filesystem, > > > or hardcoded in Cygwin? > > > > Hardcoded in Cygwin since 2017, based on a discussion in terms of > > file access performance, especially when using stdio.h functions: > > > > https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c > > OUCH. > > While I can understand the motivation, FAT32 on multi-GB-devices > having 64k block size, and Win32 API on Win95/98/ME/Win7 being > optimized to that insane block size, it is absolutely WRONG with > today's NTFS and even more so with ReFS. This only works if you stream > files, but as soon as you are doing random read/writes the performance > is terrible due to cache thrashing. That could explain the many > complaints about Cygwin's IO performance. > It can also explain why Cygwin is so slow on SMB filesystems. If it really works on 64k blocks, then this would pull all small files over the wire, even if only a bit gets touched. Thinking more about this, this basically defeats every driver/tcp/ip/ethernet/net/switch optimization like jumbo frames et al. /usr/bin/stat %o (optimum IO block size hint) also returns 65536 on SMB, which is NOT GOOD: This needs to be confirmed, and if this is really true, then it should be fixed for SMB. I need a coffee to think about this... > So, what can be done? I'm not a benchmarking guru, so I'd like to > propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to > the CYGWIN env variable (marked as "experimental"), so the > benchmarking guys can do performance testing without recompiling > everything, get perf results for Cygwin 3.6, and decide what to do for > Cygwin 3.7. I would agree, but I would also clamp the minimum at page size (4096 bytes on x86) and 8M (4x 2M hugepage size) to prevent abuse. Does Cygwin partake in the GSOC programm (Google Summer Of Code)? This would IMO be a high priority item. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Fwd: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-03-28 ...
Good evening! For your consideration - we need FEEDBACK, please! New is ACL+UNC path (cd //host@port/path1/path2 work in bash and ksh93)+chgrp+nfs:// support, and the NFS server no longer needs the "insecure" export switch. Ced -- Forwarded message - From: Roland Mainz Date: Thu, 28 Mar 2024 at 13:20 Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-03-28 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem client driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #4c9528a5e9094cd80906c0e8af9785e1748b0a25, git bundle in tarball), for testing and feedback (download URL in "Download" section below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. What is this ? NFSv4.1 client and filesystem driver for Windows 10/11 # 2. Features: - Full NFSv4.1 protocol support - idmapper (mapping usernames and uid/gid values between server and client) - Support for custom ports (NFSv4 defaults to TCP port 2049, this client can use different ports per mount) - Support for nfs://-URL * Why ? nfs://-URLs are crossplatform, portable and Character-Encoding independent descriptions of NFSv4 server resources (exports). - including custom ports and raw IPv6 addresses - nfs://-URL conversion utility (/usr/bin/nfsurlconv) to convert URLs, including non-ASCII/Unicode characters in mount path - Support ssh forwarding, e.g. mounting NFSv4 filesystems via ssh tunnel - Support for long paths (up to 4096 bytes), no Windows MAXPATH limit - Unicode support - UNC paths - IPv6 support in UNC paths - /sbin/nfs_mount prints UNC paths in Win32+Cygwin formats - Cygwin bash+ksh93 support UNC paths, e.g. cd //derfwnb4966@2049/nfs4/bigdisk/mysqldb4/ - IPv6 support - IPv6 address within '[', ']' (will be converted to *.ipv6-literal.net) - Windows ACLs - Win32 C:\Windows\system32\icacls.exe - Cygwin /usr/bin/setfacl+/usr/bin/getfacl - Windows Explorer ACL dialog - SFU/Cygwin support, including: - uid/gid - Cygwin symlinks - Software compatibility: - Any NFSv4.1 server (Linux, Solaris, Illumos, FreeBSD, nfs4j, ...) - All tools from Cygwin/MinGW - Visual Studio - VMware Workstation (can use VMs hosted on NFSv4.1 filesystem) # 3. Requirements: - Windows 10 or Windows 11 - Cygwin 3.5.1 (or 3.6.x-devel) - Packages (required): cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less libiconv libiconv2 pax pbzip2 procps-ng sed tar time util-linux wget - Packages (recommended): libnfs (for /usr/bin/nfs-ls) make git dos2unix # 4. Download: $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240328_12h28m_git4c9528a.tar.bz2' $ openssl sha256 "msnfs41client_cygwin_binaries_20240328_12h28m_git4c9528a.tar.bz2" SHA2-256(msnfs41client_cygwin_binaries_20240328_12h28m_git4c9528a.tar.bz2)= ff683f954109f344403ce7f54830c36827984a1fc999460aa7054a97ca6ad473 # 5. Installation (as "Administrator"): $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240328_12h28m_git4c9528a.tar.bz2 ) $ /sbin/msnfs41client install # 6. Deinstallation: $ (set -o xtrace ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240328_12h28m_git4c9528a.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) # 7. Usage: # Run the NFSv4 client daemon: # - run this preferably as "Administrator", but this is not a requirement # - requires separate terminal $ /sbin/msnfs41client run_daemon # Mount a filesystem and use it $ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2 Successfully mounted '10.49.20.110@2049' to drive 'N:' $ cd /cygdrive/n/ $ ls -la total 4 drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 . dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 .. drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030 drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2 # Unmount filesystem: $ cd ~ && /sbin/nfs_mount -d N: # OR $ cd ~ $ net use N: /delete # List mounted NFSv4.1 filesystems: $ /sbin/nfs_mount # 8. Notes: - Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4 server works via /lib/msnfs41client/cygwin_idmapper.ksh, which either uses builtin static data, or /usr/bin/getent passwd and /usr/bin/getent group. As getent uses the configured
No Win ACLs for NFS? Re: ACEs and ACLs
On Mon, 18 Mar 2024 at 15:43, Corinna Vinschen via Cygwin wrote: > > On Mar 18 08:30, J. Terry Corbet via Cygwin wrote: > > Thank you for the greatly needed assistance, but the reference to which you > > have pointed me says that noacl will be ignored in the case of ntfs file > > systems. > > No, it doesn't say that. It says > > "The flag is ignored on NFS filesystems." > ^^^ > not NTFS Do ACLs work for NFS in Cygwin, or are they turned off for NFS? I recall the Exceed (now OpenText) docs say that Exceed NFSv4 for Windows supports ACLs, but they are defunct for Cygwin2. Is this true? I'm also asking because the ms-nfs41-client Windows NFSv4.1 driver now has ACL support (like the Exceed NFSv4 driver), but it would be frustrating if Cygwin just turns this off. Or maybe I am misinterpreting this... @Corinna Vinschen? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?
Good morning!
How does cygwin uname -s work, i.e. how does it get the OS version,
revision and build number ('10.0-19045')?
I tried to replicate it via GetVersionEx(), but that is deprecated.
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin 3.5.1: unable to open output file '/cygdrive/t/tmpdir/x-01564d.o': 'Operation not permitted'
On Fri, 8 Mar 2024 at 03:25, Dan Shelton via Cygwin wrote:
>
> Hello!
>
> I've run into a problem with clang on Cygwin 3.5.1 and 3.6. My machine
> does not have much disk space left, so I switched TMPDIR to the
> network drive. But clang then failed, like this:
>
> $ cat x.c
> #include
> int main(int ac, char *av[]) { puts("hello world"); return 0 ; }
> $ mkdir /cygdrive/t/tmpdir
> $ TMPDIR=/cygdrive/t/tmpdir clang x.c
> error: unable to open output file '/cygdrive/t/tmpdir/x-01564d.o':
> 'Operation not permitted'
> 1 error generated.
>
> /cygdrive/t/tmpdir/ is a clean dir.
> /usr/bin/touch /cygdrive/t/tmpdir/dummy works without problems
I can confirm the bug, but only for the clang version linked to
Cygwin. clang version of MinGW does not show that error, so this looks
like a Cygwin bug.
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin 3.5.1: unable to open output file '/cygdrive/t/tmpdir/x-01564d.o': 'Operation not permitted'
On Fri, 8 Mar 2024 at 03:25, Dan Shelton via Cygwin wrote:
>
> Hello!
>
> I've run into a problem with clang on Cygwin 3.5.1 and 3.6. My machine
> does not have much disk space left, so I switched TMPDIR to the
> network drive. But clang then failed, like this:
>
> $ cat x.c
> #include
> int main(int ac, char *av[]) { puts("hello world"); return 0 ; }
> $ mkdir /cygdrive/t/tmpdir
> $ TMPDIR=/cygdrive/t/tmpdir clang x.c
> error: unable to open output file '/cygdrive/t/tmpdir/x-01564d.o':
> 'Operation not permitted'
> 1 error generated.
>
> /cygdrive/t/tmpdir/ is a clean dir.
> /usr/bin/touch /cygdrive/t/tmpdir/dummy works without problems
>
> How can I debug this further?
>
Which "network drive" do you use? SMB, AFS, Microsoft NFSv3,
msnfs41client NFSv4.1, OpenText Exceed NFSv3/NFSv4?
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever
On Mon, 29 Jan 2024 at 10:46, Mark Geisert via Cygwin wrote: > > On 1/28/2024 11:56 PM, Mark Geisert via Cygwin wrote: > > On 1/28/2024 10:44 PM, Cedric Blancher via Cygwin wrote: > >> Good morning! > >> > >> rm -f myfile > >> touch myfile > >> /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5 > > ^ length goes here, before filename > > > > I believe you need to > > chattr +S myfile > > after the 'touch' to make the file sparsifiable. > > > > Also, I'm unsure if '-d' makes sense when initially allocating a sparse > > file. If I leave it off, fallocate does the right thing. > > > > I'm separately investigating the 'takes forever' report. > > It's taking forever because it's in an infinite loop looking for data > followed by a hole, which is not present in a new file being created. > This is due to the '-d' option being specified. fallocate ought to > diagnose this condition rather than looping forever, but here we are. > > It's possible we have an old fallocate from an old util-linux package. > I'll try to investigate this further when I have time. > Any update on this issue? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Convert a file descriptor from Cygwin openat() to Win32 file HANDLE?
On Mon, 4 Mar 2024 at 09:35, Andrey Repin wrote: > > Greetings, Cedric Blancher! > > > How can I convert a file descriptor from Cygwin openat() to Win32 file > > HANDLE? > > In general, you should not attempt to do such thing. > If you have a very specific idea in mind, it would be best to describe, what > you are trying to achieve, so community could provide a more meaningful > suggestion to solve your specific issue. The idea is to add raw SID support for users and groups to /bin/ls, and that only works with access to the raw SIDs Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Regression: Cygwin 3.5.1 freezes when launching several mingw processes in parallel
Same here building gcc with the msnfs41client NFSv4.1 filesystem driver (https://sourceforge.net/p/ms-nfs41-client/mailman/message/58741244/), make -j8 and make -j128 builds now succeed again. Is a Cygwin 3.5.x backport required? Ced On Mon, 4 Mar 2024 at 20:38, Dimitry Andric via Cygwin wrote: > > Same here, did a bunch of make -j8 builds, and with 3.6.0-0.71.gb160b690b6ac > they now complete without any hangs. > > Thanks for the quick fix! > > -Dimitry > > > On 4 Mar 2024, at 16:58, Kate Deplaix via Cygwin wrote: > > > > After testing that version without interruption for the past 3 hours on > > test cases that would've failed before, I can happily say it looks like > > it's fixed! > > > > Thank you so much!! > > > > From: Cygwin on behalf > > of Takashi Yano via Cygwin > > Sent: 04 March 2024 12:06 > > To: cygwin@cygwin.com > > Subject: Re: Regression: Cygwin 3.5.1 freezes when launching several mingw > > processes in parallel > > > > On Mon, 4 Mar 2024 20:00:13 +0900 > > Takashi Yano via Cygwin wrote: > >> On Sun, 3 Mar 2024 13:07:11 +0900 > >> Takashi Yano wrote: > >>> On Sat, 2 Mar 2024 11:49:36 + > >>> Kate Deplaix wrote: > I'm running cygwin on baremetal on an Intel i5-750 (4 cores), with 7GB > of RAM and with an up-to-date Windows 10. > >>> > >>> Thanks for the information. I could reproduce the problem with > >>> Core i5 M 540 + 8GB RAM machine. > >>> > >>> Let consider how to debug. > >> > >> I found the cause. I'll push the patch shortly. > >> Thanks! > > > > Please try: > > cygwin 3.6.0-0.71.gb160b690b6ac (TEST) > > > > -- > > Takashi Yano > > > > -- > > Problem reports: https://cygwin.com/problems.html > > FAQ: https://cygwin.com/faq/ > > Documentation:https://cygwin.com/docs.html > > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > > > -- > > Problem reports: https://cygwin.com/problems.html > > FAQ: https://cygwin.com/faq/ > > Documentation:https://cygwin.com/docs.html > > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation:https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How can I get the group SID for a file in cmd.exe (powershell is not an option)?
On Tue, 27 Feb 2024 at 16:07, Corinna Vinschen via Cygwin
wrote:
>
> On Feb 27 15:41, Cedric Blancher via Cygwin wrote:
> > Good afternoon!
> >
> > How can I get the group SID for a file in cmd.exe (powershell is not an
> > option)?
> > Is it possible to add an option to ls -l to list the user/group SIDs too?
>
> No, but you can use Cygwin tools:
>
> getent -w group $(ls -gn foo | awk '{print $3}') | awk -F: '{print $4}'
I need the exact Windows SID strings, without intermediate
translation. It's for a test suite written in cmd.exe (whoever had
that idea, but I do not have the resources to rewrite it).
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How can I get the group SID for a file in cmd.exe (powershell is not an option)?
On Tue, 27 Feb 2024 at 23:27, Bill Stewart via Cygwin wrote: > > On Tue, Feb 27, 2024 at 7:42 AM Cedric Blancher wrote: > > How can I get the group SID for a file in cmd.exe (powershell is not an > > option)? > > > > Why is PowerShell not an option? It's for a test suite written in cmd.exe (whoever had that idea, but I do not have the resources to rewrite it). Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Convert a file descriptor from Cygwin openat() to Win32 file HANDLE?
Good morning! How can I convert a file descriptor from Cygwin openat() to Win32 file HANDLE? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
How can I get the group SID for a file in cmd.exe (powershell is not an option)?
Good afternoon! How can I get the group SID for a file in cmd.exe (powershell is not an option)? Is it possible to add an option to ls -l to list the user/group SIDs too? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Fwd: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-02-24 ...
For your consideration - we need FEEDBACK, please! Ced -- Forwarded message - From: Roland Mainz Date: Sat, 24 Feb 2024 at 15:19 Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-02-24 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem client driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #5e7d973b97c4ec831b22e468e66289f555318d45, git bundle in tarball), for testing and feedback (download URL in "Download" section below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. What is this ? NFSv4.1 client and filesystem driver for Windows 10/11 # 2. Features: - Full NFSv4.1 protocol support - idmapper (mapping usernames and uid/gid values between server and client) - Support for custom ports (NFSv4 defaults to TCP port 2049, this client can use different ports per mount) - Support for nfs://-URL * Why ? nfs://-URLs are cross platform, portable and Character-Encoding independent descriptions of NFSv4 server resources (exports). - including custom ports and raw IPv6 addresses - nfs://-URL conversion utility (/usr/bin/nfsurlconv) to convert URLs, including non-ASCII/Unicode characters in mount path - Support ssh forwarding, e.g. mounting NFSv4 filesystems via ssh tunnel - Support for long paths (up to 4096 bytes), no Windows MAXPATH limit - Unicode support - UNC paths - IPv6 support in UNC paths - IPv6 support - IPv6 address within '[', ']' (will be converted to *.ipv6-literal.net) - Windows ACLs - SFU/Cygwin support, including: - uid/gid - Cygwin symlinks - Software compatibility: - Any NFSv4.1 server (Linux, Solaris, Illumos, FreeBSD, nfs4j, ...) - All tools from Cygwin/MinGW - Visual Studio - VMware Workstation (can use VMs hosted on NFSv4.1 filesystem) # 3. Requirements: - Windows 10 or Windows 11 - Cygwin 3.5.0 - Packages: cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less libiconv libiconv2 pax pbzip2 procps-ng sed tar time util-linux wget # 4. Download: $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240224_11h39m_git5e7d973.tar.bz2' $ openssl sha256 msnfs41client_cygwin_binaries_20240224_11h39m_git5e7d973.tar.bz2 SHA2-256(msnfs41client_cygwin_binaries_20240224_11h39m_git5e7d973.tar.bz2)= b854115d86f2aad0c305c9690d9361d3c160edb05304def510363a6be85fbeca # 5. Installation (as "Administrator"): $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240224_11h39m_git5e7d973.tar.bz2 ) $ /sbin/msnfs41client install # 6. Deinstallation: $ (set -x ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240224_11h39m_git5e7d973.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) # 7. Usage: # Run the NFSv4 client daemon: # - run this preferably as "Administrator", but this is not a requirement # - requires separate terminal $ /sbin/msnfs41client run_daemon # Mount a filesystem and use it # - requires that NFSv4 server accepts connections from a TCP port # number > 1024, which can be archived on Linux with the "insecure" # export option in /etc/exports, or "resvport" on Solaris/Illumos # (see nfs(5)) $ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2 Successfully mounted '10.49.20.110@2049' to drive 'N:' $ cd /cygdrive/n/ $ ls -la total 4 drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 . dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 .. drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030 drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2 # Unmount filesystem: $ cd ~ && /sbin/nfs_mount -d N: # OR $ cd ~ $ net use N: /delete # List mounted NFSv4.1 filesystems: $ /sbin/nfs_mount # 8. Notes: - Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4 server works via /lib/msnfs41client/cygwin_idmapper.ksh, which either uses builtin static data, or /usr/bin/getent passwd and /usr/bin/getent group. As getent uses the configured name services it should work with LDAP too. This is still work-in-progress, with the goal that both NFSv4 client and server can use different uid/gid numeric values for client and server side. - UNC paths are supported, after successful mounting /sbin/nfs_mount will list the paths in Cygwin UNC format. - SIDs work, users with valid Windows accounts (see Cygwin idmapping
Fwd: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-02-05 ...
For your consideration - we need FEEDBACK, please! As Cygwin 3.5.0 is now officially out it should be much easier to test. Ced -- Forwarded message - From: Roland Mainz Date: Mon, 5 Feb 2024 at 15:41 Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-02-05 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem client driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #27efbb0606616388c7c5d0e656165061d0fc4dd8, git bundle in tarball), for testing and feedback (download URL in "Download" section below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. What is this ? NFSv4.1 client and filesystem driver for Windows 10/11 # 2. Features: - Full NFSv4.1 protocol support - idmapper (mapping usernames and uid/gid values between server and client) - Support for custom ports (NFSv4 defaults to TCP port 2049, this client can use different ports per mount) - Support for nfs://-URL - including custom ports and raw IPv6 addresses - nfs://-URL conversion utility (/usr/bin/nfsurlconv) to convert URLs, including non-ASCII/Unicode characters in mount path - Support ssh forwarding, e.g. mounting NFSv4 filesystems via ssh tunnel - Support for long paths (up to 4096 bytes), no Windows MAXPATH limit - Unicode support - UNC paths - IPv6 support in UNC paths - IPv6 support - IPv6 address within '[', ']' (will be converted to *.ipv6-literal.net) - Windows ACLs - SFU/Cygwin support, including: - uid/gid - Cygwin symlinks - Software compatibility: - Any NFSv4.1 server (Linux, Solaris, Illumos, FreeBSD, nfs4j, ...) - All tools from Cygwin/MinGW - Visual Studio - VMware Workstation (can use VMs hosted on NFSv4.1 filesystem) # 3. Requirements: - Windows 10 or Windows 11 - Cygwin 3.5.0 - Packages: cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less libiconv libiconv2 pax pbzip2 procps-ng sed tar time util-linux wget # 4. Download: $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240205_14h31m_git27efbb0.tar.bz2' $ openssl sha256 msnfs41client_cygwin_binaries_20240205_14h31m_git27efbb0.tar.bz2 SHA2-256(msnfs41client_cygwin_binaries_20240205_14h31m_git27efbb0.tar.bz2)= 0da0d3035c9368c5f89ccc1ab24c155b5938fe34fc24d85f77319d93d52382ea # 5. Installation (as "Administrator"): $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240205_14h31m_git27efbb0.tar.bz2 ) $ /sbin/msnfs41client install # 6. Deinstallation: $ (set -x ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240205_14h31m_git27efbb0.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) # 7. Usage: # Run the NFSv4 client daemon: # - run this preferably as "Administrator", but this is not a requirement # - requires separate terminal $ /sbin/msnfs41client run_daemon # Mount a filesystem and use it # - requires that NFSv4 server accepts connections from a TCP port # number > 1024, which can be archived on Linux with the "insecure" # export option in /etc/exports, or "resvport" on Solaris/Illumos # (see nfs(5)) $ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2 Successfully mounted '10.49.20.110@2049' to drive 'N:' $ cd /cygdrive/n/ $ ls -la total 4 drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 . dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 .. drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030 drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2 # Unmount filesystem: $ cd ~ && /sbin/nfs_mount -d N: # OR $ cd ~ $ net use N: /delete # List mounted NFSv4.1 filesystems: $ /sbin/nfs_mount # 8. Notes: - Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4 server works via /lib/msnfs41client/cygwin_idmapper.ksh, which either uses builtin static data, or /usr/bin/getent passwd and /usr/bin/getent group. As getent uses the configured name services it should work with LDAP too. This is still work-in-progress, with the goal that both NFSv4 client and server can use different uid/gid numeric values for client and server side. - UNC paths are supported, after successful mounting /sbin/nfs_mount will list the paths in Cygwin UNC format. - SIDs work, users with valid Windows accounts (see Cygwin idmapping above get their SIDs, unknown users with valid uid/gid values get
Re: /usr/bin/fallocate -v -d -l 2 myfile takes forever
On Mon, 29 Jan 2024 at 09:33, Brian Inglis via Cygwin wrote: > > On 2024-01-28 23:44, Cedric Blancher via Cygwin wrote: > > rm -f myfile > > touch myfile > > /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5 > > It takes much longer without Cygwin 3.5! > > Your fallocate(1) option -l has no length numeric argument and no -o offset > numeric argument to specify the start of the range. Apologies, I typed this from a screenshot, and made a mistake. The line should be: /usr/bin/fallocate -v -d -l 2 myfile like in the Subject Offset is 0 i this case, because no -o was given Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
/usr/bin/fallocate -v -d -l 2 myfile takes forever
Good morning! rm -f myfile touch myfile /usr/bin/fallocate -v -d -l myfile takes forever with latest Cygwin 3.5 Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Fwd: [Ms-nfs41-client-devel] NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-01-28 ...
For your consideration - we need FEEDBACK, please! Ced -- Forwarded message - From: Roland Mainz Date: Sun, 28 Jan 2024 at 23:54 Subject: [Ms-nfs41-client-devel] NFSv4.1 filesystem client Windows driver binaries for Windows 10/11 for testing, 2024-01-28 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem client driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #38817b806bc146316c193954bfbfdc2209a7b3c6, git bundle in tarball), for testing and feedback (download URL in "Installation" section below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. What is this ? NFSv4.1 filesystem client driver for Windows 10/11 # 2. Features: - Full NFSv4.1 protocol support - idmapper (mapping usernames and uid/gid values between server and client) - Support for custom ports (NFSv4 defaults to TCP port 2049, this client can use different ports per mount) - Support for nfs://-URL, including custom ports and raw IPv6 addresses - Support ssh forwarding, e.g. mounting NFSv4 filesystems via ssh tunnel - Support for long paths (up to 4096 bytes), no Windows MAXPATH limit - Unicode support - UNC paths - IPv6 support in UNC paths - IPv6 support - IPv6 address within '[', ']' (will be converted to *.ipv6-literal.net) - Windows ACLs - SFU/Cygwin support, including: - uid/gid - Cygwin symlinks - Software compatibility: - Any NFSv4.1 server (Linux, Solaris, Illumos, FreeBSD, nfs4j, ...) - All tools from Cygwin/MinGW - Visual Studio - VMware Workstation (can use VMs hosted on NFSv4.1 filesystem) # 3. Requirements: - Windows 10 or Windows 11 - Cygwin 3.5.0 - Packages: cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less pax pbzip2 procps-ng sed tar time util-linux wget # 4. Installation (as "Administrator"): $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240128_17h08m_git38817b8.tar.bz2' $ openssl sha256 msnfs41client_cygwin_binaries_20240128_17h08m_git38817b8.tar.bz2 SHA2-256(msnfs41client_cygwin_binaries_20240128_17h08m_git38817b8.tar.bz2)= e449955c1ea9c99d30ec02b9126047f587b2780c47753f3286bc5f2b5f9e6164 $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240128_17h08m_git38817b8.tar.bz2 ) $ /sbin/msnfs41client install # 5. Deinstallation: $ (set -x ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240128_17h08m_git38817b8.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) # 6. Usage: # Run the NFSv4 client daemon: # - run this preferably as "Administrator", but this is not a requirement # - requires separate terminal $ /sbin/msnfs41client run_daemon # Mount a filesystem and use it # - requires that NFSv4 server accepts connections from a TCP port # number > 1024, which can be archived on Linux with the "insecure" # export option in /etc/exports, or "resvport" on Solaris/Illumos # (see nfs(5)) $ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2 Successfully mounted '10.49.20.110@2049' to drive 'N:' $ cd /cygdrive/n/ $ ls -la total 4 drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 . dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 .. drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030 drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2 # Unmount filesystem: $ cd ~ && /sbin/nfs_mount -d N: # OR $ cd ~ $ net use N: /delete # List mounted NFSv4.1 filesystems: $ /sbin/nfs_mount # 7. Notes: - Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4 server works via /lib/msnfs41client/cygwin_idmapper.ksh, which either uses builtin static data, or /usr/bin/getent passwd and /usr/bin/getent group. As getent uses the configured name services it should work with LDAP too. This is still work-in-progress, with the goal that both NFSv4 client and server can use different uid/gid numeric values for client and server side. - UNC paths are supported, after successful mounting /sbin/nfs_mount will list the paths in Cygwin UNC format. - SIDs work, users with valid Windows accounts (see Cygwin idmapping above get their SIDs, unknown users with valid uid/gid values get Unix_User+id/Unix_Group+id SIDs, and all others are mapped to nobody/nogroup SIDs. - Cygwin symlinks are supported, but might require $ fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1 #. This includes symlinks to UNC paths, e.g. as Admin $ cmd /c
Source code for bash package?
Good morning! Where can I find the source code for the Cygwin bash package? Not the bash source itself, I am looking for the data which is used to build the Cygwin bash package. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/fallocate missing in Cygwin 3.5's "util-linux" ...
On Thu, 25 Jan 2024 at 10:37, Cedric Blancher wrote: > > On Thu, 25 Jan 2024 at 04:53, Mark Geisert via Cygwin > wrote: > > > > On 1/23/2024 4:41 PM, Mark Geisert via Cygwin wrote: > > > On 1/23/2024 3:36 AM, Roland Mainz via Cygwin wrote: > > >> Small bug report: > > >> Cygwin 3.5. now has support for SEEK_HOLE (thanks! :-) ), but > > >> /usr/bin/fallocate is still missing in the "util-linux" package. > > >> > > >> Can someone please enable that tool ? > > > > > > I'll look into this. > > > > A new build of the util-linux package, 2.33.1-3, now includes fallocate > > and its man page. The updated package is now making its way to the > > Cygwin mirrors. fallocate requires Cygwin version >= 3.5.0. > > Thank you. > > The GNU coreutils package (cp, mv), GNU tar (tar), pax (pax) and > libarchive packages need to be rebuild, as they all support SEEK_HOLE. > > @Corinna Vinschen Could you please include these updated packages as > 'sparse file aware', as they support SEEK_HOLE? ... Include in the Cygwin 3.5 release notes. My fault, I hit the send button too fast Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/fallocate missing in Cygwin 3.5's "util-linux" ...
On Thu, 25 Jan 2024 at 04:53, Mark Geisert via Cygwin wrote: > > On 1/23/2024 4:41 PM, Mark Geisert via Cygwin wrote: > > On 1/23/2024 3:36 AM, Roland Mainz via Cygwin wrote: > >> Small bug report: > >> Cygwin 3.5. now has support for SEEK_HOLE (thanks! :-) ), but > >> /usr/bin/fallocate is still missing in the "util-linux" package. > >> > >> Can someone please enable that tool ? > > > > I'll look into this. > > A new build of the util-linux package, 2.33.1-3, now includes fallocate > and its man page. The updated package is now making its way to the > Cygwin mirrors. fallocate requires Cygwin version >= 3.5.0. Thank you. The GNU coreutils package (cp, mv), GNU tar (tar), pax (pax) and libarchive packages need to be rebuild, as they all support SEEK_HOLE. @Corinna Vinschen Could you please include these updated packages as 'sparse file aware', as they support SEEK_HOLE? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/fallocate missing in Cygwin 3.5's "util-linux" ...
On Wed, 24 Jan 2024 at 07:33, Cedric Blancher wrote: > > On Tue, 23 Jan 2024 at 12:37, Roland Mainz via Cygwin > wrote: > > > > Hi! > > > > - > > > > Small bug report: > > Cygwin 3.5. now has support for SEEK_HOLE (thanks! :-) ), but > > /usr/bin/fallocate is still missing in the "util-linux" package. > > > > Can someone please enable that tool ? > > Good catch! > > @Corinna Vinschen > Does /usr/bin/cp in Cygwin copy holes correctly? cp, mv, tar, pax must be sparse file aware. Also Schilly star. Plus /usr/bin/fallocate Anything else? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: /usr/bin/fallocate missing in Cygwin 3.5's "util-linux" ...
On Tue, 23 Jan 2024 at 12:37, Roland Mainz via Cygwin wrote: > > Hi! > > - > > Small bug report: > Cygwin 3.5. now has support for SEEK_HOLE (thanks! :-) ), but > /usr/bin/fallocate is still missing in the "util-linux" package. > > Can someone please enable that tool ? Good catch! @Corinna Vinschen Does /usr/bin/cp in Cygwin copy holes correctly? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Package request: ms-nfs41-client
Good morning! Could Cygwin package the ms-nfs41-client (NFSv4 client with Cygwin support), please? Ced -- Forwarded message - From: Roland Mainz Date: Mon, 1 Jan 2024 at 22:47 Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 Windows driver binaries for Windows 10/11 for testing, 2024-01-01 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #43852f547ce80b3b33bb05c2e993e322d2264dfa + patches which should fix the VC runtime issues, git bundle in tarball), for testing and feedback (download URL below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. Requirements: - Windows 10 - Cygwin 3.5.0 - Packages: cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less pax pbzip2 procps-ng sed tar time util-linux wget # 2. Installation (as "Administrator"): $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240101_22h08m_git683af48.tar.bz2' $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240101_22h08m_git683af48.tar.bz2 ) $ /sbin/msnfs41client install # 3. Deinstallation: $ (set -x ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240101_22h08m_git683af48.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) ## ## Usage ## # Run the NFSv4 client daemon: # - run this preferably as "Administrator", but this is not a requirement # - requires separate terminal $ /sbin/msnfs41client run_daemon # Mount a filesystem and use it # - requires that NFSv4 server accepts connections from a TCP port # number > 1024, which can be archived on Linux with the "insecure" # export option in /etc/exports, or "resvport" on Solaris/Illumos # (see nfs(5)) $ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2 Successfully mounted '10.49.20.110@2049' to drive 'N:' $ cd /cygdrive/n/ $ ls -la total 4 drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 . dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 .. drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030 drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2 # Unmount filesystem: $ cd ~ && /sbin/nfs_mount -d N: # OR $ cd ~ $ net use N: /delete # # Notes: # - Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4 server works via /lib/msnfs41client/cygwin_idmapper.ksh, which either uses builtin static data, or /usr/bin/getent passwd and /usr/bin/getent group. As getent uses the configured name services it should work with LDAP too. This is still work-in-progress, with the goal that both NFSv4 client and server can use different uid/gid numeric values for client and server side. - UNC paths are supported, after successful mounting /sbin/nfs_mount will list the paths in Cygwin UNC format. - SIDs work, users with valid Windows accounts (see Cygwin idmapping above get their SIDs, unknown users with valid uid/gid values get Unix_User+id/Unix_Group+id SIDs, and all others are mapped to nobody/nogroup SIDs. - Cygwin symlinks are supported, but might require $ fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1 #. This includes symlinks to UNC paths, e.g. as Admin $ cmd /c 'mklink /d c:\home\rmainz \\derfwpc5131_ipv6@2049\nfs4\export\home2\rmainz' # and then $ cd /cygdrive/c/home/rmainz/ # should work - performance: All binaries are build without any optimisation, so the filesystem is much slower than it could be. - bad performance due to Windows Defender AntiVirus: Option 1: # disable Windows defender realtime monitoring # (requires Admin shell) powershell -Command 'Set-MpPreference -DisableRealtimeMonitoring 1' Option 2: Add "nfsd.exe", "nfsd_debug.exe", "ksh93.exe", "bash.exe", "git.exe" and other offending commands to the process name whitelist. - performance: Use vmxnet3 in VMware to improve performance - ACLs are supported via the normal Windows ACL tools, but on Linux require the nfs4_getfacl/nfs4_setfacl utilities to see the data. Example (assuming that Windows, Linux NFSv4 client and NFSv4 server have a user "siegfried_wulsch"): - On Windows on a NFSv4 filesystem, : $ icacls myhorribledata.txt /grant "siegfried_wulsch:WD" # - On Linux NFSv4 clients you will then see this: snip $ nfs4_getfacl myhorribledata.txt A::OWNER@:rwatTcCy A::siegfried_wul...@global.loc:rwatcy A::GROUP@:rtcy A::EVERYONE@:rtcy snip -
Cygwin support for btime/birth time? Fwd: [PATCH 0/5] fuse: support birth time
Good evening! Does Cygwin support the btime/birth time? Ced -- Forwarded message - From: Miklos Szeredi Date: Thu, 10 Aug 2023 at 14:19 Subject: [PATCH 0/5] fuse: support birth time To: Add the infrastructure for btime support in the form of a new STATX request. The format of the STATX reply is binary compatible with struct statx on Linux, but the structure is defined separately in the fuse API for other OS's. Currently STATX only supports basic attributes, same as the GETATTR request, plus btime. But nothing prevents extending support to other (present and future) statx fields. --- Miklos Szeredi (5): fuse: handle empty request_mask in statx fuse: add STATX request fuse: add ATTR_TIMEOUT macro fuse: implement statx fuse: cache btime fs/fuse/dir.c | 145 +++--- fs/fuse/fuse_i.h | 15 +++- fs/fuse/inode.c | 30 ++-- fs/fuse/readdir.c | 6 +- include/uapi/linux/fuse.h | 56 ++- 5 files changed, 218 insertions(+), 34 deletions(-) -- 2.40.1 -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin tools to read/write NTFS alternate data streams?
On Thu, 11 Jan 2024 at 17:41, Corinna Vinschen via Cygwin wrote: > > On Jan 11 15:00, Martin Wege via Cygwin wrote: > > On Mon, Jan 8, 2024 at 3:11 PM Corinna Vinschen via Cygwin > > wrote: > > so this is IMO OK. > > Yeah, but... > > It's not just an open flag, it requires extending functionality of other > APIs and, for full support, tools, see > https://docs.oracle.com/cd/E19253-01/816-5175/6mbba7f02/. > > It's also rather weird to call alternate data streams "extended > attributes", because that's just a small part of streams and it's > already supported via the Linux functions getxattr(2) and friends, see > https://man7.org/linux/man-pages/man2/getxattr.2.html > > > > Apart from that, this sounds like a nice idea for Cygwin 3.6, > > > provided somebody implements it, https://cygwin.com/acronyms/#SHTDI > > > > > > Assuming we can live without actually having a subdir and just > > > allowing to open and create a file with the O_XATTR flag, it might be > > > pretty simple to implement. The path handling code would just have to > > > drop the colon from the list of characters converted to the private-use > > > Unicode area. > > > > > > Implementing the subdir is a bit more complicated, especially when > > > taking opendir/readdir of that virtual subdir into account, but it > > > would certainly be doable. > > > > How do other OSes implement the O_XATTR subdir? > > IDK. But there are a few points we have to keep in mind from the > Solaris man page: > > - Given fd is an open file descriptor, > > openat(fd, ".", O_RDONLY|O_XATTR); > > opens the virtual subdir containing the ADS of a file. It's the > only valid way to open the virtual dir. That's a bit of a relief > because it simplifies handling this in openat(2). > > openat(fd, "foo", ...|O_XATTR); > > allows to create or open an ADS for an open file fd. > > - unlinkat, renameat, fstatat, fchownat, futimesat and fdopendir need to > be made ADS-aware. They only work on ADS if the fd arg already > resolves to an ADS, or if fd is the virtual ADS dir of a file and > (except in case of fdopendir) the path argument is the name of an > existing ADS. > > - pathconf needs to support a new flag XATTR_ENABLED. > > This can be made to work given the current fhandler framework in Cygwin, > but again, https://cygwin.com/acronyms/#SHTDI bash has cd -@ to enter a virtual dir with XATTRs, if O_XATTR works. Aside from that, I volunteer with testing of O_XATTR, on demand with 1 or 2 students too. Still working on testing the cool and new SEEK_HOLE support in Cygwin Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Qemu packages for Cygwin?
On Mon, 18 Dec 2023 at 18:33, Brian Inglis via Cygwin wrote: > > On 2023-12-18 00:14, Dan Shelton via Cygwin wrote: > > On Mon, 18 Dec 2023 at 07:54, Marco Atzeri via Cygwin wrote: > >> On 18/12/2023 07:42, Dan Shelton via Cygwin wrote: > >>> Does Cygwin come with qemu packages? > > >> why should an "Unix Emulation layer" that run in "User Space" > >> trying to run a "Full-system emulation" (https://www.qemu.org/) ? > >> It seems like using square wheels > > > Nope. The Qemu packages on LInux have a much more wider functionality, > > and more features, compared to the TUSBTADI "The User Should Be > > Treated As Dumb Idiot" versions on Windows. It absolutely makes sense. > > Cygwin has trouble providing some POSIX features under Windows due to its API > limitations, using "documented" Windows APIs, allowing it to run under > ReactOS, > Wine, and similar emulators, with which there is some interchange. > > It does not provide or support much in the way of privileged Unix or Windows > features or access, except for providing emulation for running daemons as > services, and some caching of SAM, AD, and process info. > > It is totally inadequate as a platform to provide or support any x86 > virtualization or even virtual machine management features, as it can not > provide access to any machine features that do not have "documented" Windows > APIs. > Look at the /proc filesystem emulation limitations for examples of where > Windows > lacks APIs to get information available under Linux. > > You complain elsewhere about performance: I am also complaining about performance, but it is NOT "performance in general": What really sucks is the filesystem inode operations and file name lookup, e.g. /bin/find are absurdly slow because of the link emulation and the inflation of syscalls caused by it - just up to five times more accesses just to handle filename, filename.lnk, filename.lnk.exe, filename.lnk.bat etc Other areas have adequate performance, e.g. read/write performance is OK > can you imagine how bad it would be, > compared to Hyper-V, WSL, VirtualBox, etc. No, qemu is called the "quick emulator", and has more functionality than just being a hypervisor, like being able to emulate ARM and SPARC platforms. Even if the x86 hypervisor cannot be used, there is still lots of functionality. If there are bugs in the POSIX emulation layer, then those bugs should be 1) documented and 2) fixed. Also please do not shoot down package proposals out of the FEAR that "something" might not work. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Package request: ytdl-org/youtube-dl
Good morning! I'd like to request https://github.com/ytdl-org/youtube-dl as a Cygwin package, so we can mirror training videos locally. Also, it is a cool demo to show the power of Cygwin, to show that it can download youtube videos without paying for "Youtube Premium" =:-) Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Fwd: [Ms-nfs41-client-devel] ANN: NFSv4.1 Windows driver binaries for Windows 10/11 for testing, 2024-01-01 ...
For your consideration - we need FEEDBACK, please! Ced -- Forwarded message - From: Roland Mainz Date: Mon, 1 Jan 2024 at 22:47 Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 Windows driver binaries for Windows 10/11 for testing, 2024-01-01 ... To: Hi! I've created a set of test binaries for the NFSv4.1 filesystem driver for Windows 10/11, based on https://github.com/kofemann/ms-nfs41-client (commit id #43852f547ce80b3b33bb05c2e993e322d2264dfa + patches which should fix the VC runtime issues, git bundle in tarball), for testing and feedback (download URL below). Please send comments, bugs, test reports, complaints etc. to the MailMan mailing list at https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel # 1. Requirements: - Windows 10 - Cygwin 3.5.0 - Packages: cygwin cygwin-devel cygrunsrv cygutils cygutils-extra bash bzip2 coreutils getent gdb grep hostname less pax pbzip2 procps-ng sed tar time util-linux wget # 2. Installation (as "Administrator"): $ mkdir -p ~/download $ cd ~/download $ wget 'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20240101_22h08m_git683af48.tar.bz2' $ (cd / && tar -xf ~/download/msnfs41client_cygwin_binaries_20240101_22h08m_git683af48.tar.bz2 ) $ /sbin/msnfs41client install # 3. Deinstallation: $ (set -x ; cd / && tar -tf ~/download/msnfs41client_cygwin_binaries_20240101_22h08m_git683af48.tar.bz2 | while read i ; do [[ -f "$i" ]] && rm "$i" ; done) ## ## Usage ## # Run the NFSv4 client daemon: # - run this preferably as "Administrator", but this is not a requirement # - requires separate terminal $ /sbin/msnfs41client run_daemon # Mount a filesystem and use it # - requires that NFSv4 server accepts connections from a TCP port # number > 1024, which can be archived on Linux with the "insecure" # export option in /etc/exports, or "resvport" on Solaris/Illumos # (see nfs(5)) $ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2 Successfully mounted '10.49.20.110@2049' to drive 'N:' $ cd /cygdrive/n/ $ ls -la total 4 drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 . dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 .. drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030 drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2 # Unmount filesystem: $ cd ~ && /sbin/nfs_mount -d N: # OR $ cd ~ $ net use N: /delete # # Notes: # - Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4 server works via /lib/msnfs41client/cygwin_idmapper.ksh, which either uses builtin static data, or /usr/bin/getent passwd and /usr/bin/getent group. As getent uses the configured name services it should work with LDAP too. This is still work-in-progress, with the goal that both NFSv4 client and server can use different uid/gid numeric values for client and server side. - UNC paths are supported, after successful mounting /sbin/nfs_mount will list the paths in Cygwin UNC format. - SIDs work, users with valid Windows accounts (see Cygwin idmapping above get their SIDs, unknown users with valid uid/gid values get Unix_User+id/Unix_Group+id SIDs, and all others are mapped to nobody/nogroup SIDs. - Cygwin symlinks are supported, but might require $ fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1 #. This includes symlinks to UNC paths, e.g. as Admin $ cmd /c 'mklink /d c:\home\rmainz \\derfwpc5131_ipv6@2049\nfs4\export\home2\rmainz' # and then $ cd /cygdrive/c/home/rmainz/ # should work - performance: All binaries are build without any optimisation, so the filesystem is much slower than it could be. - bad performance due to Windows Defender AntiVirus: Option 1: # disable Windows defender realtime monitoring # (requires Admin shell) powershell -Command 'Set-MpPreference -DisableRealtimeMonitoring 1' Option 2: Add "nfsd.exe", "nfsd_debug.exe", "ksh93.exe", "bash.exe", "git.exe" and other offending commands to the process name whitelist. - performance: Use vmxnet3 in VMware to improve performance - ACLs are supported via the normal Windows ACL tools, but on Linux require the nfs4_getfacl/nfs4_setfacl utilities to see the data. Example (assuming that Windows, Linux NFSv4 client and NFSv4 server have a user "siegfried_wulsch"): - On Windows on a NFSv4 filesystem, : $ icacls myhorribledata.txt /grant "siegfried_wulsch:WD" # - On Linux NFSv4 clients you will then see this: snip $ nfs4_getfacl myhorribledata.txt A::OWNER@:rwatTcCy A::siegfried_wul...@global.loc:rwatcy A::GROUP@:rtcy A::EVERYONE@:rtcy snip - nfs_mount only works when the NFSv4 server allows
Fwd: [Ms-nfs41-client-devel] ANN: NFSv4.1 Windows driver binaries for Windows 10 for testing, 2023-12-30 ...
For your consideration - we need FEEDBACK, please!
Ced
-- Forwarded message -
From: Roland Mainz
Date: Sat, 30 Dec 2023 at 16:34
Subject: [Ms-nfs41-client-devel] ANN: NFSv4.1 Windows driver binaries
for Windows 10 for testing, 2023-12-30 ...
To:
Hi!
I've created a set of test binaries for the NFSv4.1 filesystem driver
for Windows, based on https://github.com/kofemann/ms-nfs41-client
(commit id #43852f547ce80b3b33bb05c2e993e322d2264dfa), for testing and
feedback (download URL below).
Please send comments, bugs, test reports, complaints etc. to
https://sourceforge.net/projects/ms-nfs41-client/lists/ms-nfs41-client-devel
# 1. Requirements:
- Windows 10 (64bit, without SecureBoot!!)
- Cygwin 3.5.0
(Install in Cygwin setup.exe, Install with checkboxes "Testing" and "Sync")
- Packages:
cygwin
cygwin-devel
cygrunsrv
cygutils
cygutils-extra
bash
bzip2
coreutils
getent
gdb
grep
hostname
less
pax
pbzip2
procps-ng
sed
tar
time
util-linux
wget
# 2. Installation (as "Administrator"):
$ mkdir -p ~/download
$ cd ~/download
$ wget
'http://www.nrubsig.org/people/gisburn/work/msnfs41client/releases/testing/msnfs41client_cygwin_binaries_20231230_14h12m_git43852f5.tar.bz2'
$ (cd / && tar -xf
~/download/msnfs41client_cygwin_binaries_20231230_14h12m_git43852f5.tar.bz2
)
$ /sbin/msnfs41client install
# 3. Deinstallation:
$ (set -x ; cd / && tar -tf
~/download/msnfs41client_cygwin_binaries_20231230_14h12m_git43852f5.tar.bz2
| while read i ; do [[ -f "$i" ]] && rm "$i" ; done)
##
## Usage
##
# Run the NFSv4 client daemon:
# - run this preferably as "Adminstrator", but this is not a requirement
# - requires separate terminal
$ /sbin/msnfs41client run_daemon
# Mount a filesystem and use it
$ /sbin/nfs_mount -o rw N 10.49.20.110:/net_tmpfs2
Successfully mounted '10.49.20.110@2049' to drive 'N:'
$ cd /cygdrive/n/
$ ls -la
total 4
drwxrwxrwt 5 Unix_User+0 Unix_Group+0 100 Dec 7 14:17 .
dr-xr-xr-x 1 roland_mainz Kein0 Dec 14 13:48 ..
drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 80 Dec 12 16:24 10492030
drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 13 17:58 directory_t
drwxr-xr-x 3 Unix_User+197608 Unix_Group+197121 60 Dec 7 11:01 test2
# Unmount filesystem:
$ cd ~
$ net use N: /delete
#
# Notes:
#
- Idmapping (including uid/gid mapping) between NFSv4 client and NFSv4
server works via /lib/msnfs41client/cygwin_idmapper.ksh, which
either uses builtin static data, or /usr/bin/getent passwd and
/usr/bin/getent group.
As getent uses the configured name services it should work with LDAP
too.
This is still work-in-progress, with the goal that both NFSv4 client
and server can use different uid/gid numeric values for client and
server side.
- UNC paths are supported, after successful mounting /sbin/nfs_mount
will list the paths in Cygwin UNC format.
- SIDs work, users with valid Windows accounts (see Cygwin idmapping
above get their SIDs, unknown users with valid uid/gid values get
Unix_User+id/Unix_Group+id SIDs, and all others are mapped
to nobody/nogroup SIDs.
- Cygwin symlinks are supported, but might require
$ fsutil behavior set SymlinkEvaluation L2L:1 R2R:1 L2R:1 R2L:1 #.
This includes symlinks to UNC paths, e.g. as Admin
$ cmd /c 'mklink /d c:\home\rmainz
\\derfwpc5131_ipv6@2049\nfs4\export\home2\rmainz' #
and then $ cd /cygdrive/c/home/rmainz/ # should work
- performance: All binaries are build without any optimisation, so
the filesystem is much slower than it could be.
- bad performance due to Windows Defender AntiVirus:
Option 1:
# disable Windows defender realtime monitoring
# (requires Admin shell)
powershell -Command 'Set-MpPreference -DisableRealtimeMonitoring 1'
Option 2:
Add "nfsd.exe", "nfsd_debug.exe", ksh93.exe, bash.exe, git.exe and
other offending commands to process name whitelist.
- performance: Use vmxnet3 in VMware to improve performance
- ACLs are supported via the normal Windows ACL tools, but on
Linux require the nfs4_getfacl/nfs4_setfacl utilties to see the
data.
Example (assuming that Windows, Linux NFSv4 client and NFSv4
server have a user "siegfried_wulsch"):
- On Windows on a NFSv4 filesystem, :
$ icacls myhorribledata.txt /grant "siegfried_wulsch:WD" #
- On Linux NFSv4 clients you will then see this:
snip
$ nfs4_getfacl myhorribledata.txt
A::OWNER@:rwatTcCy
A::siegfried_wul...@global.loc:rwatcy
A::GROUP@:rtcy
A::EVERYONE@:rtcy
snip
#
# Known issues:
#
- The kernel driver ("nfs41_driver.sys") does not have a cryptographic
signature for SecureBoot - which means it will only work if SecureBoot
is turned off (otherwise $ /sbin/msnfs41client install # will FAIL!)
- If nfsd_debug.exe crashes or gets killed, the only safe way
to run it again requires a
Re: rfe: CYGWIN fslinktypes option? Re: Catastrophic Cygwin find . -ls, grep performance on samba share compared to WSL
On Thu, 21 Dec 2023 at 13:17, Martin Wege via Cygwin wrote:
>
> On Wed, Dec 20, 2023 at 6:21 PM Kaz Kylheku via Cygwin
> wrote:
> >
> > On 2023-12-17 22:22, Dan Shelton via Cygwin wrote:
> > > It would be nice if someone from the Cygwin authors could assist me in
> > > figuring out why this happens.
> >
> > Cygwin is famously slow; this is nothing new. We are grateful
> > for Cygwin because it makes stuff work at all; if it were blazing
> > fast that would be a bonus.
> >
> > E.g. git operations (clone, rebase, ...); ./configure scripts; ...: all
> > run like molasses.
> >
> > The following is just my fast and loose opinion, shot from the hip,
> > and possibly off or wrong, but it likely has to do with the layering.
> > Cygwin's core API is based on a C library called Newlib. Cygwin bolts
> > Newlib to Windows by means of an additional shim below Newlib that
> > is based on C++ objects, where there is path munging going on and such,
> > and that's where the Win32 calls get made. It's an additional abstraction.
>
> I disagree with that. Ok, part of that is that the layering causes
> more memory allocations and copies, but this is not the root cause.
>
> The root cause is IMO the extra Win32 syscalls (>= 3 per file lookup,
> compared to 1 on Linux) to lookup the *.lnk and *.exe.lnk files on
> filesystems which have native link support (NTFS, ReFS, SMBFS, NFS).
> On SMBFS and NFS it hurts the most, because access latency is the
> highest for networked filesystems.
>
> So my proposal would be to add an option ('fslinktypes') to the CYGWIN
> environment variable to define which types of links are supported:
> default 'all'. which is an shortcut for 'native,lnk,lnkexe'.
> So in case people do not want 'lnk' link support they just add
> CYGWIN+=' fslinktypes:native' to env, to turn off support for
> lnk/lnk.exe style links, and be happy.
>
> @Corinna Vinschen Would that be acceptable?
+1 for this proposal, which is almost the same idea as I proposed in
https://www.mail-archive.com/cygwin@cygwin.com/msg174612.html
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin generates syscalls for *.lnk files on filesystems with native symlink support?
On Fri, 1 Sept 2023 at 13:00, Corinna Vinschen via Cygwin wrote: > > On Sep 1 06:23, Cedric Blancher via Cygwin wrote: > > Good morning! > > > > During a Cygwin 3.4.8-1.x86_64 debugging session I noticed something > > odd when I looked at the network traffic generated by one of our > > cluster nodes: > > It seems that for each call to a tool (i.e. starting "sed" from > > "bash") Cygwin searches for *.lnk files. > > > > Is this correct even when the filesystem in question has native > > symlink support (e.g. NFS)? > > Yes. During file handling, Cygwin doesn't know what filesystem a > file is on until it could actually open the file and request file > and filesystem info from the open handle. Why? If you have the path name you could lookup the (cached) mount points, and determine the filesystem type. Same solution applies for UNC paths, where you can easily lookup the filesystem type, and cache it per-process or in Cygserver. > So if Cygwin couldn't open > "foo" because the NtCreateFile call returned with status > STATUS_OBJECT_PATH_NOT_FOUND or STATUS_OBJECT_NAME_NOT_FOUND, or > STATUS_NO_SUCH_FILE, or one of the countless other status codes the > kernel (or the driver) might return in case a file doesn't exist, > it will tack on .lnk and .exe and, for historical reasons, .exe.lnk, > and try again. Can this machinery please be turned off via CYGWIN env var option? As discussed in https://www.mail-archive.com/cygwin@cygwin.com/msg174547.html this machinery causes very bad filesystem lookup performance, and it would IMO a good idea to have an option to turn this off, and just allow and expect native links (for NTFS, ReFS and NFS). Maybe CYGWIN env var option winsymlinks_expect:native? winsymlinks_expect takes a : seperated list which symlink types are to be expected. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: BugZilla for Cygwin, PLEASE!
On Mon, 18 Dec 2023 at 07:58, Marco Atzeri via Cygwin wrote: > > On 18/12/2023 07:45, Dan Shelton via Cygwin wrote: > > Hello! > > > > I don't see whether this has been discussed before, > > but PLEASE could Cygwin get a bug tracker (BUGZILLA!)? > > > > Dan > > https://cygwin.com/acronyms/#SHTDI > > the bandwidth of current maintainers is the major constraint > > The current bugtracker is this mailing list Yeah, from a project administrator and project management viewpoint this is just stupid. A bug tracker like Bugzilla would improve the "little bandwidth" the maintainers have, by organising them better. It would make them more efficient. That said, PLEASE don't use JIRA or AZURE bugtracker, that would be an overkill and lead to worse performance. Just plain old Bugzilla. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Sparse file support for SMB by default? Re: Comment about "Cygwin: sparse support: enable automatic sparsifying of files on SSDs", extend feature to VMware/qemu disks?
On Wed, 6 Dec 2023 at 10:39, Corinna Vinschen via Cygwin wrote: > > On Dec 6 09:49, Cedric Blancher via Cygwin wrote: > > On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin > > VMware emulates NVME SSD these days, so this should work, yes? > > It doesn't matter, see below. > > > Primary concern is how to detect whether sparse file support is ON in > > Cygwin for a specific filesystem? Maybe /sbin/mount without options > > should have a sparse/nosparse mount flag to reflect what is used? > > What concern? I explained exactly what happens, and it all revolves > around the fact that a filesystem sets the FILE_SUPPORTS_SPARSE_FILES > flag or not. This is the only interesting fact. Nothing else > matters. > > If you want to see if a filesystem supports that flag, just use the > /usr/lib/csih/getVolInfo tool: > > $ /usr/lib/csih/getVolInfo . > Device Type: 7 > Characteristics: 20020 > Volume Name: > Serial Number : 2292001085 > Max Filenamelength : 255 > Filesystemname : > Flags : 3e72eff > FILE_CASE_SENSITIVE_SEARCH: TRUE > FILE_CASE_PRESERVED_NAMES : TRUE > FILE_UNICODE_ON_DISK : TRUE > FILE_PERSISTENT_ACLS : TRUE > FILE_FILE_COMPRESSION : TRUE > FILE_VOLUME_QUOTAS: TRUE > FILE_SUPPORTS_SPARSE_FILES: TRUE > FILE_SUPPORTS_REPARSE_POINTS : TRUE > FILE_SUPPORTS_REMOTE_STORAGE : FALSE > FILE_RETURNS_CLEANUP_RESULT_INFO : TRUE > FILE_SUPPORTS_POSIX_UNLINK_RENAME : TRUE > FILE_VOLUME_IS_COMPRESSED : FALSE > FILE_SUPPORTS_OBJECT_IDS : TRUE > FILE_SUPPORTS_ENCRYPTION : TRUE > FILE_NAMED_STREAMS: TRUE > FILE_READ_ONLY_VOLUME : FALSE > FILE_SEQUENTIAL_WRITE_ONCE: FALSE > FILE_SUPPORTS_TRANSACTIONS: TRUE > FILE_SUPPORTS_HARD_LINKS : TRUE > FILE_SUPPORTS_EXTENDED_ATTRIBUTES : TRUE > FILE_SUPPORTS_OPEN_BY_FILE_ID : TRUE > FILE_SUPPORTS_USN_JOURNAL : TRUE > FILE_SUPPORTS_INTEGRITY_STREAMS : FALSE > FILE_SUPPORTS_BLOCK_REFCOUNTING : FALSE > FILE_SUPPORTS_SPARSE_VDL : FALSE > FILE_DAX_VOLUME : FALSE > FILE_SUPPORTS_GHOSTING: FALSE > > > > What Cygwin gets to see is that you want to access a file on some > > > disk. It opens the file and fetches volume information. So it finds: > > > > > > - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES > > > > > > So we can create sparse files. > > > > > > - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag. > > > > > > So we're on a drive having no seek penalty. This is *probably* > > > an SSD, but may be some other type of drive. > > > > What about SMB? SMB supports sparse files, and should certainly use > > this by DEFAULT, as it makes many HPC apps operate much faster. > > BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a > > drive, and not for network filesystem. > > > > Same applies to the NFSv4.1 filesystem driver, I'm going to pester > > Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue > > as SMB here. > > Again, the filesystem doesn't matter. It either sets the > FILE_SUPPORTS_SPARSE_FILES flag or not, as simple as that. > > If it does, you can create sparse files with chattr +S, or you can rely > on the lseek/ftruncate/posix_fallocate automatism, or you stamp a hole > into the file with fallocate(FALLOC_FL_PUNCH_HOLE). > > The *only* difference is if you have to use the "sparse" mount option or > not. > > Basically, with 3.4, you always have to set the "sparse" mount option, > with 3.5, on local SSDs you don't. I don't see a problem here. How can I remount the existing C: mount with the "sparse" option in Cygwin 3.4? I cannot get it working, or I am too stupid. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Custom package repository in addition to default Cygwin packages?
Good morning! Can Cygwin setup.exe be modified to support more than one package repository, e.g. first the default Cygwin package repro, and then a company package repro (e.g. fr.pasteur.cygwin.packages ?). Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Sparse file support for SMB by default? Re: Comment about "Cygwin: sparse support: enable automatic sparsifying of files on SSDs", extend feature to VMware/qemu disks?
On Fri, 1 Dec 2023 at 11:43, Corinna Vinschen via Cygwin wrote: > > On Dec 1 11:12, Cedric Blancher via Cygwin wrote: > > Good morning! > > > > Small comment about 3.5.0 commit "Cygwin: sparse support: enable > > automatic sparsifying of files on SSDs" > > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=832e91422c4ae9b2dcd0c307779f3dd53ee9c0ac > > > > Can this functionality be extended to VMware and QEMU virtual disks, please? > > Not without a complete rewrite and lots of slowing down code. > > VMware and QEMU virtual disks are not exposed to the OS as VMware and > QEMU virtual disks as far as the volume and filesystem information is > going. They are just some arbitrary harddisks managed by some > arbitrary driver. VMware emulates NVME SSD these days, so this should work, yes? Primary concern is how to detect whether sparse file support is ON in Cygwin for a specific filesystem? Maybe /sbin/mount without options should have a sparse/nosparse mount flag to reflect what is used? > > What Cygwin gets to see is that you want to access a file on some > disk. It opens the file and fetches volume information. So it finds: > > - The filesystem returns the FILE_SUPPORTS_SPARSE_FILES > > So we can create sparse files. > > - The volume returns the SSINFO_FLAGS_NO_SEEK_PENALTY flag. > > So we're on a drive having no seek penalty. This is *probably* > an SSD, but may be some other type of drive. What about SMB? SMB supports sparse files, and should certainly use this by DEFAULT, as it makes many HPC apps operate much faster. BUT: SSINFO_FLAGS_NO_SEEK_PENALTY is only for filesystems with a drive, and not for network filesystem. Same applies to the NFSv4.1 filesystem driver, I'm going to pester Roland Mainz to add FILE_SUPPORTS_SPARSE_FILES support, so same issue as SMB here. Ced -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Maximum path and path element length, Win32 vs Cygwin vs SFU?
Good evening! What is the maximum path length and path element (single file, dirname length) in Win32 API, Cygwin APIs on Windows 11, and SFU on Win7? There seems to be some confusion, docs cite 256, 260 and 32k, and getconf PATH_MAX / returns 4096. What is the maximum supported in the Win32 API, what is the maximum supported in Cygwin, and what applications and drivers need to expect? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
_PC_MIN_HOLE_SIZE pathconf() Re: Cygwin api to punch a hole into a file?
On Tue, 28 Nov 2023 at 11:30, Corinna Vinschen via Cygwin wrote: > > On Nov 24 12:01, Corinna Vinschen via Cygwin wrote: > > On Nov 23 23:36, Cedric Blancher via Cygwin wrote: > > > Linux has fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, ...) > > > to punch a hole into a file, i.e. deallocate the blocks given and make > > > the file a "sparse file". > > > > We don't support the Linux-specific fallocate(2) call, only ftruncate(2) > > and posix_fallocate(3). Patches, as usual, thoughtfully considered. > > The next test release cygwin-3.5.0-0.485.g65831f88d6c4 introduces > the Linux-specific fallocate(2) call. Naturally we can't support > all flags, but the following flag combinations are allowed: > > - 0 same as posix_fallocate(3) > - FALLOC_FL_KEEP_SIZE > - FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE > - FALLOC_FL_ZERO_RANGE > - FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE > > A few comments: > > - With 0 and FALLOC_FL_KEEP_SIZE, sparse blocks in the given range > are re-allocated as per the POSIX requirements. For that, it uses > the same code as FALLOC_FL_ZERO_RANGE, but only for the holes within > the range. > > - With FALLOC_FL_KEEP_SIZE, over-allocation is done by setting > the allocation size of a file while keeping EOF the same. > However, in contrast to your typical Linux filesystem, over- > allocation on Windows filesystems is only temporary. The > over-allocated blocks are returned to free blocks as soon as > the last HANDLE to the file is closed. > > - With FALLOC_FL_KEEP_SIZE, no over-allocation is performed on sparse > files. The reason is that over-allocation on sparse files has no > effect on Windows. > > - FALLOC_FL_ZERO_RANGE is implemented as writing zeros to the > given range. For parts of the range which are already allocated > data blocks, as much zeros are written as necessary. For holes > in sparse files, only a single zero byte is written to the > hole per 64K chunk, which is the allocation granularity of > sparse files. > > - FALLOC_FL_ZERO_RANGE is NOT atomic. > > Please give it a try. Could you please implement _PC_MIN_HOLE_SIZE in pathconf(), so that people can query the minimum hole size for a (path in a) filesystem? Should return nothing for FAT and NFSv3, should work on NTFS and NFSv4.2 Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin api to punch a hole into a file?
On Tue, 28 Nov 2023 at 11:30, Corinna Vinschen via Cygwin wrote: > > On Nov 24 12:01, Corinna Vinschen via Cygwin wrote: > > On Nov 23 23:36, Cedric Blancher via Cygwin wrote: > > > Linux has fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, ...) > > > to punch a hole into a file, i.e. deallocate the blocks given and make > > > the file a "sparse file". > > > > We don't support the Linux-specific fallocate(2) call, only ftruncate(2) > > and posix_fallocate(3). Patches, as usual, thoughtfully considered. > > The next test release cygwin-3.5.0-0.485.g65831f88d6c4 introduces > the Linux-specific fallocate(2) call. Naturally we can't support > all flags, but the following flag combinations are allowed: > > - 0 same as posix_fallocate(3) > - FALLOC_FL_KEEP_SIZE > - FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE > - FALLOC_FL_ZERO_RANGE > - FALLOC_FL_ZERO_RANGE | FALLOC_FL_KEEP_SIZE > > A few comments: > > - With 0 and FALLOC_FL_KEEP_SIZE, sparse blocks in the given range > are re-allocated as per the POSIX requirements. For that, it uses > the same code as FALLOC_FL_ZERO_RANGE, but only for the holes within > the range. > > - With FALLOC_FL_KEEP_SIZE, over-allocation is done by setting > the allocation size of a file while keeping EOF the same. > However, in contrast to your typical Linux filesystem, over- > allocation on Windows filesystems is only temporary. The > over-allocated blocks are returned to free blocks as soon as > the last HANDLE to the file is closed. > > - With FALLOC_FL_KEEP_SIZE, no over-allocation is performed on sparse > files. The reason is that over-allocation on sparse files has no > effect on Windows. > > - FALLOC_FL_ZERO_RANGE is implemented as writing zeros to the > given range. For parts of the range which are already allocated > data blocks, as much zeros are written as necessary. For holes > in sparse files, only a single zero byte is written to the > hole per 64K chunk, which is the allocation granularity of > sparse files. > > - FALLOC_FL_ZERO_RANGE is NOT atomic. > > Please give it a try. Corinna, THANK you! Related question about commit "Cygwin: pwrite(2): sparsify file" https://cygwin.com/git/?p=newlib-cygwin.git;a=blobdiff;f=winsup/cygwin/fhandler/disk_file.cc;h=c70afed49f1ecdb11812d31a9663d18c0a5f03f7;hp=b49b25c71ad030a9391db89a157cadcd095d4f36;hb=f64f3eced8e0f51753bc199f3ba96fab06a54848;hpb=114f89caff7b9b62b0b12bc2c6d143daf47b8042 I see the value of 128k (128*1024 bytes) quite often in your sparse file commits. Can you please make this value a per filesystem tunable? Not all filesystems have a 128k block/stripe size, and certainly most filesystems have smaller minimum hole sizes than 128k (e.g. 512bytes is common, ref pathconf _PC_MIN_HOLE_SIZE). Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Comment about "Cygwin: sparse support: enable automatic sparsifying of files on SSDs", extend feature to VMware/qemu disks?
Good morning! Small comment about 3.5.0 commit "Cygwin: sparse support: enable automatic sparsifying of files on SSDs" https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=832e91422c4ae9b2dcd0c307779f3dd53ee9c0ac Can this functionality be extended to VMware and QEMU virtual disks, please? Also, does Cygwin have a syslog where people can check whether the feature is on or off? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Cygwin api to punch a hole into a file?
Linux has fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, ...) to punch a hole into a file, i.e. deallocate the blocks given and make the file a "sparse file". But how can I do that with the Cygwin API? Does anyone have an example for Cygwin, which is the Win equivalent to Linux fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, 1048576, 2097152);? I tried with Cygwin 3.5, but the file was not sparse after that. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Capturing a Cygwin instance from another PC
On Wed, 22 Nov 2023 at 17:15, Eliot Moss via Cygwin wrote: > > On 11/22/2023 10:43 AM, Bill Stewart via Cygwin wrote: > > On Wed, Nov 22, 2023 at 7:53 AM Jose Isaias Cabrera wrote: > > > > I have a new Win11 PC, and I wanted to capture the same Cygwin setup that I > >> have in another Win10 PC. I copied the C:\cygwin64 folder from the Win10 pc > >> to the Win11 pc, then I downloaded a fresh setup-x86_64.exe from > >> cygwin.com to the win11 PC. I ran it and chose "Install from Local > >> Directory" and only one shortcut for "Cygwin64 Terminal" was created on the > >> desktop. No Cygwin nor Cygwin-X folders were created on the Start menu. The > >> original cygwin from the Win10 PC had X installed also. > >> > > > > I don't think "Install from Local Directory" means "reproduce same cygwin > > installation on a separate computer". > > In fact, it means to install using the package files in some local directory, > i.e., "assume the packages are already downloaded". > > It would be reasonable to copy over downloaded packages. > > I believe there are commands / techniques that would then make it fairly > easy to install that specific set of packages "from scratch" (but from > the local copy of the packages). Copying *installed* file hierarchies > is trickier because of permission / ownership concerns. Does Cygwin have a way to dump the list of installed packages, and feed that list back to Cygwin setup.exe? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Set localisation locale of Windows netstat?
Good morning! How can I set the localisation locale of Windows netstat cmd from Cygwin? LC_ALL=C does not help, a French or German Windows will use French or German language still Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: binfmt support in Cygwin?
On Tue, 26 Sept 2023 at 07:15, Cedric Blancher wrote: > > Good morning! > > Does Cygwin support something like Linux binfmt, to register new binary types? > > Goal would be to recognise python compiled code, or > https://wiki.debian.org/QemuUserEmulation ? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Qt6 in Cygwin?
Good afternoon Are there Qt6 - https://www.qt.io/product/qt6 - packages for Cygwin available? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Run a Cygwin process with "higher" scheduler priority
On Fri, 11 Aug 2023 at 19:14, Brian Inglis wrote: > > On 2023-08-11 09:44, Cedric Blancher via Cygwin wrote: > > How can we set the scheduler priority for a Cygwin process to "higher" > > than normal on start? > > > > Basically we want a Cygwin compatible script version of the Windows > > task manager feature Name/PID--->Details--->Set priority--->Higher > > Pretend the system is BSD or Linux and web search e.g. SO for that capability, > facility, or feature, then check Cygwin with Search Packages or: > > $ cygcheck -p usr/bin/nice > Found 6 matches for usr/bin/nice I know about nice and renice, but I don't see whether it is working or not. > coreutils-debuginfo-8.26-2 - coreutils-debuginfo: Debug info for coreutils > ... > coreutils-9.0-1 - coreutils: GNU core utilities (includes fileutils, sh-utils > and textutils) > $ cygcheck -p usr/bin/renice > Found 6 matches for usr/bin/renice > util-linux-debuginfo-2.32.1-1 - util-linux-debuginfo: Debug info for > util-linux > util-linux-debuginfo-2.33.1-1 - util-linux-debuginfo: Debug info for > util-linux > util-linux-debuginfo-2.33.1-2 - util-linux-debuginfo: Debug info for > util-linux > util-linux-2.32.1-1 - util-linux: Collection of basic system utilities > util-linux-2.33.1-1 - util-linux: Collection of basic system utilities > util-linux-2.33.1-2 - util-linux: Collection of basic system utilities > $ cygcheck -p usr/bin/chrt > Found 2 matches for usr/bin/chrt > util-linux-debuginfo-2.33.1-2 - util-linux-debuginfo: Debug info for > util-linux > util-linux-2.33.1-2 - util-linux: Collection of basic system utilities > $ cygcheck -p usr/bin/taskset > Found 2 matches for usr/bin/taskset > util-linux-debuginfo-2.33.1-2 - util-linux-debuginfo: Debug info for > util-linux > util-linux-2.33.1-2 - util-linux: Collection of basic system utilities > > so you may need to install util-linux (coreutils is already installed as a > base > package for Cygwin) then: > > $ man nice renice taskset chrt chrt does not work, the syscalls for realtime seem to be not implemented. > > You may need to run these from an elevated admin shell for other user ids, > unowned processes, or higher (negative) priority. > > It has been years since I tried these so can no longer remember if -20 > corresponds to Windows "RealTime" or if you need to use PowerShell to set > that e.g. > > $proc = Get-Process -ProcessName "..." > $proc.PriorityClass = "RealTime" > > and that needs to be done from an elevated admin (Power)Shell. > > Of course, never set that on any system except on a very idle, very I/O bound, > well tested process, such as ntpd, that needs stably fast interrupt service > and > non-preemptible cpu access for ms every few seconds, preferably on a single or > few cpus to which affinity is set. Is there anything in /proc which can tell me the current *Windows* scheduler priority level? Or a command line tool? Ced -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
ConvertStringSidToSidA("S-1-22-1-1", ) fails with invalid sid for Unix_User+1
Good afternoon!
I tried a simple three line hello world program, which tries to get
the SID for Unix_User+1:
bash -c 'getent passwd "Unix_User+1"'
Unix_User+1:*:4278190081:4278190081:U-Unix_User\1,S-1-22-1-1:/:/sbin/nologin
But putting "S-1-22-1-1" into ConvertStringSidToSidA("S-1-22-1-1",
) fails with ERROR_INVALID_SID on Cygwin.
Why? I thought SMB had "S-1-22-1-*" reserved for its usage, and Cygwin
uses Unix_User+1 in it's output.
How can I get ConvertStringSidToSidA("S-1-22-1-1", ) working in my program?
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin gdb to load debug into from Windows VC?
On Tue, 26 Sept 2023 at 06:39, Cedric Blancher wrote: > > Good morning! > > Is it possible to load Visual Studio (VC19, VC22) debug info (*.pdb > files) into Cygwin gdb? Anyone? Or can VC19/VC22 create DWARF debug data? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Cygwin 3.5 bug? /usr/bin/ls.exe -l and Windows dir /q disagree about file owner names
Good afternoon! Likely question for Lady Corinna: Can anyone please explain why Windows dir command and Cygwin ls -l list different owners for a file? The directory has two files, one owned by root (without mapping to Windows, so getting a NULL SID) and one owned by me (user cedrictest17) Is this a Cygwin bug? --cut--here--cut--here--cut--here-- $ uname -a CYGWIN_NT-10.0-19045 winzott1984 3.5.0-0.423.g7ced682549ae.x86_64 2023-09-07 17:19 UTC x86_64 Cygwin $ cmd /C 'dir /q' 03.10.2023 13:38 WINZOTT1984\cedrictest17. 03.10.2023 13:35 WINZOTT1984\cedrictest17.. 03.10.2023 13:38 0 WINZOTT1984\cedrictest17owned_by_cedric 03.10.2023 05:57 0 \NULL SID owned_by_root 2 Datei(en), 8.192 Bytes 2 Verzeichnis(se), 52.775.219.200 Bytes frei $ ls -l total 0 -rw-r--r-- 1 Unix_User+0 Unix_Group+0 0 Oct 3 13:38 owned_by_cedric -rw-r--r-- 1 Unix_User+0 Unix_Group+0 0 Oct 3 05:57 owned_by_root --cut--here--cut--here--cut--here-- Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Cygwin bash package data?
Good morning! Where (URL!!) can I find the package config for the Cygwin bash package? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
binfmt support in Cygwin?
Good morning! Does Cygwin support something like Linux binfmt, to register new binary types? Goal would be to recognise python compiled code, or https://wiki.debian.org/QemuUserEmulation Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Cygwin generates syscalls for *.lnk files on filesystems with native symlink support?
On Fri, 1 Sept 2023 at 13:00, Corinna Vinschen via Cygwin wrote: > > On Sep 1 06:23, Cedric Blancher via Cygwin wrote: > > Good morning! > > > > During a Cygwin 3.4.8-1.x86_64 debugging session I noticed something > > odd when I looked at the network traffic generated by one of our > > cluster nodes: > > It seems that for each call to a tool (i.e. starting "sed" from > > "bash") Cygwin searches for *.lnk files. > > > > Is this correct even when the filesystem in question has native > > symlink support (e.g. NFS)? > > Yes. During file handling, Cygwin doesn't know what filesystem a > file is on until it could actually open the file and request file > and filesystem info from the open handle. So if Cygwin couldn't open > "foo" because the NtCreateFile call returned with status > STATUS_OBJECT_PATH_NOT_FOUND or STATUS_OBJECT_NAME_NOT_FOUND, or > STATUS_NO_SUCH_FILE, or one of the countless other status codes the > kernel (or the driver) might return in case a file doesn't exist, > it will tack on .lnk and .exe and, for historical reasons, .exe.lnk, > and try again. Our concern here is that this produces an absurd amount of extra syscalls (6x more!!, compared to native POSIX), which hurts CIFS and NFS performance. The example with calling sed in a script is especially BAD, as trying to execute "sed" in a bash shell script leads it try for sed.exe, sed.sh, sed.bat, multiplied by looking for *.lnk files (i.e. sed.exe.lnk, sed.sh.lnk, sed.bat.lnk). And what finally made my brain go BOOM was the (IMO) absurd combination like "sed.bat.exe.lnk" Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Cygwin gdb to load debug into from Windows VC?
Good morning! Is it possible to load Visual Studio (VC19, VC22) debug info (*.pdb files) into Cygwin gdb? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: NFS mkfifo support in cygwin 3.5.0 Re: [ANNOUNCEMENT] cygwin 3.4.9-1
On Thu, 7 Sept 2023 at 23:20, Martin Wege via Cygwin wrote: > > On Wed, Sep 6, 2023 at 5:27 PM Corinna Vinschen via Cygwin-announce > via Cygwin wrote: > > > > The following packages have been uploaded to the Cygwin distribution: > > > > * cygwin-3.4.9-1 > > * cygwin-devel-3.4.9-1 > > * cygwin-doc-3.4.9-1 > > > > Bug Fixes > > - > > > > - Fix a bug introduced in cygwin 3.4.0 that switch_to_nat_pipe flag is > > not cleared properly when non-cygwin app is terminated in the case > > where pseudo console is not activated. > > > > - For the time being, disable creating special files using mknod/mkfifo > > on NFS. > > Addresses: https://cygwin.com/pipermail/cygwin/2023-August/254266.html > > While I am unhappy that you disabled this for Cygwin 3.4.9-1, we > noticed that you added mkfifo support for Cygwin 3.5.0 in > https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=622fb0776ea333dd708ff312f08ec98311138fbe > :-) > > Thanks > > We are testing 3.5.0-0.423.g7ced682549ae.x86_64 - which includes > 622fb0776ea333dd708ff312f08ec98311138fbe - right now, feedback will > come on Monday That's amazing news. But an old bug crept in: H is NFS file system $ uname -a CYGWIN_NT-10.0-19045 bastille0x44ef 3.5.0-0.423.g7ced682549ae.x86_64 2023-09-07 17:19 UTC x86_64 Cygwin $ mount C:/cygwin64/bin on /usr/bin type ntfs (binary,auto) C:/cygwin64/lib on /usr/lib type ntfs (binary,auto) C:/cygwin64 on / type ntfs (binary,auto) C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto) H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto) M: on /cygdrive/m type nfs (binary,posix=0,user,noumount,auto) N: on /cygdrive/n type nfs (binary,posix=0,user,noumount,auto) O: on /cygdrive/o type nfs (binary,posix=0,user,noumount,auto) P: on /cygdrive/p type nfs (binary,posix=0,user,noumount,auto) $ cd /cygdrive/h/tmp/ $ mkfifo f1.fifo ; echo $? 0 $ mkfifo -m 0600 f2.fifo ; echo $? mkfifo: cannot set permissions of 'f2.fifo': No such file or directory 1 So chmod() for a FIFO inode on NFS fails. Tested with MSFT NFSv3 and new builds of the NFSv4.1 ms-nfs41-client filesystems. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Cygwin generates syscalls for *.lnk files on filesystems with native symlink support?
Good morning! During a Cygwin 3.4.8-1.x86_64 debugging session I noticed something odd when I looked at the network traffic generated by one of our cluster nodes: It seems that for each call to a tool (i.e. starting "sed" from "bash") Cygwin searches for *.lnk files. Is this correct even when the filesystem in question has native symlink support (e.g. NFS)? Excerpt from tshark: 1746 3.800878384 192.168.2.152 → 192.168.2.150 NFS 298 V4 Call lookup LOOKUP DH: 0x49b89d4e/staden | LOOKUP sed 1747 3.800910202 192.168.2.150 → 192.168.2.152 NFS 318 V4 Reply (Call In 1746) lookup LOOKUP | LOOKUP Status: NFS4ERR_NOENT 1748 3.801293956 192.168.2.152 → 192.168.2.150 NFS 266 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.exe 1749 3.801319051 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1748) lookup LOOKUP Status: NFS4ERR_NOENT 1750 3.801636441 192.168.2.152 → 192.168.2.150 NFS 266 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.lnk 1751 3.801659474 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1750) lookup LOOKUP Status: NFS4ERR_NOENT 1752 3.801981533 192.168.2.152 → 192.168.2.150 NFS 270 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.exe.lnk 1753 3.802003824 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1752) lookup LOOKUP Status: NFS4ERR_NOENT 1754 3.810270585 192.168.2.152 → 192.168.2.150 NFS 266 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.bat 1755 3.810345438 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1754) lookup LOOKUP Status: NFS4ERR_NOENT 1756 3.810768112 192.168.2.152 → 192.168.2.150 NFS 270 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.bat.exe 1757 3.810795216 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1756) lookup LOOKUP Status: NFS4ERR_NOENT 1758 3.811234114 192.168.2.152 → 192.168.2.150 NFS 270 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.bat.lnk 1759 3.811254882 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1758) lookup LOOKUP Status: NFS4ERR_NOENT 1760 3.811728762 192.168.2.152 → 192.168.2.150 NFS 274 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.bat.exe.lnk 1761 3.811802518 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1760) lookup LOOKUP Status: NFS4ERR_NOENT 1762 3.820028572 192.168.2.152 → 192.168.2.150 NFS 266 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.sh 1763 3.820128384 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1762) lookup LOOKUP Status: NFS4ERR_NOENT 1764 3.820495809 192.168.2.152 → 192.168.2.150 NFS 270 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.sh.exe 1765 3.820560543 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1764) lookup LOOKUP Status: NFS4ERR_NOENT 1766 3.821000559 192.168.2.152 → 192.168.2.150 NFS 270 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.sh.lnk 1767 3.821071034 192.168.2.150 → 192.168.2.152 NFS 162 V4 Reply (Call In 1766) lookup LOOKUP Status: NFS4ERR_NOENT 1768 3.821519236 192.168.2.152 → 192.168.2.150 NFS 274 V4 Call lookup LOOKUP DH: 0x2cf62c0d/sed.sh.exe.lnk Also, searching for sed.sh.exe.lnk looks weird. Bug, or feature? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Documenting Cygwin on NFS, no longer only exchange-medium! Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a d
On Sun, 27 Aug 2023 at 14:35, Corinna Vinschen via Cygwin wrote: > > On Aug 26 19:44, Martin Wege via Cygwin wrote: > > On Fri, Aug 25, 2023 at 2:19 PM Corinna Vinschen via Cygwin > > wrote: > > > > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO, > > > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'". > > > > > > > > AFAIK there are (at least) these two options to fix the problems: > > > > 1. Check whether the filesystem for the fifos path is NFS > > > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it > > > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created > > > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|, > > > > |open()| etc. should treat this inode as FIFO. > > > > > > The downside is that it is not possible to diffentiate between Cygwin > > > FIFOs and real FIFOs created from the remote side in `ls -l' > > > output. Note that Cygwin returns the NFS stat info verbatim, so > > > a real FIFO is returned as a real FIFO: > > > > > > linux$ mkfifo bar > > > cygwin$ ls -l bar > > > prw-r--r-- 1 corinna vinschen 0 Aug 25 13:58 bar > > > > > > The idea was always to use NFS as exchange medium, but not as > > > installation medium for the entire distro or to keep Cygwin home > > > dirs on NFS. There were times where NFS was pretty unstable. > > > I used NFS for quite some time to build Cygwin packages, but at one > > > point I got trouble (performance problems with multiple concurrent > > > processes accessing an NFS share, build errors out of the blue), > > > so I switched to Samba shares, albeit grudgingly. I'm not yet > > > sure if the problems are fixed. At least a recent OpenSSH build > > > ran through without problems... > > > > Corinna, could you please update the Cygwin documentation then? We use > > NFS, both Windows builtin and CITI NFSv4.1, with Cygwin for years > > without major problems, so that comment about exchange-medium-only is > > clearly wrong! > > I'm talking about the idea how users would use NFS, and the idea was > that nobody in their right mind would use a crippled filesystem (from > the Windows POV) for the entire installation. Before you came up with > your complaints, nobody here even knew that somebody actually did that. What is missing in the Windows NFS filesystem to uncripple (if there is such a word in English) it from a Windows perspective? > > NFS is a niche filesystem on Windows and was never in the focus of > anybody here, except me, because I was toying around with it and was > happy to have it working. As far as I knew until a few days ago, > I was the only user of Cygwin's NFS support. Well, surprise, you were wrong. The Windows NFS filesystem has lots of users, but there are no centralised statistics. Just as an indicator, NFS on Windows has at least enough users to feed a couple of engineers at OpenText to work on their OpenText (once Hummingbird) NFS client product. Plus the two free implementations, Windows builtin NFSv3, and the CITI ms-nfsv41-client. > > As for FIFOs on NFS, as I wrote, they never worked as desired on NFS. > For kicks I tried back until Cygwin 3.3. Given this isn't a regression, > it won't be fixed for Cygwin 3.4. I am pretty sure this is a regression, as blast, staden and other software were built and operated on Cygwin. Just my PAIN job is how to rebuild our compute cluster (after total loss due water damage), new Windows, new blast, new Cygwin. And promptly we got hit by this issue. > > As Brian wrote in his reply to your mail, we are all volunteers here. > Everybody working on Cygwin is working in their spare time and/or on the > goodwill of their employer. > > Cygwin is open source. You can fix or implement stuff yourself and send > the patches to the cygwin-patches ML for review. Otherwise you will > have to let us, the people actually working on the project, decide when > and how something gets implemented. I agree. > > Having said that, I have a local patch enabling FIFOs on NFS. I might > push it at one point to the main branch, but the pressure you guys are > currently building up on this mailing list puts me right off. This was not intended, at least not from my side. But in any case, I humbly apologise. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: process substitution to create a virtual file doesn't work in chroot environment
On Mon, 28 Aug 2023 at 00:14, Cary Lewis via Cygwin wrote: > > In a cygwin process that is started either from mintty or bash directly the > following: > > $ user=234 > > $ ./cat <(echo $user) > 234 > works as expected. > > But after a chroot: > > $ chroot . ./bash > user=234 > $ ./cat <(echo $user) > ./cat: /dev/fd/63: No such file or directory > > In the directory I am chrooting in, I created a tmp folder, as well as > proc, proc/self, and proc/self/fd, and a dev directory. Can you strace -f the bash process to see what happens? How does ls -l /dev/ look like? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Fri, 25 Aug 2023 at 17:15, Roland Mainz via Cygwin wrote: > > On Fri, Aug 25, 2023 at 2:18 PM Corinna Vinschen via Cygwin > wrote: > > > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO, > > > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'". > > > > > > AFAIK there are (at least) these two options to fix the problems: > > > 1. Check whether the filesystem for the fifos path is NFS > > > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it > > > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created > > > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|, > > > |open()| etc. should treat this inode as FIFO. > > > > The downside is that it is not possible to diffentiate between Cygwin > > FIFOs and real FIFOs created from the remote side in `ls -l' > > output. Note that Cygwin returns the NFS stat info verbatim, so > > a real FIFO is returned as a real FIFO: > > > > linux$ mkfifo bar > > cygwin$ ls -l bar > > prw-r--r-- 1 corinna vinschen 0 Aug 25 13:58 bar > > I know. > > > The idea was always to use NFS as exchange medium, but not as > > installation medium for the entire distro or to keep Cygwin home > > dirs on NFS. There were times where NFS was pretty unstable. > > I used NFS for quite some time to build Cygwin packages, but at one > > point I got trouble (performance problems with multiple concurrent > > processes accessing an NFS share, build errors out of the blue), > > so I switched to Samba shares, albeit grudgingly. I'm not yet > > sure if the problems are fixed. At least a recent OpenSSH build > > ran through without problems... > > I think most issues have been fixed for the Microsoft NFSv3 client, > and for the CITI NFSv4.1 client (technically it's called > "ms-nfsv41-client") the situation is even better since it's > opensource, and we can fix problems even faster there. > From what I see the ms-nfsv41-client is stable enough for daily > routine usage, and I know that other institutions like DFG and CERN > use it for daily work too. The only nasty part is the damn lack of > documentation, and that there are no signed binaries, so any kernel > with UEFI/SecureBook cannot use them. > > > Anyway. How would you like to make sure that a Cygwin application > > can differ between real FIFOs and Cygwin FIFOs? > > For now I can provide a migration script, and in the medium term we > should get Microsoft to provide some kind of |mknod()| API. see below. > > > > 2. Check whether the filesystem for the fifos path is NFS > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > > |ENOSYS| (not implemented) > > > > I like the idea. > > See below. > > > > Better algorithm for [1] might be to check whether the inode is a > > > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|, > > > assuming this is more performant. > > > > Even better would be if we could just create and use real FIFOs > > on NFS(*). But while NtQueryEaFile can be used to fetch real > > NFS file info, and while NtCreateFile can be used to create real > > synmlinks via NFS, I don't see an interface resembling mknod/mkfifo. > > Looking at the ms-nfs41-client source code, there is no API for that *YET*. > > So my plan would be like this: > 1. Cygwin: implements the proposed devnodes-as-symlink emulation code, > if the env var CYGWIN has "nfs_emulate_dev_special_files_as_symlink" > set > 2. Cygwin: By default |mkfifo()| will fail with |ENOSYS| (not |EPERM|, > as we intend to fix the issue!) on NFS filesystems, unless > CYGWIN=nfs_emulate_dev_special_files_as_symlink I think the emulation should be on by default, or you break things. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Fri, 25 Aug 2023 at 14:26, Corinna Vinschen via Cygwin wrote: > > On Aug 25 14:18, Corinna Vinschen via Cygwin wrote: > > On Aug 23 01:05, Roland Mainz via Cygwin wrote: > > > 2. Check whether the filesystem for the fifos path is NFS > > > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with > > > |ENOSYS| (not implemented) > > > > I like the idea. > > EPERM, btw. See https://man7.org/linux/man-pages/man2/mknod.2.html > I am vehemently opposed to that idea. That will IMHO not only be the wrong solution, it will break actual things like autoconf on NFS - they create fifos for testing, but do not use them. If you just make this return EPERM, then you break stuff we have to rely on. mkfifo should just work with Cygwin on NFS. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: How to fix |mkfifo()| failure if |pathname| is on NFS ? / was: Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Thu, 24 Aug 2023 at 18:45, Martin Wege via Cygwin wrote:
>
> On Wed, Aug 23, 2023 at 1:06 AM Roland Mainz via Cygwin
> wrote:
> >
> > On Tue, Aug 22, 2023 at 4:52 PM Lavrentiev, Anton (NIH/NLM/NCBI) [C]
> > via Cygwin wrote:
> > > > FIFOs which don't make *any* sense
> > > > ... FWIW, a remote NFS fileystem.
> > >
> > > I got an impression that the OP is trying to deploy something (maybe the
> > > entire Cygwin) onto an
> > > NFS share. So the named FIFO "file" is also created in there.
> >
> > I agree with that impression. This is basically what large sites
> > (universities etc) do with UNIX and Linux: The machines mount an
> > user's ${HOMR} directory via automounter over NFS, and users are
> > discouraged (e.g. grumpy admin visiting you in person, blocking all
> > escape routes... =:-) ) from using the machine's local filesystems (in
> > Cygwin's case that includes "C:"!).
> >
> > In that case people want to use |mkfifo()|/|mkfifoat()| and/or
> > /usr/bin/mkfifo in their home directory, and don't expect that it does
> > not work.
> >
> > But that is what happens on Cygwin 3.4.8 right now, if someone tries
> > to do a |mkfifo()| on a NFS home directory (tested with MS NFSv3 and
> > CITI NFSv4 clients):
> > |mkfifo()| succeeds, but the resulting inode is *NOT* a FIFO as requested
> >
> > Example (/cygdrive/h/ is my home directory shared from my Linux machine):
> > snip
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ uname -a
> > CYGWIN_NT-10.0-19045 winkrakra1 3.4.8-1.x86_64 2023-08-17 17:02 UTC
> > x86_64 Cygwin
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ mount
> > C:/cygwin64/bin on /usr/bin type ntfs (binary,auto)
> > C:/cygwin64/lib on /usr/lib type ntfs (binary,auto)
> > C:/cygwin64 on / type ntfs (binary,auto)
> > C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> > H: on /cygdrive/h type nfs (binary,posix=0,user,noumount,auto)
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 1
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> > cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat -n cygwin_mkfifo_on_nfs.c
> > 1 #include
> > 2 #include
> > 3 #include
> > 4 #include
> > 5 #include
> > 6 #include
> > 7
> > 8 int main(int ac, char *av[])
> > 9 {
> > 10 (void)puts("# start");
> > 11
> > 12 if
> > (mkfifo("/cygdrive/h/work/cygwin_mkfifo_on_nfs/myfifo.fifo", 0) != 0)
> > 13 perror("mkfifo failed");
> > 14 (void)puts("# done.");
> > 15 return EXIT_SUCCESS;
> > 16 }
> > 17
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ gcc -g cygwin_mkfifo_on_nfs.c
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ./a
> > # start
> > # done.
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ ls -l
> > total 68
> > -rwxr-xr-x 1 Unix_User+0 Unix_Group+0 66951 Aug 23 00:12 a.exe
> > -rw-rw-rw- 1 Unix_User+0 Unix_Group+0 330 Aug 22 23:58
> > cygwin_mkfifo_on_nfs.c
> > lrwxrwxrwx 1 Unix_User+0 Unix_Group+011 Aug 23 00:12 myfifo.fifo
> > -> ':\0:c4:1000'
> >
> > roland_mainz@winkrakra1 /cygdrive/h/work/cygwin_mkfifo_on_nfs
> > $ cat > -bash: myfifo.fifo: No such file or directory
> > snip
> >
> > Note that Cygwin does not interpret the file |myfifo.fifo| as FIFO,
> > instead it comes back as a symlink "myfifo.fifo -> ':\0:c4:1000'".
> >
> > AFAIK there are (at least) these two options to fix the problems:
> > 1. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and if it is a symlink check if it
> > starts with ':\0:c4:' (assuming "c4" is the prefix for inodes created
> > with |mkfifo()|). If this condition is |true|, then cygwin |stat()|,
> > |open()| etc. should treat this inode as FIFO.
> > 2. Check whether the filesystem for the fifos path is NFS
> > (cgywin.dll's |fs.fs_is_nfs()|), and then just refuse |mkfifo()| with
> > |ENOSYS| (not implemented)
> >
> > Better algorithm for [1] might be to check whether the inode is a
> > symlink, and then do a |fs.fs_is_nfs()| on the symlinks |pathname|,
> > assuming this is more performant.
>
> We would like to see option 1 implemented,
+1
> maybe as an option in the
> CYGWIN environment variable
> (CYGWIN=nfs_emulate_dev_special_files_as_symlink ?)
+1
Ced
--
Cedric Blancher
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [EXTERNAL] Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Wed, 23 Aug 2023 at 17:44, Jeffrey Altman via Cygwin wrote: > > On 8/22/2023 10:52 AM, Lavrentiev, Anton (NIH/NLM/NCBI) [C] via Cygwin > wrote: > >> FIFOs which don't make *any* sense > >> ... FWIW, a remote NFS fileystem. > > I got an impression that the OP is trying to deploy something (maybe the > > entire Cygwin) onto an NFS share. So the named FIFO "file" is also created > > in there. > > > > It's pointless to assume that the FIFO can be used as a communication > > device between the hosts that can mount the share, but it can be quite > > feasible to envision a scenario, in which the same host opens the FIFO > > located on the share from two processes and establish the communication > > using that special "file" (which is basically a special data-less i-node). > > I've been following this thread with quite a bit of curiosity. For those > who do not know me, I'm the lead developer of the AFS filesystem on > Windows. There have been requests for more than two decades for AFS > clients to add support for locally created pipe files because AFS > volumes are often used as home directories (even on Windows) and so many > applications allocate a pipe in the home directory as a method of > inter-process communication or a lock. The problem is that even if the > intended usage of the file is entirely local, the directory object, the > directory entry and the allocated inode (or equivalent) is network > visible. What happens when the user executes two copies of an > application such as PyCharm on two separate machines sharing the same > home directory? Does the directory entry and inode get reused on startup > and/or deleted on exit? How does that impact the process instance on the > other machine? The conclusion I came to long ago is that if pipes are to > be implemented in a network file system namespace then the pipes must be > fully functional network pipes. In just about all cases applications can > be configured to use a non-default paths. In my opinion they should not > be placed in a shared file system. This is NOT what POSIX says. We're talking about mkfifo(), which creates pipes on the local system. The filesystem is just used to use filesystem paths for identification. If you want truly networked fifos, I suggest socket(), preferably with SCTP. But this is OFFTOPIC here. We just want mkfifo() to work on NFS. > > I've also been following this thread because both the Microsoft NFSv3 > and the Citi NFSv4 clients are rather incomplete filesystem > implementations from the perspective of the Installable File System > interface and the Multiple UNC Provider interface. Good. This is actually worth writing a bug report against the CITI NFSv4 client, because the Microsoft NFSv3 client is only extended or fixed if someone is willing to pay a five-digit euro sum to M$. Literally. We asked, and got a longer document describing the expenses. So better fix the NFSv4 client. > In my opinion, > Microsoft NFSv3 is the bare minimum that must be implemented for > Microsoft to advertise that NFSv3 is available on Windows. The NFSv3 > symlink interface leveraging extended attributes as a method of setting > and reading the target predates the introduction of > IO_REPARSE_TAG_SYMLINK for NTFS. Prior to the allocation of > IO_REPARSE_TAG_SYMLINK I obtained a proprietary tag allocation from > Microsoft for AFS symlinks but switched to IO_REPARSE_TAG_SYMLINK once > it was available because then AFS symlinks work the same as NTFS. > > The Citi NFSv4 implementation is not only incomplete from a Windows > interface perspective but Citi never obtained from Microsoft the > required filesystem registrations. Please elaborate > For example, it doesn't have a > Microsoft assigned FLT_FILESYSTEM_TYPE, WNNC network type, name or > NodeTypeCode values. FLT_FSTYPE_NFS, WNNC_NET_MS_NFS, and "nfs" are > specific to the Microsoft NFSv3 (aka \FileSystem\NfsRdr) implementation. This is not correct. The CITI NFSv4 client was originally written using $$$ from a grant from Microsoft, and is intended to be a drop-in replacement for the NFSv3 client. Hence it shares the same "nfs" name, uses the same extension apis (fsattr3), and should behave almost the same. > The NodeTypeCode range used by the Citi NFSv4 filesystem (0xFC00) is > outside the block allocated to third party filesystems and appears to be > from a published sample which could lead to corruption if two > filesystems allocating FileControlBlocks with the same NodeTypeCodes are > present on the system. The returned WNNC value is also from a sample. Please file a bug report. This should be fixable. > Perhaps the original poster knows where development of the Citi NFSv4 > client is currently taking place. The source code repos I've been able > to find do not have any commits since 2012 (tag v1.0.0) and have been > flagged as archival on GitHub. Seriously just my personal opinion, not speaking for my employer: 2013 on the github, and the DFG (Deutsche Forschungs
Cygwin testing coverage with MSFT NFSv3?
Good morning! Is Cygwin tested with NFSv3 shares on a regular basis, maybe CI? For example install Cygwin, then build all Cygwin software packages on a NFS share, instead of NTFS? Also, is there a script to checkout the sources of Cygwin software packages, and then build the software packages? (Cygwin software packages, not Cygwin itself!!) I am asking, because nasty bugs (at least 1, /usr/bin/mkfifo does not work on NFS share) have crept in. And I do not want to see those bugs again, even if I have to set up CI for each single Cygwin commit myself - i.e. grab commit, build Cygwin, install it, mount NFS share, build all software on that NFS share. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: [ANNOUNCEMENT] cygwin 3.4.8-1
On Thu, 17 Aug 2023 at 20:02, Corinna Vinschen via Cygwin-announce via Cygwin wrote: > > The following packages have been uploaded to the Cygwin distribution: > > * cygwin-3.4.8-1 > * cygwin-devel-3.4.8-1 > * cygwin-doc-3.4.8-1 > > Bug Fixes > - > > - Make safe for c89 compilations. > Addresses: https://cygwin.com/pipermail/cygwin-patches/2023q3/012308.html > > - Make gcc-specific code in compiler-agnostic. > Addresses: https://cygwin.com/pipermail/cygwin/2023-July/253927.html > > - Fix AT_EMPTY_PATH handling in fchmodat and fstatat if dirfd referres to > a file other than a directory > Addresses: https://cygwin.com/pipermail/cygwin-patches/2023q2/012306.html Wasn't this supposed to fix the mkfifo bug? Or just mkfifo on NTFS? We are still unable to use mkfifo on NFS (Windows builtin, I think you call it MSFT NFSv3) on Cygwin 3.4.8-1 Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: mkfifo: cannot set permissions of 'x.fifo': Not a directory
On Fri, 18 Aug 2023 at 10:35, Corinna Vinschen via Cygwin wrote: > > On Aug 18 06:02, Martin Wege via Cygwin wrote: > > On Mon, Aug 7, 2023 at 4:42 PM Martin Wege wrote: > > > > > > Hello, > > > > > > We get a weird mkfifo failure in Cygwin on NTFS: > > > > > > /usr/bin/mkfifo -m 600 x.fifo > > > mkfifo: cannot set permissions of 'x.fifo': Not a directory > > > > > > Is there a known workaround? So far named fifos cannot be created somehow. > > > > This is fixed for NTFS, but still broken for Windows builtin NFS v3 client: > > It was actually never supposed to work on NFS. Are you sure? AT AST and ARB (rDNA data cruncher) certainly have worked with Cygwin on NFS shares (Windows builtin, I think you call it MSFT NFSv3) in the past in our institute. Tested with Cygwin 3.4.8: AST is not even building without such support, and ARB throws nasty errors in the cluster run setup phase II, because it cannot create the command fifo used for each cluster node. > > Cygwin FIFOs are > created as symlinks of the type Windows shortcut with the R/O attribute > set. Those are only generated on NTFS and FAT filesystems. NFS symlinks > are generated using special magic. NFS doesn't support DOS attributes. So no Linux mkfifo support on NFS with Cygwin? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Mount Windows share for all users?
Good evening! Can I mount a Windows share (CIFS, builtin Windows, ms-nfsv41, ...) at Windows 10 system boot time, so all users (and Cygwin sessions) can use it? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Persistent and same Cygwin session for Windows login and ssh login?
Good evening! How can I get a Cygwin session for one Windows login, and get the same Cygwin session when I log into the same Windows machine via ssh? Also, where does Cygwin store session information? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
openldap server package?
Good afternoon! Does Cygwin has an openldap server package? So far I could only find the client libraries and tool, but not the slapd server. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: lsof for Windows processes?
Good afternoon!
On Fri, 11 Aug 2023 at 18:21, Brian Inglis wrote:
>
> On 2023-08-11 09:45, Cedric Blancher via Cygwin wrote:
> > Does Cygwin have a tool like lsof which can list which files or
> > directories are in use by a Windows process?
>
> You can check these things yourself using the Cygwin Search Packages page:
>
> https://cygwin.com/packages/
>
> type "usr/bin/lsof" and hit Enter:
>
> https://cygwin.com/cgi-bin2/package-grep.cgi?grep=usr%2Fbin%2Ffuser
>
> Search Results
>
> Found 0 matches for usr/bin/lsof
>
> and the same on the command line:
>
> $ cygcheck -p usr/bin/lsof
> Found 0 matches for usr/bin/lsof
>
> but Cygwin does have fuser:
>
> $ cygcheck -p usr/bin/fuser
> Found 6 matches for usr/bin/fuser
> cygfuse-3.2.0-2 - cygfuse: Cygwin interface to Windows FUSE providers
> cygfuse-3.2.0-3 - cygfuse: Cygwin interface to Windows FUSE providers
> psmisc-debuginfo-22.20-1 - psmisc-debuginfo: Debug info for psmisc
> psmisc-debuginfo-23.4-4 - psmisc-debuginfo: Debug info for psmisc
> psmisc-22.20-1 - psmisc: Utilities for managing processes on your system
> psmisc-23.4-4 - psmisc: Utilities for managing processes on your system
>
> and Windows has openfiles:
>
> $ cmd /c help openfiles
>
> OPENFILES /parameter [arguments]
>
> Description:
> Enables an administrator to list or disconnect files and folders
> that have been opened on a system.
>
> Parameter List:
> /Disconnect Disconnects one or more open files.
>
> /Query Displays files opened locally or from shared
> folders.
>
> /Local Enables / Disables the display of local open files.
>
> /? Displays this help message.
>
> Examples:
> OPENFILES /Disconnect /?
> OPENFILES /Query /?
> OPENFILES /Local /?
Thank you.
But neither openfiles or fuser seem to work in this case, even if I
have a file open in my bash shell, and call fuser in the same bash
script:
bash -c 'command exec {n}
[https://plus.google.com/u/0/+CedricBlancher/]
Institute Pasteur
--
Problem reports: https://cygwin.com/problems.html
FAQ: https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
lsof for Windows processes?
Good evening! Does Cygwin have a tool like lsof which can list which files or directories are in use by a Windows process? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Run a Cygwin process with "higher" scheduler priority
Good evening! How can we set the scheduler priority for a Cygwin process to "higher" than normal on start? Basically we want a Cygwin compatible script version of the Windows task manager feature Name/PID--->Details--->Set priority--->Higher Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: how to set mount posix=1 option for an existing bind mount?
On Tue, 8 Aug 2023 at 20:53, Corinna Vinschen wrote: > > On Aug 8 16:52, Cedric Blancher via Cygwin wrote: > > Good afternoon! > > > > How do I set mount posix=1 option for an existing bind mount? mount -o > > remount does not work. > > Did you read https://cygwin.com/cygwin-ug-net/mount.html? > > Mount points created by mount(1) only exist in the current user session, > and they will disappear with the last process of that user. Where are the sessions stored? How can I list, create and destroy sessions? > Our mount(1) doesn't support remount, so you have to unmount and mount > again. Again, this only affects the current user session, not other > users on the system. > > Last but not least, posix=1 is default and is not shown in the output > of mount(1). Only posix=0 will be printed. So this: > > C:/foo on /home/bar type smbfs (binary,user,bind) > > means the mount point is mounted with posix=1. But on my Cygwin 4.7 installation C: defaults to posix=1 posix=0 is default for /cygwin/c: $ mount C:/cygwin64/bin on /usr/bin type ntfs (binary,auto) C:/cygwin64/lib on /usr/lib type ntfs (binary,auto) C:/cygwin64 on / type ntfs (binary,auto) C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto) Why? Where do I change the default? Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
how to set mount posix=1 option for an existing bind mount?
Good afternoon! How do I set mount posix=1 option for an existing bind mount? mount -o remount does not work. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Re: Kernel stack trace for Winows 10 blue screen when running Cygwin?
On Mon, 7 Aug 2023 at 11:55, Corinna Vinschen wrote: > > On Aug 7 11:29, Cedric Blancher via Cygwin wrote: > > Forwarding to cygwin@cygwin.com > > > > Honestly I find it deeply concerning that a plain, unprivileged user > > can Bluescreen a machine, and more so that it happens during normal > > Cygwin usage. > > Same here. Cygwin is userspace only! > > If any call in Cygwin can generate a bluescreen, it's a bug in the > kernel or in the driver. Naturally, we have neither control over the > kernel, nor over the NTFS driver. You might want to open a support case > with Microsoft. Well, a colleague is handling that. The feedback sent to her however was that: - Cygwin is not a Windows product - We should WSL instead - Cygwin might make use private Windows apis - We should run this by the "Cygwin company" It is my turn now to provide a kernel stack trace to prove them wrong - IF I can manage to make one. That's why I am asking for help here. HELP! Ced -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
Fwd: Kernel stack trace for Winows 10 blue screen when running Cygwin?
Forwarding to cygwin@cygwin.com Honestly I find it deeply concerning that a plain, unprivileged user can Bluescreen a machine, and more so that it happens during normal Cygwin usage. Ced -- Forwarded message - From: Cedric Blancher Date: Sun, 6 Aug 2023 at 23:56 Subject: Kernel stack trace for Winows 10 blue screen when running Cygwin? To: Good evening! How can we get a Windows kernel stack trace for a blue screen - aka Windows kernel crash? We're experiencing regular blue screens when we run Cygwin git operations for a large repository (~6GB of text files, sort of DNA data). Usually it goes booom in the ntfs module with a git clone, or git commit, but a more detailed stack trace would be greatly appreciated. Ced -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Cedric Blancher [https://plus.google.com/u/0/+CedricBlancher/] Institute Pasteur -- Problem reports: https://cygwin.com/problems.html FAQ: https://cygwin.com/faq/ Documentation:https://cygwin.com/docs.html Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple
