Re: /usr/bin/sg in Cygwin?

[Corinna Vinschen via Cygwin]

On Apr 22 07:55, Cedric Blancher via Cygwin wrote:
> On Mon, 22 Apr 2024 at 01:20, Dan Shelton via Cygwin  
> wrote:
> >
> > On Sat, 20 Apr 2024 at 07:39, Brian Inglis via Cygwin  
> > wrote:
> > >
> > > On 2024-04-19 17:47, Dan Shelton via Cygwin wrote:
> > > > On Fri, 23 Feb 2024 at 22:25, Dan Shelton wrote:
> > > >> Is there a package which provides /usr/bin/sg (execute shell commands
> > > >> in a different group)?
> > >
> > > The POSIX standard command is newgrp - install cygwin-doc to see Cygwin 
> > > (and
> > > newlib libc) man pages (and info, html, PDF docs), man-pages-posix to see 
> > > POSIX
> > > man pages, and man-pages-linux if you want to see recent Linux release 
> > > man pages.
> > >
> > > To see possibly relevant commands, run:
> > >
> > > $ apropos -s 1,1p group
> > > chgrp (1)- change group ownership
> > > chgrp (1p)   - change the file group ownership
> > > chown (1)- change file owner and group
> > > g3topbm (1)  - convert a Group 3 fax file into a PBM image
> > > groups (1)   - print the groups a user is in
> > > id (1)   - print real and effective user and group IDs
> > > make (1) - GNU Make utility to maintain groups of programs
> > > make (1p)- maintain, update, and regenerate groups of programs
> > > mkgroup (1)  - Write /etc/group-like output to stdout
> > > newgrp (1)   - change primary group for a command
> > > newgrp (1p)  - change to a new group
> > > pbmtog3 (1)  - convert a PBM image into a Group 3 MH fax file
> >
> > newgrp(1) is USELESS. It only opens an interactive shell, but does not
> > allow the user to execute a non-interactive script with the requested
> > group like bash -c does.
> 
> Linux /usr/bin/sg source is in
> https://github.com/shadow-maint/shadow/blob/master/src/newgrp.c
> So this is just a packaging issue that whoever does the Cygwin newgrp
> package has to package /usr/bin/sg too

newgrp(1) is a Cygwin util from the base package:

  $ cygcheck -f /usr/bin/newgrp
  cygwin-3.6.0-0.109.ga0a25849f9dd

Reason being that the functionality under Windows is pretty limited
compared to "real" POSIX systems...

https://cygwin.com/cygwin-ug-net/newgrp.html

...and fetching the default environment of the user (to implement the
dash option '-') is pretty different from any other known POSIX system,
having to use Windows functions:

  https://cygwin.com/cgit/newlib-cygwin/tree/winsup/utils/newgrp.c

If anybody thinks he or she can provide a useful shadow-utils
package, feel free.  But it might not be worth the effort, probably.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: /usr/bin/sg in Cygwin?

[Corinna Vinschen via Cygwin]

On Apr 21 21:35, Eliot Moss via Cygwin wrote:
> On 4/21/2024 7:20 PM, Dan Shelton via Cygwin wrote:
> > On Sat, 20 Apr 2024 at 07:39, Brian Inglis via Cygwin  
> > wrote:
> > > 
> > > On 2024-04-19 17:47, Dan Shelton via Cygwin wrote:
> > > > On Fri, 23 Feb 2024 at 22:25, Dan Shelton wrote:
> > > > > Is there a package which provides /usr/bin/sg (execute shell commands
> > > > > in a different group)?
> > > 
> > > The POSIX standard command is newgrp - install cygwin-doc to see Cygwin 
> > > (and
> > > newlib libc) man pages (and info, html, PDF docs), man-pages-posix to see 
> > > POSIX
> > > man pages, and man-pages-linux if you want to see recent Linux release 
> > > man pages.
> > > 
> > > To see possibly relevant commands, run:
> > > 
> > > $ apropos -s 1,1p group
> > > chgrp (1)- change group ownership
> > > chgrp (1p)   - change the file group ownership
> > > chown (1)- change file owner and group
> > > g3topbm (1)  - convert a Group 3 fax file into a PBM image
> > > groups (1)   - print the groups a user is in
> > > id (1)   - print real and effective user and group IDs
> > > make (1) - GNU Make utility to maintain groups of programs
> > > make (1p)- maintain, update, and regenerate groups of programs
> > > mkgroup (1)  - Write /etc/group-like output to stdout
> > > newgrp (1)   - change primary group for a command
> > > newgrp (1p)  - change to a new group
> > > pbmtog3 (1)  - convert a PBM image into a Group 3 MH fax file
> > 
> > newgrp(1) is USELESS. It only opens an interactive shell, but does not
> > allow the user to execute a non-interactive script with the requested
> > group like bash -c does.
> 
> ??
> 
> The man pages has:
> 
>  newgrp [-] [group] [command [args...]]
> 
> which implies to me that you can give a command.
> 
> Did that not work for you?  EM

WJFFM:

  $ id
  uid=1049577(corinna) gid=1049701(vinschen) 
groups=1049701(vinschen),545(Users),14(REMOTE INTERACTIVE 
LOGON),4(INTERACTIVE),11(Authenticated Users),15(This 
Organization),4095(CurrentSession),66048(LOCAL),1049089(Domain 
Users),70145(Authentication authority asserted identity),1049148(Denied RODC 
Password Replication Group),401408(Medium Mandatory Level)
  $ newgrp Users id -g
  545


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: 3.5.x regression: misquoting command line arguments from native processes

[Corinna Vinschen via Cygwin]

Hi David,

On Apr 20 08:43, David Allsopp via Cygwin wrote:
> Hi Corinna,
> 
> > On Apr  9 22:38, Corinna Vinschen via Cygwin wrote:
> > > On Apr  3 16:53, David Allsopp via Cygwin wrote:
> > > > I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> > > > a CI system lagging behind, we've only just discovered.
> > > > [...]
> > > > $ ./t.exe 'C:\Devel\реализация-mingw64\flexdll\flexdll_mingw64.o'
> > > > stat: cannot stat
> > > > '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> > > > No such file or directory
> > >
> > > Thanks a lot for the STC!
> > >
> > > I think I fixed that for 3.5.4.  I pushed a patch and the test release
> > > cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
> > > for testing in an hour or two.
> > >
> > > Please give it a try.
> >
> > Sorry for nagging, but do you have some feedback, be it bad or good?
> 
> Sorry for having needed nagging! It does indeed fix it, thank you -
> our smoke-test Bactrian camels can be restored 

Great, nice to read this. I'll keep my camel testcase around, just in
case :)


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: 3.5.x regression: misquoting command line arguments from native processes

[Corinna Vinschen via Cygwin]

Hi David,

On Apr  9 22:38, Corinna Vinschen via Cygwin wrote:
> On Apr  3 16:53, David Allsopp via Cygwin wrote:
> > I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> > a CI system lagging behind, we've only just discovered.
> > [...]
> > $ ./t.exe 'C:\Devel\реализация-mingw64\flexdll\flexdll_mingw64.o'
> > stat: cannot stat
> > '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> > No such file or directory
> 
> Thanks a lot for the STC!
> 
> I think I fixed that for 3.5.4.  I pushed a patch and the test release
> cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
> for testing in an hour or two.
> 
> Please give it a try.

Sorry for nagging, but do you have some feedback, be it bad or good?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Win32 FILE_BASIC_INFORMATION.*Time.QuadPart - which value should be used if filesystem does not support it, so Cygwin /usr/bin/stat lists the value as '-'?

[Corinna Vinschen via Cygwin]

On Apr  9 22:30, Brian Inglis via Cygwin wrote:
> On 2024-04-09 15:14, Corinna Vinschen via Cygwin wrote:
> > On Apr  5 04:26, Martin Wege via Cygwin wrote:
> > > On Fri, Apr 5, 2024 at 2:05 AM Martin Wege  
> > > wrote:
> > > > I have problems with debugging, so a quick help would be appreciated,
> > > > as I cannot figure this out after several hours of digging.
> > > > 
> > > > Cygwin /usr/bin/stat returns "Birth: -" for some files. Which value
> > > > must the CreationTime member of FILE_BASIC_INFORMATION have to cause
> > > > /usr/bin/stat ti return "-"? 0, -1, or something else?
> > > 
> > > In a related matter:
> > > The Win32 FILE_BASIC_INFORMATION structure defines four time values:
> > > 
> > > LARGE_INTEGER CreationTime;
> > > LARGE_INTEGER LastAccessTime;
> > > LARGE_INTEGER LastWriteTime;
> > > LARGE_INTEGER ChangeTime;
> > > 
> > > How can a filesystem indicate if it does not support a particular
> > > timestamp, such as ChangeTime? Should ChangeTime.QuadPart then be -1,
> > > -2 or 0, or another value?
> > 
> > I'm not aware of a filesystem not supporting ChangeTime, that is,
> > st_ctime.  Usually only CreationTime (st_birthtime) is missing.
> 
> R/O media like CD/DVD-R or FS w/o write support?

Oh yes, that makes sense, CDFS and the likes of them.

> > I think setting the timestamp to 0 works for indicating that this kind
> > of timestamp is not supported.  Cygwin is handling Windows timestamps
> > this way, but I can't find this in documentation ATM.
> 
> See upthread?:
> 
> Caller or application can set 0 to mean keep/return current value, caller or
> driver can set -1 to mean don't update/return current value:
> 
> https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/ns-wdm-_file_basic_information#remarks

I'm aware of that.  This remarks session is copy/pasted in other MSFT
articles, too.  But it doesn't really answer Martin's question.

It describes what a consumer (application or driver) is supposed to set
the timestamps to when passing timestamps to a filesystem driver via
ZwSetInformationFile (or its friends on the lower OS levels).

It does *not* describe the other direction of the call stack, i. e.,
what a filesystem driver is supposed to set a timestamp to, if its
underlying filesystem doesn't support this kind of timestamp.

And that's the direction Martin is asking about.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: /proc//{cwd, root} links to for cygrunsrv, daemons, and shells

[Corinna Vinschen]

On Apr  9 10:38, Brian Inglis wrote:
> On 2024-04-09 07:08, Corinna Vinschen wrote:
> > That's typically a permission problem.  On Linux you get something like
> > 
> >ls: cannot read symbolic link '/proc/1/cwd': Permission denied
> 
> Thanks Corinna,
> 
> That now makes sense, as Cygwin ps -a and btop showed the processes,
> although procps and top did not, and other info is visible, I never thought
> about permissions as there were links, but I see from elevated admin sh:
> [...]
> so I think perms on these should be 440 or 550 not 444 or 555, but that may
> involve a lot of work to decide that for each entry?

Not really.  Have a look into fhandler/proc.cc, fhandler/process.cc,
etc.  We can add a permisions member to struct virt_tab_t and add
this as static info to every member in the list.  Doesn't sound overly
complicated to me (*nudge, nudge*).

Changing  to a "Permission denied" when trying to open a
virtual symlink may be a bit more involved, but maybe not very much.


Corinna

Re: Win32 FILE_BASIC_INFORMATION.*Time.QuadPart - which value should be used if filesystem does not support it, so Cygwin /usr/bin/stat lists the value as '-'?

[Corinna Vinschen via Cygwin]

On Apr  5 04:26, Martin Wege via Cygwin wrote:
> On Fri, Apr 5, 2024 at 2:05 AM Martin Wege  wrote:
> >
> > Hello,
> >
> > I have problems with debugging, so a quick help would be appreciated,
> > as I cannot figure this out after several hours of digging.
> >
> > Cygwin /usr/bin/stat returns "Birth: -" for some files. Which value
> > must the CreationTime member of FILE_BASIC_INFORMATION have to cause
> > /usr/bin/stat ti return "-"? 0, -1, or something else?
> 
> In a related matter:
> The Win32 FILE_BASIC_INFORMATION structure defines four time values:
> 
> LARGE_INTEGER CreationTime;
> LARGE_INTEGER LastAccessTime;
> LARGE_INTEGER LastWriteTime;
> LARGE_INTEGER ChangeTime;
> 
> How can a filesystem indicate if it does not support a particular
> timestamp, such as ChangeTime? Should ChangeTime.QuadPart then be -1,
> -2 or 0, or another value?

I'm not aware of a filesystem not supporting ChangeTime, that is,
st_ctime.  Usually only CreationTime (st_birthtime) is missing.

I think setting the timestamp to 0 works for indicating that this kind
of timestamp is not supported.  Cygwin is handling Windows timestamps
this way, but I can't find this in documentation ATM.

But the FS driver can also just fill the CreationTime field with the
same value as LastWriteTime or ChangeTime.  MSFT NFS does that.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: 3.5.x regression: misquoting command line arguments from native processes

[Corinna Vinschen via Cygwin]

Hi David,

On Apr  3 16:53, David Allsopp via Cygwin wrote:
> I have what appears to be a regression in Cygwin 3.5.0 which, owing to
> a CI system lagging behind, we've only just discovered.
> [...]
> $ ./t.exe 'C:\Devel\реализация-mingw64\flexdll\flexdll_mingw64.o'
> stat: cannot stat
> '"C:\Devel\'$'\360\237\220\253''реализация-mingw64\flexdll\flexdll_mingw64.o':
> No such file or directory

Thanks a lot for the STC!

I think I fixed that for 3.5.4.  I pushed a patch and the test release
cygwin-3.6.0-0.115.g579064bf4d40 is just building and should be ready
for testing in an hour or two.

Please give it a try.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: /proc//{cwd, root} links to for cygrunsrv, daemons, and shells

[Corinna Vinschen]

On Apr  7 13:34, Brian Inglis wrote:
> ISTM anomalous that for cygrunsrv, daemons, cron processes, and shells
> /proc//{cwd,root} have bad symlinks to "", normally a process
> or exe status:
> 
> /proc/732/exe   -> /usr/bin/cygrunsrv
> /proc/732/root  -> 
> /proc/732/cwd   -> 
> |  /proc/733/exe   -> /usr/sbin/cygserver
>  ->/proc/733/root  -> 
>/proc/733/cwd   -> 
> /proc/740/exe   -> /usr/bin/cygrunsrv
> /proc/740/root  -> 
> /proc/740/cwd   -> 
> |  /proc/741/exe   -> /usr/sbin/syslog-ng
>  ->/proc/741/root  -> 
>/proc/741/cwd   -> 
> /proc/748/exe   -> /usr/bin/cygrunsrv
> /proc/748/root  -> 
> /proc/748/cwd   -> 
> |  /proc/749/exe   -> /usr/sbin/cron
>  ->/proc/749/root  -> 
>/proc/749/cwd   -> 
>|  /proc/2080/exe  -> /usr/sbin/cron
> ->/proc/2080/root -> 
>   /proc/2080/cwd  -> 
>   |  /proc/2082/exe  -> /usr/bin/bash
>->/proc/2082/root -> 
>  /proc/2082/cwd  -> 
> 
> Should we consider changing that to root "/", or nothing, null, or something
> meaningful?

That's typically a permission problem.  On Linux you get something like

  ls: cannot read symbolic link '/proc/1/cwd': Permission denied

But on Cygwin the content of those links require to open the processes'
signal pipe and send/receive a message containing the information.  I
didn't look into the code for a while but it seems we don't check why we
couldn't connect to a process to fetch the info.  IIRC the current
fhandler_process framework doesn't have a way to communicate that
info.

If you want to change that, feel free!


Corinna

Re: cygwin 3.5.3-1 : "Device or resource busy" from 'find {pagefile,swapfile}.sys'

[Corinna Vinschen via Cygwin]

Hi Bruce,

On Apr  4 07:22, Bruce Jerrick via Cygwin wrote:
> As of cygwin 3.5.3-1, 'find' reports "Device or resource busy" and
> exits with status 1 on these two files:
> 
>   $ find /cygdrive/c/{pagefile,swapfile}.sys
>   /cygdrive/c/pagefile.sys
>   find: ‘/cygdrive/c/pagefile.sys’: Device or resource busy
>   /cygdrive/c/swapfile.sys
>   find: ‘/cygdrive/c/swapfile.sys’: Device or resource busy
> 
>   $ echo $?
>   1

Thanks for the report.

Should be fixed in cygwin-3.6.0-0.113.g38b513983590, which is
just being built and should be ready in an hour or two.

This will go into 3.5.4, of course.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin file prefetch, block sizes?

[Corinna Vinschen via Cygwin]

On Apr  3 10:14, Corinna Vinschen via Cygwin wrote:
>   
> https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_fs_sector_size_information
> 
>   If the filesystem is local and SSINFO_FLAGS_NO_SEEK_PENALTY is set, we
>   could stick to 64K.
> 
>   Otherwise the PhysicalBytesPerSectorForPerformance member might be
>   helpful I guess.  Needs checking, of course.

It's not helpful. This is the output for NTFS:
  (gdb) p ffssi
  $5 = {LogicalBytesPerSector = 512, PhysicalBytesPerSectorForAtomicity = 512,
PhysicalBytesPerSectorForPerformance = 512,
FileSystemEffectivePhysicalBytesPerSectorForAtomicity = 512, Flags = 11,
ByteOffsetForSectorAlignment = 0, ByteOffsetForPartitionAlignment = 0}

D'oh

>   If this isn't any good, we can still fallback to
>   FILE_FS_FULL_SIZE_INFORMATION as in fhandler_base::fstatvfs_by_handle,
>   
> https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fhandler/disk_file.cc#n661

So ffsi.BytesPerSector * ffsi.SectorsPerAllocationUnit is it then.

But:

> On Apr  3 00:35, Martin Wege via Cygwin wrote:
> > While I can understand the motivation, FAT32 on multi-GB-devices
> > having 64k block size, and Win32 API on Win95/98/ME/Win7 being
> > optimized to that insane block size, it is absolutely WRONG with
> > today's NTFS and even more so with ReFS.

So this has supposedly changed with Win8. Where's that publically
documented?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Implementing /bin/ionice, ioprio_set() support with FILE_IO_PRIORITY_HINT_INFO?

[Corinna Vinschen via Cygwin]

On Apr  3 03:51, Martin Wege via Cygwin wrote:
> Hello,
> 
> could Cygwin implement support for /usr/bin/ionice and ioprio_set()
> via FILE_IO_PRIORITY_HINT_INFO?

Not ionice, the tool, but only ioprio_get/ioprio_set, the API.

The problem is the functionality bit allowing to set the priority
in arbitrary processes.

And, as usual, https://cygwin.com/acronyms/#SHTDI

I would again like to stress the fact that we're all volunteers here,
and we're a sadly small bunch of people actually contributing to the
core code.

It's easy to ask for new functionality, but it needs time to implement
stuff and we all have another job keeping the money coming, and that
job takes priority.

We don't want to see stuff like the xz debacle in Cygwin, right?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin file prefetch, block sizes?

[Corinna Vinschen via Cygwin]

On Apr  3 00:35, Martin Wege via Cygwin wrote:
> On Tue, Apr 2, 2024 at 3:17 PM Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Apr  2 02:04, Martin Wege via Cygwin wrote:
> > > Hello,
> > >
> > > Is there any document which describes how Cygwin and Win32 file
> > > prefetch and readahead work, and which sizes are used (e.g. always
> > > read one full page even if only 16 bytes are requested?)?
> >
> > I'm not aware of any docs, but again, keep in mind that Cygwin is a
> > usersapce DLL. We basically do what Windows does for low-level file
> > access.
> >
> > > Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that
> > > mean the file's block size is really 64k? Is this info per filesystem,
> > > or hardcoded in Cygwin?
> >
> > Hardcoded in Cygwin since 2017, based on a discussion in terms of
> > file access performance, especially when using stdio.h functions:
> >
> >   https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c
> 
> OUCH.
> 
> While I can understand the motivation, FAT32 on multi-GB-devices
> having 64k block size, and Win32 API on Win95/98/ME/Win7 being
> optimized to that insane block size, it is absolutely WRONG with
> today's NTFS and even more so with ReFS. This only works if you stream
> files, but as soon as you are doing random read/writes the performance
> is terrible due to cache thrashing. That could explain the many
> complaints about Cygwin's IO performance.

The above patch *only* sets stat::st_blksize to 64K. Nothing else
happens!

This usually means that stdio.h functions use this size for their buffer
and readahead.  It doesn't affect direct calls to read(2)/write(2) and
fread(3)/fwrite(3) at all!

> So, what can be done? I'm not a benchmarking guru, so I'd like to
> propose to add a tunable called EXPERIMENTAL_PREFERRED_IO_BLKSIZE to

No.

We have two ways to handle this *iff* there's really a reason to
handle this.

- Either we just lower PREFERRED_IO_BLKSIZE to 4K or 8K, but that's
  kind of bad in terms of pipes, the clipboard, etc.

- So we keep PREFERRED_IO_BLKSIZE at 64K but don't use it for disk
  files.  Rather, we read this info from the filesystem:

  
https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntddk/ns-ntddk-_file_fs_sector_size_information

  If the filesystem is local and SSINFO_FLAGS_NO_SEEK_PENALTY is set, we
  could stick to 64K.

  Otherwise the PhysicalBytesPerSectorForPerformance member might be
  helpful I guess.  Needs checking, of course.

  If this isn't any good, we can still fallback to
  FILE_FS_FULL_SIZE_INFORMATION as in fhandler_base::fstatvfs_by_handle,
  
https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fhandler/disk_file.cc#n661

Corinna


-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: xargs completely broken under cygwin 3.6.0-0.108.gb7f5a33200a9

[Corinna Vinschen via Cygwin]

On Apr  2 08:38, Bruce Jerrick via Cygwin wrote:
> 'xargs' is back to working with cygwin-3.6.0-0.109.ga0a25849f9dd .
> 
> Thanks for the quick fix!
> -- Bruce

Thanks for the report and testing!


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin file prefetch, block sizes?

[Corinna Vinschen via Cygwin]

On Apr  2 02:04, Martin Wege via Cygwin wrote:
> Hello,
> 
> Is there any document which describes how Cygwin and Win32 file
> prefetch and readahead work, and which sizes are used (e.g. always
> read one full page even if only 16 bytes are requested?)?

I'm not aware of any docs, but again, keep in mind that Cygwin is a
usersapce DLL. We basically do what Windows does for low-level file
access.

> Quick /usr/bin/stat /etc/profile returns "IO Block: 65536". Does that
> mean the file's block size is really 64k? Is this info per filesystem,
> or hardcoded in Cygwin?

Hardcoded in Cygwin since 2017, based on a discussion in terms of
file access performance, especially when using stdio.h functions:

  https://cygwin.com/cgit/newlib-cygwin/commit/?id=7bef7db5ccd9c

Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: xargs completely broken under cygwin 3.6.0-0.108.gb7f5a33200a9

[Corinna Vinschen via Cygwin]

On Apr  2 12:44, Corinna Vinschen via Cygwin wrote:
> On Apr  2 01:42, Bruce Jerrick via Cygwin wrote:
> > I just did binary-search regression tests on the available
> > old test versions, and found this:
> > 
> > OK:  cygwin-3.6.0-0.86.gbfe2790e7bc4.tar.xz
> > BAD: cygwin-3.6.0-0.92.g8bd6ba8f16ec.tar.xz
> 
> Thanks, I found the offending patch, but I have to think about a
> solution.
> 
> Stay tuned,
> Corinna

Just building cygwin-3.6.0-0.109.ga0a25849f9dd, should be up in
an hour or two. Please give it a try.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: xargs completely broken under cygwin 3.6.0-0.108.gb7f5a33200a9

[Corinna Vinschen via Cygwin]

On Apr  2 01:42, Bruce Jerrick via Cygwin wrote:
> I just did binary-search regression tests on the available
> old test versions, and found this:
> 
> OK:  cygwin-3.6.0-0.86.gbfe2790e7bc4.tar.xz
> BAD: cygwin-3.6.0-0.92.g8bd6ba8f16ec.tar.xz

Thanks, I found the offending patch, but I have to think about a
solution.

Stay tuned,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [tz] Ubuntu drops old-style links - tzdata split test package

[Corinna Vinschen via Cygwin-apps]

On Mar 28 02:25, Brian Inglis via Cygwin-apps wrote:
> I have released and announced a test package of tzdata 2024a-2 split into
> three install packages: base tzdata, optional tzdata-right, and redundant
> tzdata-posix, each containing all the legacy zones so that tzset continues
> to work as before.
> 
> I could not reduce the base installed zones by many, because most were used
> by tzset, but I did drop a couple of large zone source files, produced by
> the build, that were previously included to allow users to see the source
> zones, rules, and links in effect, saving ~1MB, and dropping the overall
> default base installed file sizes by ~80% to ~20% of current, and download
> tar size by ~60% to ~40%; for all zones aggregate total installed file sizes
> are dropped by ~35% to ~65%, and download tar sizes by ~30% to ~70% of
> current:
> 
> install   tar   tzdata
>  721KB  172KB   base
>  984KB   78KB   right
>  669KB   74KB   posix
> 1367KB  444KB   source
> 3667KB  452KB   current
> 
> Please check out the announcement, cygwin list echo, source and install
> package summary web pages, cygport changes, setup entries, scallywag builds,
> and let me know if there is anything you see that could benefit from
> improvement.

Fedora Rawhide is not following this scheme.  For F40 and F41 it still
prepares single tzdata packages.  FWIW, OpenSuSE also only comes with a
single timezone package in Tumbleweed.

Comparing the Cygwin and the Fedora package, the only differences are:

 Cygwin comes with two files not in the Fedora package:

   /usr/share/zoneinfo/rearguard.zi
   /usr/share/zoneinfo/zonenow.tab

 Fedora comes with two files not in the Cygwin package:

   /usr/share/zoneinfo/leap-seconds.list
   /usr/share/zoneinfo/posixrules

That's all.  And given that space is not one of the major limiting
factors anymore...

  cyg$ du -sh /usr/share/zoneinfo
  6.6M/usr/share/zoneinfo

  fed$ du -sh /usr/share/zoneinfo
  4.6M  /usr/share/zoneinfo

...I do wonder a bit if this split is really necessary after all.


Corinna

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Corinna Vinschen via Cygwin]

On Mar 24 15:42, Christian Franke via Cygwin wrote:
> Corinna Vinschen via Cygwin wrote:
> > On Mar 23 10:57, Christian Franke via Cygwin wrote:
> > > On Mar 21 16:15, Corinna Vinschen via Cygwin wrote:
> > > > On Mar 21 09:58, Christian Franke via Cygwin wrote:
> > > > > > > ..
> > > > I just gave it a try on W11. The results are even more funny than I
> > > > anticipated:
> > > > 
> > > > ...
> > > > 
> > > > So, yeah, with your observations especially on older W10 versions and
> > > > with 8.1 doing the same thing, I guess we can safely drop the extra call
> > > > to RtlGetNtVersionNumbers now.
> > > Or:
> > > Keep RtlGetVersionNumbers, compare result with RtlGetVersion and emit 
> > > debug
> > > output if both differ. This would allow to diagnose a possible future
> > > reappearance of the issue.
> > Good idea! Do you want to create a patch?
> > 
> 
> I tried, but without success. Unfortunately debug_printf() does not work
> because wincap.init() is called before get_cygwin_startup_info() which calls
> strace.activate().

small_printf should work, just needs "\r\n" in the Windows console.

But, anyway, nothing speaks aginst just sticking to the current code.
It works and probably continues to work...


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [tz] Ubuntu drops old-style links

[Corinna Vinschen via Cygwin-apps]

On Mar 23 10:38, Brian Inglis via Cygwin-apps wrote:
> It looks to me that tzset.c prioritizes the Windows label over the country,
> and it may be a better match prioritizing the country over the label, if the
> country is not 001/"", nor ZZ, which are the generic entries.

The Windows timezone is the relevant setting in the first place becasue
that's what indicvates the actual local time *as the user chose*.
The territory should only be a secondary hint to choose the right
POSIX entry.

For instance, I know people always using UTC, independently of their
territory setting. If the territory rules, this user choice would be
broken in Cygwin.

> It also is not clear what tzset should do when tzmap has a list of zones to
> choose from, for example:
> 
>   { L"Mountain Standard Time", L"CA", L"America/Edmonton
> America/Cambridge_Bay America/Inuvik" },
>   { L"Mountain Standard Time", L"US", L"America/Denver America/Boise" },
>   { L"US Mountain Standard Time", L"CA", L"America/Creston
> America/Dawson_Creek America/Fort_Nelson" },
> 
> it currently just prints the first, but perhaps it should print all relevant
> entries and the caller should handle the alternatives?

tzset is called from the shell default profile.  It has to use exactly
one, valid entry, so time works as desired without forcing interactivity.

If the user doesn't like it, the user can always override tzset's choice
in her own profile.


Corinna

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Corinna Vinschen via Cygwin]

On Mar 23 10:57, Christian Franke via Cygwin wrote:
> On Mar 21 16:15, Corinna Vinschen via Cygwin wrote:
> > On Mar 21 09:58, Christian Franke via Cygwin wrote:
> > > > > ..
> > I just gave it a try on W11. The results are even more funny than I
> > anticipated:
> > 
> > ...
> > 
> > So, yeah, with your observations especially on older W10 versions and
> > with 8.1 doing the same thing, I guess we can safely drop the extra call
> > to RtlGetNtVersionNumbers now.
> 
> Or:
> Keep RtlGetVersionNumbers, compare result with RtlGetVersion and emit debug
> output if both differ. This would allow to diagnose a possible future
> reappearance of the issue.

Good idea! Do you want to create a patch?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [tz] Ubuntu drops old-style links

[Corinna Vinschen via Cygwin-apps]

On Mar 22 10:02, Brian Inglis via Cygwin-apps wrote:
> On 2024-03-21 03:36, Corinna Vinschen via Cygwin-apps wrote:
> > We're generating the conversion from Windows to POSIX timezone via
> > the conversion table from unicode.org:
> > 
> > https://cygwin.com/cgit/newlib-cygwin/tree/winsup/utils/tzmap-from-unicode.org
> > 
> > Plus a few (7, actually) mappings the Unicode consortium missed in
> > the list (or maybe they are available in the meantime, needs checking).
> > This is the minimum list of timezone info we need in the tzdata DB.
> 
> I generated tzmap.h and generated differences since the last update cldr ~40.
> I also searched in the latest for matches for each field attached as first.
> 
> I do not know if they will be of help as I see you have already looked at 
> tzmap.
> 
> It looks as if the match might better prioritize country code over Windows 
> label.

Which match?  I'm not sure what you're trying to tell me.

Basically, we want to generate a POSIX timezone from the current user's
Windows timezone.  This boils down to four questions:

- Is the creation of tzmap.h from unicode.org via the
  tzmap-from-unicode.org script the right thing to do or not?

- If it's the wrong thing to do, what other source do you propose and do
  you have a script to perform the conversion from this source to a
  valid tzmap.h file?

- Otherwise, is the current tzmap-from-unicode.org right or wrong in
  adding these old extra timezone/territory settings, or is even
  some combination missing?

- If so, would you mind to send a patch to fix tzmap-from-unicode.org
  accordingly?


Corinna

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Corinna Vinschen via Cygwin]

On Mar 22 18:13, Ray Satiro via Cygwin wrote:
> On 3/21/2024 11:15 AM, Corinna Vinschen via Cygwin wrote:
> >https://cygwin.com/cgit/newlib-cygwin/commit/?id=48511f3d3847c
> 
> The code in that commit doesn't look right. RtlGetNtVersionNumbers is a void
> function

The code doesn't request a return value.

> and the third parameter may not be just the build number.

Fixed a month later:
https://cygwin.com/cgit/newlib-cygwin/commit/?id=63f19278b998b


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

sourceware test, please ignore

[Corinna Vinschen via Cygwin]

someproc 1234 find_fast_cwd: WARNING: Couldn't compute FAST_CWD
pointer.  Please report this problem to the public mailing list
cygwin@cygwin.com


-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Corinna Vinschen via Cygwin]

On Mar 21 09:58, Christian Franke via Cygwin wrote:
> Corinna Vinschen via Cygwin wrote:
> > On Mar 20 12:39, Christian Franke via Cygwin wrote:
> > > Corinna Vinschen via Cygwin wrote:
> > > > You have to create an application with an application manifest not
> > > > supporting your OS.
> > > > 
> > > > For Cygwin apps, this occured when you built, say, an executable under
> > > > Windows 8.1 before Windows 10 support was added to the Cygwin toolchain:
> > > > the manifest linked to the Cygwin executable didn't yet contain a GUID
> > > > entry for Windows 10 support.
> > > > 
> > > > In this case, RtlGetVersion returns an OS version 6.3 even when running
> > > > under the 10.0 kernel.  This behaviour exists back 'til Windows Vista.
> > > Could not reproduce the latter on Win10. I tested with recent Win10 and
> > > Win11 and also found a Win10 1511 (and Slackware 1.1.2, Win3.1, OS/2, ...)
> > > in my VM image museum.
> > > 
> > > Regardless of the exe manifest, RtlGetVersion and RtlGetNtVersionNumbers
> > > return the correct versions:
> > > 10.0.22621 (Win11 22H2)
> > > 10.0.19045 (Win10 22H2)
> > > 10.0.10586 (Win10 1511)
> > > 
> > > Without a manifest, GetVersionEx returns:
> > > 6.2.9200 (Win8)

I just gave it a try on W11. The results are even more funny than I
anticipated:

I created a simple application just calling GetVersionEx, RtlGetVersion
and RtlGetNtVersionNumbers.

Linked with our Cygwin default manifest claiming W10 compatibility, the
result is the expected:

  GetVersionEx  : 10.0.22631
  RtlGetVersion : 10.0.22631
  RtlGetNtVersionNumbers: 10.0.22631

The "Operating system context" in Task Manager is empty.

Next I linked against a Windows 8.1 manifest:

  GetVersionEx  : 6.3.9600
  RtlGetVersion : 10.0.22631
  RtlGetNtVersionNumbers: 10.0.22631

So GetVersionEx reports Windows 8.1, RtlGetVersion/ RtlGetNtVersionNumbers
both report W10.  The "Operating system context" in Task Manager reports

  "Windows 8.1"

No surprise there.

Next I linked against a Windows 7 manifest:

  GetVersionEx  : 6.2.9200
  RtlGetVersion : 10.0.22631
  RtlGetNtVersionNumbers: 10.0.22631

So GetVersionEx reports Windows 8, not Windows 7.

However, the "Operating system context" in Task Manager reports

  "Windows 7"

I also tried this with a Vista manifest:

  GetVersionEx  : 6.2.9200
  RtlGetVersion : 10.0.22631
  RtlGetNtVersionNumbers: 10.0.22631

  "Windows Vista"

So Task Manager reports the right context per the manifest, but
GetVersionEx doesn't go below Windows 8.

Same goes for Windows 10...

  GetVersionEx  : 6.2.9200
  RtlGetVersion : 10.0.19045
  RtlGetNtVersionNumbers: 10.0.19045

  "Windows Vista"

as well as for Windows 8.1:

  GetVersionEx  : 6.2.9200
  RtlGetVersion : 6.3.9600
  RtlGetNtVersionNumbers: 6.3.9600

  "Windows Vista"

So, yeah, with your observations especially on older W10 versions and
with 8.1 doing the same thing, I guess we can safely drop the extra call
to RtlGetNtVersionNumbers now.  After such a long time, I don't know
on which version of Windows we observed the problem.

For those interested in patch forensics, I searched the archives and
came up with two mail threads referring to GetVersionEx and RtlGetVersion:

  https://cygwin.com/pipermail/cygwin/2013-November/211795.html
  https://cygwin.com/pipermail/cygwin/2014-June/215836.html

Unfortunately I found *no* thread talking about RtlGetNtVersionNumbers,
so the only information we have now is the commit message of

  https://cygwin.com/cgit/newlib-cygwin/commit/?id=48511f3d3847c



Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [tz] Ubuntu drops old-style links

[Corinna Vinschen via Cygwin-apps]

On Mar 20 14:59, Brian Inglis via Cygwin-apps wrote:
> On 2024-03-19 02:19, brian.ing...@systematicsw.ab.ca wrote:
> > On 2024-03-18 21:12, Matt Johnson-Pint via tz wrote:
> > > I just learned that Ubuntu Noble (24.04) decided to intentionally
> > > split the tzdata package.  Old-style links such as US/Eastern are no
> > > longer included by default, but are available in the tzdata-legacy
> > > package instead.
> > > 
> > > Just thought I'd share.  I wonder if other distributions / platforms
> > > / libraries will follow suit.  What do y'all think?
> > > 
> > > See:
> > > <https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/2058249>
> > > <https://stackoverflow.com/questions/78180695/unrecognized-time-zone>
> 
> > I've been looking at that to reduce Cygwin CI and embedded build server
> > setup overhead by limiting base install data to:
> > 
> > - only the zones in zonenow.tab;
> > - optionally those in zone1970.tab not in zonenow.tab;
> > - additionally those in zone.tab in backward, and/or backzone;
> > - possibly those not in zone.tab, only in backward, and/or backzone;
> > - additions those in posix subtree, or right subtree.
> 
> As tzdata maintainer, I would like to discuss on this list first, to take
> advantage of a wide variety of experience in different environments with
> different practices and requirements, before making more definite proposals
> on the public list.
> 
> Please see the attached log for prioritized subsets of tzdata for 
> consideration:
> [...]
> What would the impact on tzset conversion from Windows to Olson tzdb be?
> We would probably have to add all of these in to any minimal install.
> I think I looked at that somewhere, sometime, not too long ago.

We're generating the conversion from Windows to POSIX timezone via
the conversion table from unicode.org:

https://cygwin.com/cgit/newlib-cygwin/tree/winsup/utils/tzmap-from-unicode.org

Plus a few (7, actually) mappings the Unicode consortium missed in
the list (or maybe they are available in the meantime, needs checking).
This is the minimum list of timezone info we need in the tzdata DB.

Corinna

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Corinna Vinschen via Cygwin]

On Mar 20 12:39, Christian Franke via Cygwin wrote:
> Corinna Vinschen via Cygwin wrote:
> > You have to create an application with an application manifest not
> > supporting your OS.
> > 
> > For Cygwin apps, this occured when you built, say, an executable under
> > Windows 8.1 before Windows 10 support was added to the Cygwin toolchain:
> > the manifest linked to the Cygwin executable didn't yet contain a GUID
> > entry for Windows 10 support.
> > 
> > In this case, RtlGetVersion returns an OS version 6.3 even when running
> > under the 10.0 kernel.  This behaviour exists back 'til Windows Vista.
> 
> Could not reproduce the latter on Win10. I tested with recent Win10 and
> Win11 and also found a Win10 1511 (and Slackware 1.1.2, Win3.1, OS/2, ...)
> in my VM image museum.
> 
> Regardless of the exe manifest, RtlGetVersion and RtlGetNtVersionNumbers
> return the correct versions:
> 10.0.22621 (Win11 22H2)
> 10.0.19045 (Win10 22H2)
> 10.0.10586 (Win10 1511)
> 
> Without a manifest, GetVersionEx returns:
> 6.2.9200 (Win8)

Please check on commit 48511f3d3847c.  It was a real, existing problem
at the time.  I wouldn't have added the RtlGetNtVersionNumbers call
just for fun.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Corinna Vinschen via Cygwin]

On Mar 19 09:18, Bill Stewart via Cygwin wrote:
> On Tue, Mar 19, 2024 at 9:01 AM Richard Campbell wrote:
> 
> On Tue, Mar 19, 2024 at 9:04 AM Bill Stewart via Cygwin 
> > wrote:
> >
> > > Can you please clarify the circumstances under which the RtlGetVersion
> > > function "may not return the correct values"?
> >
> > "Originally, using RtlGetVersion instead of GetVersionEx was supposed to
> > fix the fact that GetVersionInfo returns the wrong kernel version if the
> > executable has been built with an old manifest (or none at all), starting
> > with Windows 8.1.  Either this never really worked as desired and our
> > testing was flawed, or this has been changed again with Windows 10, so
> > that RtlGetVersion does the kernel faking twist as well.  Since we're
> > only reading the value in the first process in a process tree. the entire
> > process tree is running with a wrong OS version information in that case.
> >
> > Fortunately, the (undocumented) RtlGetNtVersionNumbers function is not
> > affected by this nonsense, so we simply override the OS version info
> > fields with the correct values now."
> >
> > https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=48511f3d3847c35352d09cded56e25f0c1b22bc9
> >
> 
> Interesting. I have not yet been able to find a scenario where the
> RtlGetVersion function gets "manifested" like GetVersionEx.
> 
> I wrote a small Win32 console utility for displaying and testing OS
> information (requires Windows Vista/Server 2008 or later):
> 
> https://github.com/Bill-Stewart/osinfo
> 
> It uses RtlGetVersion, and this function works correctly for me in all
> current Windows versions (Windows 10, Server 2016, Windows 11, Server 2019,
> Server 2022, etc.).
> 
> I'm not sure of the exact scenario that led to the "RtlGetVersion is
> subject to manifesting" conclusion, but I can't reproduce it.

You have to create an application with an application manifest not
supporting your OS.

For Cygwin apps, this occured when you built, say, an executable under
Windows 8.1 before Windows 10 support was added to the Cygwin toolchain:
the manifest linked to the Cygwin executable didn't yet contain a GUID
entry for Windows 10 support.

In this case, RtlGetVersion returns an OS version 6.3 even when running
under the 10.0 kernel.  This behaviour exists back 'til Windows Vista.

Fortunately Microsoft didn't change the required manifest GUID entry
since the introduction of Windows 10.  Even Windows 11 is still using
the same GUID.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: No Win ACLs for NFS? Re: ACEs and ACLs

[Corinna Vinschen via Cygwin]

On Mar 19 08:55, Cedric Blancher via Cygwin wrote:
> On Mon, 18 Mar 2024 at 15:43, Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Mar 18 08:30, J. Terry Corbet via Cygwin wrote:
> > > Thank you for the greatly needed assistance, but the reference to which 
> > > you
> > > have pointed me says that noacl will be ignored in the case of ntfs file
> > > systems.
> >
> > No, it doesn't say that.  It says
> >
> >   "The flag is ignored on NFS filesystems."
> >   ^^^
> > not NTFS
> 
> Do ACLs work for NFS in Cygwin, or are they turned off for NFS?

NFS uses the unofficial fattr3 interface to fetch real stat(2) info from
the remote FS, see
https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/local_includes/nfs.h
https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/nfs.cc#n19

> I'm also asking because the ms-nfs41-client Windows NFSv4.1 driver now
> has ACL support (like the Exceed NFSv4 driver), but it would be
> frustrating if Cygwin just turns this off.

Cygwin "doesn't turn them off".  Cygwin recognizes the filesystem as
being an NFS filesystem and uses special non-Windowsy access methods
provided by the MS_NFS client.

If you want ACL support for the NFSv4 client, I made a couple of
suggestions how to integrate stuff in Cygwin in November:
https://cygwin.com/pipermail/cygwin-developers/2023-November/012692.html


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] winsup/cygwin/fhandler/proc.cc: format_proc_cpuinfo() Linux 6.8 cpuinfo flags

[Corinna Vinschen]

On Mar 18 11:21, Brian Inglis wrote:
> On 2024-03-18 09:45, Corinna Vinschen wrote:
> > I see.  I just don't understands the difference between, say,
> > 
> >ftcprint (features1, 21, "avx512ifma");   /* vec int FMA */
> > + /*  ftcprint (features1, 22, ""); */  /* unused */
> >ftcprint (features1, 23, "clflushopt");   /* cache line flush opt */
> > 
> > and
> > 
> >ftcprint (features1,  3, "xsaves");   /* xsaves/xrstors */
> > + /*  ftcprint (features1,  4, "xfd"); */   /* eXtended Feature Disabling */
> > 
> > The latter makes sense, of course, but why is the first comment "unused",
> > rather than something like "PCOMMIT instruction" as in the cpuid output?
> > 
> > Note that I'm not saying that you have to change that, but I would like
> > to understand it.
> 
> Hi Corinna,
> 
> The cpuid output is not always up to date with the kernel, and there are a
> lot of bits defined, so if Linux does not use the bit I will not mention it,
> except where uses and visibility may vary because of merge/patch revisions,
> as happened recently with shstk and lam handling changes.
> 
> The cpuinfo capflags are generated by running the Linux build script
> mkcapflags.sh, from the feature symbol suffix, unless overriden by a quoted
> string at the start of the comment, and "" suppresses cpuinfo flag output.
> 
> In my weekly pulls of relevant rc sources, I generate a couple of summary
> logs to merge the cpuinfo capflags with the comments and feature bits, and
> diff everything relevant vs the previous tagged release.
> 
> I keep an eye on those diffs, and when the next release is no longer a
> candidate, I pull up the Linux changes and look at how they can be added to
> Cygwin.
> 
> I sometimes add features commented out to document bits used in a feature
> word, but not yet displayed on cpuinfo, just to make it easier to compare
> with Linux, or more obvious that an unused bit has not been missed.
> The latest additions are the result of uncertainties raised during my last
> cross check.
> 
> Below is a sample of the info used to display Linux cpuinfo flags, which I
> use to support Cygwin's, relevant to those you mentioned.
> Linux feature word 9 bit 22 is unused, and word 10 bit 4 is not displayed.
> [...]
> Many of the synthetic Linux features and flags are derived from hw boot or
> MSR info, which we can not yet access from Cygwin, so I ignore those
> changes, unless the feature can be derived from information readily
> available as a user in the cpu, Windows, or Cygwin.
> 
> I cross check the Linux and Cygwin sources occasionally to ensure I have not
> missed anything added or removed, spelling changes, Linux tweaks, or
> readability.
> 
> I have so far ignored feature disabling depending on conditions, and cpu
> errata checks and output, as some of that requires MSR info or low level
> access.
> 
> I have looked at trying to extract or generate tables from the Linux sources
> to drive our cpuinfo, use gcc cpuid and cpuinfo headers, automate or at
> least simplify maintenance, but there are many exceptions which we can not
> determine to output, and Intel's practices are not as architecturally
> structured as AMD's, so require code to decide.]

Thanks for the explanation.  Pushed.


Corinna

Re: ACEs and ACLs

[Corinna Vinschen via Cygwin]

On Mar 18 09:23, J. Terry Corbet via Cygwin wrote:
> 
> Sorry, 84-yr old eyes sometimes don't work as well.  Thanks for confirming
> that  nothing has changed with regards to these matters; clearly it is some
> change in the way Windows 11 tries to cooperate with Windows 10 in the case
> of mapped network drives being using in the file sharing mode wherein remote
> users must have ids and passwords on the target drives -- which they do and
> always have, but the key ACE entry known as NT AUTHORITY\Authenticated Users
> is correctly specified after performing an icacls /reset, but is not longer
> correctly set after editing a file with vim across the network.  I'll keep
> looking and trying to  learn.  Thank you.

Please add the "noacl" flag where you need it and try again.  The result
is using the standard Windows security, so you should see what you
expect in that case.


Thanks,
Corinna


-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] winsup/cygwin/fhandler/proc.cc: format_proc_cpuinfo() Linux 6.8 cpuinfo flags

[Corinna Vinschen]

On Mar 18 08:10, Brian Inglis wrote:
> On 2024-03-18 03:33, Corinna Vinschen wrote:
> > Hi Brian,
> > 
> > On Mar 16 10:44, Brian Inglis wrote:
> > > add Linux 6.8 cpuinfo flags:
> > > Intel 0x0007:1 eax:17 fredFlexible Return and Event 
> > > Delivery;
> > > AMD   0x801f   eax:4  sev_snp SEV secure nested paging;
> > > document unused and some unprinted bits that could look like omissions;
> > > fix typos and misalignments;
> > 
> > I'm a bit puzzled about the "unused" slots.  You're adding them
> > only in some places.  What makes them "look like omissions"?
> 
> Mainly because single bits are omitted, presumably because they do not want
> to pollute the symbol space with as yet unused features, just as they do not
> output all features as cpuinfo flags, until it indicates something about the
> build and/or system.
> 
> Compare the minimal common standard feature bits defined in the gcc lib
> cpuid.h and gcc cpuinfo.h headers, with Linux cpuinfo cpufeatures.h, and the
> output of the cpuid utility, where almost all bits in older cpuid entries
> are defined.

I see.  I just don't understands the difference between, say,

  ftcprint (features1, 21, "avx512ifma");   /* vec int FMA */
+ /*  ftcprint (features1, 22, ""); */  /* unused */
  ftcprint (features1, 23, "clflushopt");   /* cache line flush opt */

and

  ftcprint (features1,  3, "xsaves");   /* xsaves/xrstors */
+ /*  ftcprint (features1,  4, "xfd"); */   /* eXtended Feature Disabling */

The latter makes sense, of course, but why is the first comment "unused",
rather than something like "PCOMMIT instruction" as in the cpuid output?

Note that I'm not saying that you have to change that, but I would like
to understand it.


Thanks,
Corinna

Re: ACEs and ACLs

[Corinna Vinschen via Cygwin]

On Mar 18 08:30, J. Terry Corbet via Cygwin wrote:
> Thank you for the greatly needed assistance, but the reference to which you
> have pointed me says that noacl will be ignored in the case of ntfs file
> systems.

No, it doesn't say that.  It says

  "The flag is ignored on NFS filesystems."
  ^^^
not NTFS

> All of mine are and that has not changed, neither has the default
> entry in fstab, which seems always to have been:
> 
> none /cygdrive cygdrive binary, posix=0, user 0 0

Well, the code in question hasn't changed for years either.


¯\_(ツ)_/¯
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ACEs and ACLs

[Corinna Vinschen via Cygwin]

On Mar 16 18:05, J. Terry Corbet via Cygwin wrote:
> [...]
> And here is the status that icacls reports back on the original, owning
> workstation
> after having use vim to modify the two files from that remote workstation.
> 
> FileExp.txt NULL SID:(DENY)(Rc,S,REA,WEA,X,DC)
>     NW10\tcorbet:(DENY)(S,RD,WD,AD,REA,WEA,X,DC)
>     NW10\tcorbet:(D,Rc,WDAC,WO,RA,WA)
>     NW10\None:(Rc,S,RA)
>     NT AUTHORITY\Authenticated Users:(RX,W)
>     NT AUTHORITY\SYSTEM:(RX,W)
>     BUILTIN\Administrators:(RX,W)
>     BUILTIN\Users:(RX)
>     Everyone:(Rc,S,RA)
> 
> vimtest.txt NULL SID:(DENY)(Rc,S,WEA,X,DC)
>     NW10\tcorbet:(R,W,D,WDAC,WO)
>     NW10\None:(DENY)(S,X)
>     NT AUTHORITY\Authenticated Users:(DENY)(S,X)
>     NT AUTHORITY\SYSTEM:(DENY)(S,X)
>     BUILTIN\Administrators:(DENY)(S,X)
>     BUILTIN\Users:(DENY)(S,X)
>     NW10\None:(RX)
>     NT AUTHORITY\Authenticated Users:(RX,W)
>     NT AUTHORITY\SYSTEM:(RX,W)
>     BUILTIN\Administrators:(RX,W)
>     BUILTIN\Users:(RX)
>     Everyone:(R)
> 
> If my understanding is correct concerning the precedence handling of an
> ACL with multiple ACEs for the same user/ID, this result from grep
> on the original, owning workstation would not surprise you:
> 
> F:\Dev\cygshoot>grep foo fileexp.txt
> grep: fileexp.txt: Permission denied
> 
> but it blows me completely away.  Clearly I no longer have an environment
> in which I can work on any file from any workstation using any Cygwin
> utilities.
> 
> What have I messed up?

The problem is that your identity is based on the SID of every single
machine, and the machines don't know the SIDs of other machines.  The
default ACL created in Cygwin is emulating POSIX permissions.  This
becomes a problem when sharing files between machines not in the
same Windows domain.

The workaround is not to use POSIX permissions on shares.  Create
matching mount points in /etc/fstab or /etc/fstab.d/ and add the
"noacl" mount flag:

  https://cygwin.com/cygwin-ug-net/using.html#mount-table

Alternatively, you can also just add an fstab entry for the cygdrive
prefix which adds the "noacl" flag, see

  https://cygwin.com/cygwin-ug-net/using.html#cygdrive

but keep in mind that this also affects local paths if you access
them via the cygdrive prefix.


HTH,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Additional odd behavior after my upgrade to the latest CYGWIN64 version

[Corinna Vinschen via Cygwin]

On Mar 16 16:49, Michael Goldshteyn via Cygwin wrote:
> Cygwin handles (filename) case insensitivity in a very weird way at the
> Windows command prompt:
> --
> c:\Users\Michael>set CYGWIN=
> 
> c:\Users\Michael>printf "%q " 'a' 'A' "a" "A" a A
> a A a A a A
> 
> REM OK, so far so good, but, now:
> c:\Users\Michael>set CYGWIN=glob:ignorecase
> 
> C:\Users\Michael>printf "%q " 'a' 'A' "a" "A" a A
> a a a a a A
> 
> REM So, if I don't put a command line arg into any sort of quotes, it gets
> lower-cased for cygwin apps (from the Windows command line).
> REM I should also add that there is no file named 'a' or 'A' in the current
> working directory.
> 
> REM This behavior, of course, doesn't happen in Cygwin bash:
> --
> C:\Users\Michael>bash
> Michael@lambda /cygdrive/c/Users/Michael
> $ echo "$CYGWIN"
> glob:ignorecase
> Michael@lambda /cygdrive/c/Users/Michael
> $ printf "%q " 'a' 'A' "a" "A" a A
> a A a A a A
> $ # Output looks correct, quoted or not
> --
> The unsolicited conversion of case for quoted string is very odd, to say
> the least. If you remove the glob:ignorecase from the CYGWIN env var
> definition, use of filenames on Windows with Cygwin tools becomes
> completely case sensitive, which is very undesirable when compared to the
> fact that other Windows tools do not care about file case (i.e., case
> insensitive, but preserving, as MS puts it).

This was an ill-advised optimization attempt in glob.  I revert the
patch and pushed the change(*), so this will be fixed in Cygwin 3.5.2.

For the time being, you can also test this by installing the just
building Cygwin test release 3.6.0-0.85.g89afbb8d8af2, which should
be available in an hour or two.


Thanks,
Corinna


(*) https://cygwin.com/cgit/newlib-cygwin/commit/?id=70375b2205

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] winsup/cygwin/fhandler/proc.cc: format_proc_cpuinfo() Linux 6.8 cpuinfo flags

[Corinna Vinschen]

Hi Brian,

On Mar 16 10:44, Brian Inglis wrote:
> add Linux 6.8 cpuinfo flags:
> Intel 0x0007:1 eax:17 fredFlexible Return and Event 
> Delivery;
> AMD   0x801f   eax:4  sev_snp SEV secure nested paging;
> document unused and some unprinted bits that could look like omissions;
> fix typos and misalignments;

I'm a bit puzzled about the "unused" slots.  You're adding them
only in some places.  What makes them "look like omissions"?


Thanks,
Corinna

Re: newlib-cygwin build fails on dumper

[Corinna Vinschen]

On Mar 14 10:10, Brian Inglis wrote:
> Hi folks,
> 
> During newlib-cygwin build to test patches, with latest current stable
> packages as of last weekend, and yesterday's repo main/master, get a
> warning, then errors building dumper:
> 
> ...
> 
>   CC   libc/reent/libc_a-getentropyr.o
> /usr/src/newlib-cygwin/newlib/libc/reent/getentropyr.c: In function 
> ‘_getentropy_r’:
> /usr/src/newlib-cygwin/newlib/libc/reent/getentropyr.c:48:14: warning:
> implicit declaration of function ‘_getentropy’; did you mean ‘getentropy’?
> [-Wimplicit-function-declaration]
>48 |   if ((ret = _getentropy (buf, buflen)) == -1 && errno != 0)
>   |  ^~~
>   |  getentropy
> 
> ...
> 
>   CXX  dumper-dumper.o
> In file included from /usr/src/newlib-cygwin/winsup/utils/dumper.cc:23:
> /usr/include/bfd.h:2748:1: error: expected initializer before
> ‘ATTRIBUTE_WARN_UNUSED_RESULT’
>  2748 | ATTRIBUTE_WARN_UNUSED_RESULT;
>   | ^~~~
> /usr/include/bfd.h:2751:1: error: expected initializer before

I'm pretty sure this isn't the latest newlib-cygwin main.

The getentropy warning has been fixed on 2023-09-25 by commit
a9e8e3d1cb82 ("newlib: Add missing prototype for _getentropy")

The ATTRIBUTE_WARN_UNUSED_RESULT error message has been fixed on
2024-02-12 by commit 10c8c1cf4f94 ("include/ansidecl.h: import from
binutils-gdb")


Corinna

Re: Unable to 'git push' to /git/cygwin-packages/*

[Corinna Vinschen via Cygwin-apps]

On Mar 14 16:07, Jon Turney via Cygwin-apps wrote:
> On 14/03/2024 15:39, Mark Geisert via Cygwin-apps wrote:
> > On 3/14/2024 2:42 AM, Jon Turney via Cygwin-apps wrote:
> > > On 14/03/2024 05:45, Mark Geisert via Cygwin-apps wrote:
> > > > Hi folks,
> > > > I'm getting the error:
> > > > 
> > > > fatal: remote error: service not enabled: /git/cygwin-packages/sshfs
> > > > 
> > > > when I attempt 'git push' to that repository.  The same happens
> > > > with all the repositories for my packages.  It's been this way
> > > > for a couple days at least.
> > > > 
> > > > Have I forgotten some step in the connection at my end?  I'm
> > > > running ssh-agent.
> > > 
> > > This is probably due to some recent changes made on sourceware.
> > > Apologies for the inconvenience.
> 
> I forget to ask when was the last time this worked for you, so maybe
> assuming this is related is premature.
> 
> > > What is the repository URL you are trying to push to (git remote -v)?
> > 
> > /usr/src/upstaging/sshfs git remote -v
> > origin git://cygwin.com/git/cygwin-packages/sshfs (fetch)
> > origin git://cygwin.com/git/cygwin-packages/sshfs (push)
> 
> This maybe looks like pilot error.
> 
> We don't allow pushing using the git:// protocol (since this protocol
> doesn't do any authorization, pushes with a it are very rarely enabled)
> 
> 
> I suggest you need to do
> 
>   git push ssh://cyg...@cygwin.com:git/cygwin-packages/sshfs
> 
> to push successfully.
> 
> If that works, I suggest you memorialize that by doing
> 
>   git remote set-url origin --push
> ssh://cyg...@cygwin.com:git/cygwin-packages/sshfs
> 
> which will cause git to automatically use the ssh URL with a simple 'git
> push'.

You may also want to use https:// rather than git:// for reading
the repository these days, given the insecurity of the git protocol.

Corinna

Re: [PATCH] Cygwin: pipe: Restore non-blocking mode which was reset for non-cygwin app.

[Corinna Vinschen]

On Mar 12 08:17, Takashi Yano wrote:
> Subject: [PATCH v4] Cygwin: pipe: Make sure to set read pipe non-blocking for
>  cygwin apps.
> 
> If pipe reader is a non-cygwin app first, and cygwin process reads
> the same pipe after that, the pipe has been set to bclocking mode
> for the cygwin app. However, the commit 9e4d308cd592 assumes the
> pipe for cygwin process always is non-blocking mode. With this patch,
> the pipe mode is reset to non-blocking when cygwin app is started.
> 
> Addresses: https://cygwin.com/pipermail/cygwin/2024-March/255644.html
> Fixes: 9e4d308cd592 ("Cygwin: pipe: Adopt FILE_SYNCHRONOUS_IO_NONALERT flag 
> for read pipe.")
> Reported-by: wh 
> Reviewed-by: Corinna Vinschen 
> Signed-off-by: Takashi Yano 
> ---
>  winsup/cygwin/fhandler/pipe.cc  | 63 +
>  winsup/cygwin/local_includes/fhandler.h |  3 ++
>  winsup/cygwin/spawn.cc  | 34 +
>  3 files changed, 68 insertions(+), 32 deletions(-)

Looks good, please push.


Thanks,
Corinna

Re: [PATCH] Cygwin: pipe: Restore non-blocking mode which was reset for non-cygwin app.

[Corinna Vinschen]

Hi Takashi,


this looks much better.  Just one question and a few comment
changes...

On Mar 11 22:18, Takashi Yano wrote:
> Subject: [PATCH v2] Cygwin: pipe: Make sure to set read pipe non-blocking for
>  cygwin apps.
> 
> If pipe reader is a non-cygwin app first, and cygwin process reads
> the same pipe after that, the pipe has been set to bclocking mode
> for the cygwin app. However, the commit 9e4d308cd592 assumes the
> pipe for cygwin process always is non-blocking mode. With this patch,
> the pipe mode is reset to non-blocking when cygwin app is started.
> 
> Addresses: https://cygwin.com/pipermail/cygwin/2024-March/255644.html
> Fixes: 9e4d308cd592 ("Cygwin: pipe: Adopt FILE_SYNCHRONOUS_IO_NONALERT flag 
> for read pipe.")
> Reported-by: wh 
> Reviewed-by: Corinna Vinschen 
> Signed-off-by: Takashi Yano 
> ---
>  winsup/cygwin/fhandler/pipe.cc  | 54 +
>  winsup/cygwin/local_includes/fhandler.h |  2 +
>  winsup/cygwin/spawn.cc  | 35 +---
>  3 files changed, 58 insertions(+), 33 deletions(-)
> 
> diff --git a/winsup/cygwin/fhandler/pipe.cc b/winsup/cygwin/fhandler/pipe.cc
> index 29d3b41d9..b29726dcb 100644
> --- a/winsup/cygwin/fhandler/pipe.cc
> +++ b/winsup/cygwin/fhandler/pipe.cc
> @@ -18,6 +18,7 @@ details. */
>  #include "pinfo.h"
>  #include "shared_info.h"
>  #include "tls_pbuf.h"
> +#include "sigproc.h"
>  #include 
>  
>  /* This is only to be used for writing.  When reading,
> @@ -1288,3 +1289,56 @@ close_proc:
>  }
>return NULL;
>  }
> +
> +void
> +fhandler_pipe::spawn_worker (bool iscygwin, int fileno_stdin,
> +  int fileno_stdout, int fileno_stderr)
> +{
> +  bool need_send_noncygchld_sig = false;
> +
> +  /* Set read pipe itself always non-blocking for cygwin process. blocking/
> + non-blocking is simulated in the raw_read(). As for write pipe, follow
> + the is_nonblocking(). */

You can drop the articles here, i.e.

...non-blocking is simulated in raw_read().  For write pipe follow
is_nonblocking().

> +  int fd;
> +  cygheap_fdenum cfd (false);
> +  while ((fd = cfd.next ()) >= 0)
> +if (cfd->get_dev () == FH_PIPEW
> + && (fd == fileno_stdout || fd == fileno_stderr))
> +  {
> + fhandler_pipe *pipe = (fhandler_pipe *)(fhandler_base *) cfd;
> + bool mode = iscygwin ? pipe->is_nonblocking () : false;
> + pipe->set_pipe_non_blocking (mode);

What bugs me here is that we now run the loop for cygwin children
as well.  The old code only did that for non-cygwin children.
This looks like quite a performance hit, potentially. Especially
if the process has many open descriptors.  Let's say, a recursive
make or so.  Did you find this is necessary?  No way to avoid that?

> +
> + /* Setup for query_ndl stuff. Read the comment below. */
> + if (!iscygwin && pipe->request_close_query_hdl ())
> +   need_send_noncygchld_sig = true;
> +  }
> +else if (cfd->get_dev () == FH_PIPER && fd == fileno_stdin)
> +  {
> + fhandler_pipe *pipe = (fhandler_pipe *)(fhandler_base *) cfd;
> + pipe->set_pipe_non_blocking (iscygwin);
> +  }
> +
> +  /* If multiple writers including non-cygwin app exist, the non-cygwin
> + app cannot detect pipe closure on the read side when the pipe is
> + created by system account or the the pipe creator is running as
 ^^^


Thanks,
Corinna

openssh 9.7p1-1

[Corinna Vinschen via Cygwin-announce]

The following packages have been uploaded to the Cygwin distribution:

* openssh-9.7p1-1

OpenSSH is a program for logging into a remote machine and for
executing commands on a remote machine.  It can replace rlogin
and rsh, providing encrypted communication between two machines.



PLEASE READ THE BELOW FUTURE DEPRECATION NOTICE!!!



OpenSSH 9.7 has just been released. It will be available from the
mirrors listed at https://www.openssh.com/ shortly.

OpenSSH is a 100% complete SSH protocol 2.0 implementation and
includes sftp client and server support.

Once again, we would like to thank the OpenSSH community for their
continued support of the project, especially those who contributed
code or patches, reported bugs, tested snapshots or donated to the
project. More information on donations may be found at:
https://www.openssh.com/donations.html

Future deprecation notice
=

OpenSSH plans to remove support for the DSA signature algorithm in
early 2025 and compile-time disable it later this year.

DSA, as specified in the SSHv2 protocol, is inherently weak - being
limited to a 160 bit private key and use of the SHA1 digest. Its
estimated security level is only 80 bits symmetric equivalent.

OpenSSH has disabled DSA keys by default since 2015 but has retained
run-time optional support for them. DSA was the only mandatory-to-
implement algorithm in the SSHv2 RFCs[3], mostly because alternative
algorithms were encumbered by patents when the SSHv2 protocol was
specified.

This has not been the case for decades at this point and better
algorithms are well supported by all actively-maintained SSH
implementations. We do not consider the costs of maintaining DSA in
OpenSSH to be justified and hope that removing it from OpenSSH can
accelerate its wider deprecation in supporting cryptography
libraries.

This release makes DSA support in OpenSSH compile-time optional,
defaulting to on. We intend the next release to change the default
to disable DSA at compile time. The first OpenSSH release of 2025
will remove DSA support entirely.

Changes since OpenSSH 9.6
=

This release contains mostly bugfixes.

New features


 * ssh(1), sshd(8): add a "global" ChannelTimeout type that watches
   all open channels and will close all open channels if there is no
   traffic on any of them for the specified interval. This is in
   addition to the existing per-channel timeouts added recently.

   This supports situations like having both session and x11
   forwarding channels open where one may be idle for an extended
   period but the other is actively used. The global timeout could
   close both channels when both have been idle for too long.

 * All: make DSA key support compile-time optional, defaulting to on.

Bugfixes


 * sshd(8): don't append an unnecessary space to the end of subsystem
   arguments (bz3667)

 * ssh(1): fix the multiplexing "channel proxy" mode, broken when
   keystroke timing obfuscation was added. (GHPR#463)

 * ssh(1), sshd(8): fix spurious configuration parsing errors when
   options that accept array arguments are overridden (bz3657).

 * ssh-agent(1): fix potential spin in signal handler (bz3670)

 * Many fixes to manual pages and other documentation, including
   GHPR#462, GHPR#454, GHPR#442 and GHPR#441.

 * Greatly improve interop testing against PuTTY.

Portability
---

 * Improve the error message when the autoconf OpenSSL header check
   fails (bz#3668)

 * Improve detection of broken toolchain -fzero-call-used-regs support
   (bz3645).

 * Fix regress/misc/fuzz-harness fuzzers and make them compile without
   warnings when using clang16

Checksums:
==

 - SHA1 (openssh-9.7.tar.gz) = 163272058edc20a8fde81661734a6684c9b4db11
 - SHA256 (openssh-9.7.tar.gz) = gXDWrF4wN2UWyPjyjvVhpjjKd7D2qI6LyZiIYhbJQVg=

 - SHA1 (openssh-9.7p1.tar.gz) = ce8985ea0ea2f16a5917fd982ade0972848373cc
 - SHA256 (openssh-9.7p1.tar.gz) = SQQm92bYKidj/KzY2D6j1weYdQx70q/y5X3FZg93P/0=

Please note that the SHA256 signatures are base64 encoded and not
hexadecimal (which is the default for most checksum tools). The PGP
key used to sign the releases is available from the mirror sites:
https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/RELEASE_KEY.asc

Reporting Bugs:
===

- Please read https://www.openssh.com/report.html
  Security bugs should be reported directly to open...@openssh.com

-- 
  *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
.

Re: Request for a rough release timeline for Cygwin 3.5.2

[Corinna Vinschen via Cygwin]

On Mar 11 11:48, Kate Deplaix via Cygwin wrote:
> Hi,
> 
> Sorry to bother you all,
> 
> Following the fixes (thank you so much again) of the two critical
> issues in cygwin 3.5.0/3.5.1
> https://cygwin.com/pipermail/cygwin/2024-February/255561.html , we
> were wondering if cygwin 3.5.2 is likely to be released soon. We would
> be extremely grateful and happy for that.

Probably still this month.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  9 15:29, Marcin Wisnicki via Cygwin wrote:
> I did more testing and found out that the problem does not happen in
> cygwin by default because cygwin mounts with acl which doesn't do
> header sniffing while msys uses noacl.
> 
> Testing on an mp4 file in OneDrive, when I use noacl in cygwin it
> triggers the read as well.
> After upgrading to the test version the read is gone and an mp4 file
> is not executable.
> 
> Thank you!

Thanks a lot for testing.  I backported the changes (minus the lsattr(1)/
chattr(1) changes) to the 3.5 branch so it will be in released with
3.5.2 in the next few weeks.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Mar 11 02:28, Dan Shelton via Cygwin wrote:
> On Wed, 6 Mar 2024 at 14:01, Corinna Vinschen via Cygwin
>  wrote:
> > On Mar  5 23:38, Dan Shelton via Cygwin wrote:
> > > Always editing /etc/nsswitch.conf
> > > forth and back is not a elegant solution, aside from race conditions
> > > with other users on a system
> >
> > So, here we go again.
> >
> > - What exactly are you trying to accomplish by enumerating the accounts?
> >   Maybe you won't actually need it for your task at hand.
> 
> We're trying to do several things, including but not limited to:
> - Finding which local groups exist. Part of our customer software
> expects that certain groups exist. Unfortunately the group names vary
> between installations, and sometimes names are prefixed with site
> names. Trying to do all permutations with just getent passwd
> $iteration means too many combinations (>= 400). So just
> enumerating all local groups with getent group would be the way to go.

Then just change /etc/nsswitch.conf to enumerate "local" as well
and be done with it.

You can even go so far as to use the Windows enumerator, i.e.,

  $ net localgroup

and than script it to use its output as input to getent group
for only the groups you really need info for.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Mar  9 16:26, Glenn Strauss via Cygwin wrote:
> On Wed, Mar 06, 2024 at 02:01:06PM +0100, Corinna Vinschen via Cygwin wrote:
> > On Mar  5 23:38, Dan Shelton via Cygwin wrote:
> > > On Sat, 24 Feb 2024 at 14:11, Corinna Vinschen via Cygwin
> > >  wrote:
> > > >
> > > > On Feb 23 22:15, Dan Shelton via Cygwin wrote:
> > > > > HOWEVER, there is another Cygwin bug:
> > > > > "getent group mywingrp1" does not list any group members, even after
> > > > > "net localgroup mywingrp1 mywinuser44 /add", which is a POSIX
> > > > > violation.
> > > >
> > > > Not a bug.  Two problems:
> > > >
> > > > - Getting members of a group can be an extremly costly operation
> > > >   in a domain or, worse, a domain forest, or even worse, if the
> > > >   domain or domain forest is remote.
> > > >
> > > > - Alonmg the same lines, getting members of a group can be extremly
> > > >   costly in big orgs with thousands of users.  Nobody want's to clutter
> > > >   up space with the list of members in the "Domain Users" group.
> > > >
> > > > - Permissions to enumerate members of a group are restricted.
> > > >   By default only admins and group members are allow to enumerate
> > > >   members and this can be restricted further by domain admins.
> > > >
> > > > Therefore we dropped even trying to populate gr_mem, considering
> > > > that even in its original form on Unix systems, it's used only
> > > > to add supplementary groups.  To do this right on Windows is even
> > > > more costly than blindly enumerating.
> > > >
> > > > It's not a bug, it's a feature :)
> > > 
> > > Could you add an option to getent so that the full lookup can be
> > > requested via command line, pls?
> > 
> > That's not possible.  getent just calls getpwent/getgrent.
> > 
> > > Always editing /etc/nsswitch.conf
> > > forth and back is not a elegant solution, aside from race conditions
> > > with other users on a system
> > 
> > So, here we go again.
> > 
> > - What exactly are you trying to accomplish by enumerating the accounts?
> >   Maybe you won't actually need it for your task at hand.
> > 
> > - Why do you have to change nsswitch.conf "back and forth"?
> >   Just change it once and you're done.
> > 
> > 
> > Corinna
> 
> Hello
> > > Dan Shelton - Cluster Specialist Win/Lin/Bsd
> 
> > > Always editing /etc/nsswitch.conf
> > > forth and back is not a elegant solution, aside from race conditions
> > > with other users on a system
> 
> Please check the man page for getent.
> 
> man getent
> getent --help
> 
> You can use -s or --service to override the service used without
> editing nsswitch.conf.  The man page on Linux provides an example
> with a bit more details than the man page for getent under cygwin.
> https://www.man7.org/linux/man-pages/man1/getent.1.html

The -s option is just available for compatibility, but otherwise a no-op
on Cygwin.  The -i option works as upstream, -w is a Cygwin-only option.

The Cygwin getent.1 man page is a bit old, given it's taken from
upstream and just slightly adapted from glibc 2.18.90, a good 10 years
ago.  The upstream man page at the time did not document the options for
some reason.  These days it documents the options, so we could fetch
this and create a new manpage from there.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Cygwin: pipe: Restore non-blocking mode which was reset for non-cygwin app.

[Corinna Vinschen]

Hi Takashi,

On Mar 10 19:31, Takashi Yano wrote:
> @@ -590,6 +591,10 @@ child_info_spawn::worker (const char *prog_arg, const 
> char *const *argv,
> {
>   fhandler_pipe *pipe = (fhandler_pipe *)(fhandler_base *) cfd;
>   pipe->set_pipe_non_blocking (false);
> + pipew_duped = (fhandler_pipe *)
> + ccalloc (HEAP_FHANDLER, 1, sizeof (fhandler_pipe));
> + pipew_duped = new (pipew_duped) fhandler_pipe;
> + pipe->dup (pipew_duped, 0);
>   if (pipe->request_close_query_hdl ())
> need_send_sig = true;
> }

The code setting up pipes and the dummy_tty is sufficiently complex,
so that I wonder if it shouldn't have

- its own methods and
- comments to describe why this stuff is necessary.

What about adding two methods, kind of like (the names are only
suggestion, albeit bad ones):

  child_info_spawn::noncygwin_child_pre_fork()

to keep the above stuff together (plus comments) and

  child_info_spawn::noncygwin_child_post_fork()

for the below code?

> @@ -597,6 +602,10 @@ child_info_spawn::worker (const char *prog_arg, const 
> char *const *argv,
> {
>   fhandler_pipe *pipe = (fhandler_pipe *)(fhandler_base *) cfd;
>   pipe->set_pipe_non_blocking (false);
> + piper_duped = (fhandler_pipe *)
> + ccalloc (HEAP_FHANDLER, 1, sizeof (fhandler_pipe));
> + piper_duped = new (piper_duped) fhandler_pipe;
> + pipe->dup (piper_duped, 0);
> }
>  
> if (need_send_sig)
> @@ -905,6 +914,19 @@ child_info_spawn::worker (const char *prog_arg, const 
> char *const *argv,
> term_spawn_worker.cleanup ();
> term_spawn_worker.close_handle_set ();
>   }
> +   if (pipew_duped)
> + {
> +   bool is_nonblocking = pipew_duped->is_nonblocking ();
> +   pipew_duped->set_pipe_non_blocking (is_nonblocking);

Is that really right?  You're asking pipew_duped for its
nonblocking flag and then set pipew_duped to the same value...?

> +   pipew_duped->close ();
> +   cfree (pipew_duped);
> + }


Thanks,
Corinna

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  8 14:56, Corinna Vinschen via Cygwin wrote:
> On Mar  8 08:15, Jeffrey Altman via Cygwin wrote:
> > On 3/8/2024 7:52 AM, Thomas Wolff via Cygwin wrote:
> > > Am 08.03.2024 um 11:37 schrieb Corinna Vinschen via Cygwin:
> > > >    FILE_OPEN_NO_RECALL (0x0040)
> > > > [...]
> > > > This sounds like we could simply add this flag to all NtOpenFile
> > > > used for path conversion or stat-like calls, without having to care
> > > > for any file attributes specificially.
> > > > 
> > > > Does that make sense?
> > > Sounds good, without even studying the other details...
> > > I speculate some more handling would still be needed to avoid executable
> > > detection via magic tags.
> > > 
> > Agreed.   FILE_OPEN_NO_RECALL has been defined for at least a decade but was
> > not documented by Microsoft relatively recently.
> 
> Thanks for the feedback, guys.
> 
> > Another suggestion would be to try opening the file with
> > FILE_READ_ATTRIBUTES instead of GENERIC_READ if the file data is not
> > required.  See
> > 
> > https://github.com/microsoft/BuildXL/commit/4fb8e7ce07d243ccd95de0d66da551538a794493
> 
> Cygwin uses the minimum of required permissions in NtCreateFile/
> NtOpenFile calls anyway.
> 
> I'm just running a test cygwin DLL locally with a lot of added
> FILE_OPEN_NO_RECALL bits and a couple of added attribute checks for
> being offline to allow skipping some code.
> 
> I think I'll push this change in a bit so we get a test release out
> so people using OneDrive can test.

I pushed this change as well as a followup change to make sure we don't
inadvertently recall an offline file.  I also added handling for the
Pinned and Unpinned attributes to chattr(1) and lsattr(1).

The full set of changes can be tested by installing the Cygwin test
release 3.6.0-0.77.g06aa5a751682.

Please give it a try.  If you encounter a situation which still results
in recalling an offline file in a situation which doesn't qualify for
it, please report.  We will have to analyze that situation further
then.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  8 08:15, Jeffrey Altman via Cygwin wrote:
> On 3/8/2024 7:52 AM, Thomas Wolff via Cygwin wrote:
> > Am 08.03.2024 um 11:37 schrieb Corinna Vinschen via Cygwin:
> > > Yesterday I stumbled over a certain NtCreateFile flag:
> > > 
> > >    FILE_OPEN_NO_RECALL (0x0040)
> > > 
> > >  Instructs any filters that perform offline storage or
> > > virtualization
> > >  to not recall the contents of the file as a result of this open.
> > > 
> > > MS-CIFS described it like this:
> > > 
> > >    FILE_OPEN_NO_RECALL
> > >    0x0040
> > > 
> > >  In a hierarchical storage management environment, this option
> > >  requests that the file SHOULD NOT be recalled from tertiary storage
> > >  such as tape. A file recall can take up to several minutes in a
> > >  hierarchical storage management environment. The clients can
> > > specify
> > >  this option to avoid such delays.
> > > 
> > > This sounds like we could simply add this flag to all NtOpenFile
> > > used for path conversion or stat-like calls, without having to care
> > > for any file attributes specificially.
> > > 
> > > Does that make sense?
> > Sounds good, without even studying the other details...
> > I speculate some more handling would still be needed to avoid executable
> > detection via magic tags.
> > 
> Agreed.   FILE_OPEN_NO_RECALL has been defined for at least a decade but was
> not documented by Microsoft relatively recently.

Thanks for the feedback, guys.

> Another suggestion would be to try opening the file with
> FILE_READ_ATTRIBUTES instead of GENERIC_READ if the file data is not
> required.  See
> 
> https://github.com/microsoft/BuildXL/commit/4fb8e7ce07d243ccd95de0d66da551538a794493

Cygwin uses the minimum of required permissions in NtCreateFile/
NtOpenFile calls anyway.

I'm just running a test cygwin DLL locally with a lot of added
FILE_OPEN_NO_RECALL bits and a couple of added attribute checks for
being offline to allow skipping some code.

I think I'll push this change in a bit so we get a test release out
so people using OneDrive can test.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

Hi Jeffrey,

On Mar  6 13:55, Jeffrey Altman via Cygwin wrote:
> On 3/6/2024 12:19 PM, Corinna Vinschen via Cygwin wrote:
> > We can add an explicit call to
> > 
> >RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);
> > [...]
> Files and directories that are placeholders should have either the
> FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS or FILE_ATTRIBUTE_RECALL_ON_OPEN file
> attributes set. When these attributes are set, applications and mini filters
> are advised not to "read" or "open" the files or directories unless they
> absolutely need to because doing so will cause the placeholder to be
> replaced by an object containing the actual data which might take a long
> time to fetch,

Yesterday I stumbled over a certain NtCreateFile flag:

  FILE_OPEN_NO_RECALL (0x0040)

Instructs any filters that perform offline storage or virtualization
to not recall the contents of the file as a result of this open.

MS-CIFS described it like this:

  FILE_OPEN_NO_RECALL
  0x0040

In a hierarchical storage management environment, this option
requests that the file SHOULD NOT be recalled from tertiary storage
such as tape. A file recall can take up to several minutes in a
hierarchical storage management environment. The clients can specify
this option to avoid such delays.

This sounds like we could simply add this flag to all NtOpenFile
used for path conversion or stat-like calls, without having to care
for any file attributes specificially.

Does that make sense?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

Hi Jeffrey,


apart from the attribute stuff...


On Mar  6 13:55, Jeffrey Altman via Cygwin wrote:
> The default ProcessPlaceholderCompaibilityMode is PHCM_EXPOSE_PLACEHOLDERS
> which makes the FILE_ATTRIBUTE flags and reparse tags visible. Microsoft
> maintains a database of processes for which PHCM_DISGUISE_PLACEHOLDER is set
> which hides that information. Its unclear to me that explicitly setting the
> placeholder compatibility mode is useful.

What I see as a problem here is this:

https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlsetprocessplaceholdercompatibilitymode

Quote:

  "Most Windows applications see exposed placeholders by default. For
   
   compatibility reasons, Windows may decide that certain applications
  ^^
   see disguised placeholders by default."

But then again, in other news from Microsoft:

https://learn.microsoft.com/en-us/windows/win32/cfapi/build-a-cloud-file-sync-engine#compatibility-with-applications-that-use-reparse-points

Quote:

  "[...] the cloud files API always hides its reparse points from all
 
   applications except for sync engines and processes whose main image
   ^^^
   resides under %systemroot%. Applications that understand reparse
   ^^^
   points correctly can force the platform to expose cloud files API
   reparse points using RtlSetProcessPlaceholderCompatibilityMode or
   RtlSetThreadProcessPlaceholderCompatibilityMode.

Considering these two statements, it's totally unclear to a process, if
it just defaults to "exposed" or "disguised".

Fortunately we can ask Windows by calling the
RtlQueryProcessPlaceholderCompatibilityMode() function, right?

Lets have a look into the documentation at
https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/ntifs/nf-ntifs-rtlqueryprocessplaceholdercompatibilitymode

Quote:

  "Return value

  This function returns the process's placeholder compatibily mode
  (PHCM_xxx), or a negative value on error (PCHM_ERROR_xxx). Contains
  one of the following values:

  Compatibility Mode   Value
  PHCM_APPLICATION_DEFAULT   0
  PHCM_DISGUISE_PLACEHOLDER  1
  PHCM_EXPOSE_PLACEHOLDERS   2
  PHCM_MAX   2
  PHCM_ERROR_INVALID_PARAMETER  -1
  PHCM_ERROR_NO_TEB -2"

So I called the function right at the start of the Cygwin DLL, and it
returns the value 0, i. e., PHCM_APPLICATION_DEFAULT.

At this point the process *still* has no idea if placeholders are
exposed or disguised.  What a great API! \o/

So, from the above, and if we really want to be sure that placeholders
will be exposed, I don't see any way around calling
RtlSetProcessPlaceholderCompatibilityMode(PHCM_EXPOSE_PLACEHOLDERS)
explicitely at DLL startup.

What do you think?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

Hi Jeffrey,


looks like writing our mails overlapped:
https://cygwin.com/pipermail/cygwin/2024-March/255622.html

On Mar  6 13:55, Jeffrey Altman via Cygwin wrote:
> On 3/6/2024 12:19 PM, Corinna Vinschen via Cygwin wrote:
> > We can add an explicit call to
> > 
> >RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);
> > 
> > and we can recognize the IO_REPARSE_TAG_FILE_PLACEHOLDER and
> > IO_REPARSE_TAG_CLOUD_* tags during symlink evaluation, but even then
> > we still have to know what the reparse point buffer actually contains.
> > 
> > Given that the content of reparse points with these reparse tags are
> > undocumented, some people using cloud services should examine these
> > reparse points so we can add some suitable code to Cygwin.
> > 
> > 
> > Corinna
> I'm not an expert in this area by any means but here are my recollections
> from when Microsoft presented in-person on cloud placeholders to filter and
> filesystem developers many years ago.
> 
> Files and directories that are placeholders should have either the
> FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS or FILE_ATTRIBUTE_RECALL_ON_OPEN file
> attributes set. When these attributes are set, applications and mini filters
> are advised not to "read" or "open" the files or directories unless they
> absolutely need to

Per 
https://learn.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants
FILE_ATTRIBUTE_RECALL_ON_OPEN only appears in directory listing
classes, but not in standard FILE_BASIC_INFORMATION and alike.
That's a bit of a problem considering how we check files during
path conversion.

The MSDN article doesn't state the same for
FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS, which is good, I think.

But I'm a bit puzzled then in terms of FILE_ATTRIBUTE_OFFLINE.  Is it
not used for OneDrive files?

> [...]
> I'm not sure that exposing the object as a symlink is a good idea.

Yeah, that's what I realized as well, see my aforementioned mail.

> Perhaps the question that needs to be asked is whether there are opens that
> can be skipped if an object is known to not be locally present (either of
> the FILE_ATTRIBUTE flags are set)?

This may be the way to go, see my mail.  It wouldn't be much of
a problem to check all attribute bits, i.e. FILE_ATTRIBUTE_OFFLINE,
FILE_ATTRIBUTE_RECALL_ON_DATA_ACCESS and FILE_ATTRIBUTE_RECALL_ON_OPEN.
Maybe that's what we should do.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  6 18:19, Corinna Vinschen via Cygwin wrote:
> On Mar  6 06:54, Brian Inglis via Cygwin wrote:
> > On 2024-03-06 06:28, Corinna Vinschen via Cygwin wrote:
> > > On Mar  6 14:22, Corinna Vinschen via Cygwin wrote:
> > > > Given these placeholder files are actually reparse points of type
> > > > IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.
> > > > 
> > > > However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
> > > > data buffer is undocumented.  It would be helpful if somebody using
> > > > OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.
> > > > 
> > > > > [2] 
> > > > > https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548
> > > > 
> > > > The NtReadFile call at this point is not the problem.  It would be
> > > > helpful to point to Cygwin's source instead of MSYS2, btw.
> > > 
> > > Oh, btw., this is from
> > > https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4:
> > > 
> > >IO_REPARSE_TAG_FILE_PLACEHOLDER
> > >0x8015
> > > 
> > >  Obsolete.
> > >  -
> > >  Used by Windows Shell for legacy placeholder files in Windows 8.1.
> > >  Server-side interpretation only, not meaningful over the wire.
> > > 
> > > So even if we support them, what is their replacement in W10 and later?
> > 
> > May or not help:
> > 
> > https://stackoverflow.com/questions/59152220/cant-get-reparse-point-information-for-the-onedrive-folder
> 
> We can add an explicit call to
> 
>   RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);
> 
> and we can recognize the IO_REPARSE_TAG_FILE_PLACEHOLDER and
> IO_REPARSE_TAG_CLOUD_* tags during symlink evaluation, but even then
> we still have to know what the reparse point buffer actually contains.
> 
> Given that the content of reparse points with these reparse tags are
> undocumented, some people using cloud services should examine these
> reparse points so we can add some suitable code to Cygwin.

Reading further on this it seems that one cannot easily compare these
reparse points with symlinks.

The tag values are 0x8015 for IO_REPARSE_TAG_FILE_PLACEHOLDER and
0x901AL up to 0x9000F01AL for the IO_REPARSE_TAG_CLOUD_* tags.  None
of them have the name surrogate bit set, so they don't "represent
another named entity in the system".  In other words, these reparse
points represent themselves rather than pointing to some other file, as
symlinks do.

Additionally the IO_REPARSE_TAG_CLOUD_* tags all have the directory bit
set so it seems they are used on the parent(?) directory of the local
OneDrive copy only, but not on the files inside it.

Bottom line:

I wonder if the real deal is not the reparse tag and the reparse
content, but whether or not the file has the FILE_ATTRIBUTE_OFFLINE flag
set.

If so, we can try to disable any action within path conversion, as 
well as in our stat(2) and readdir(3) implementation which would
trigger onlining an offline file.

Can anybody confirm that the idea is right, or if I'm something missing?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  6 06:54, Brian Inglis via Cygwin wrote:
> On 2024-03-06 06:28, Corinna Vinschen via Cygwin wrote:
> > On Mar  6 14:22, Corinna Vinschen via Cygwin wrote:
> > > Given these placeholder files are actually reparse points of type
> > > IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.
> > > 
> > > However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
> > > data buffer is undocumented.  It would be helpful if somebody using
> > > OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.
> > > 
> > > > [2] 
> > > > https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548
> > > 
> > > The NtReadFile call at this point is not the problem.  It would be
> > > helpful to point to Cygwin's source instead of MSYS2, btw.
> > 
> > Oh, btw., this is from
> > https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4:
> > 
> >IO_REPARSE_TAG_FILE_PLACEHOLDER
> >0x8015
> > 
> >  Obsolete.
> >  -
> >  Used by Windows Shell for legacy placeholder files in Windows 8.1.
> >  Server-side interpretation only, not meaningful over the wire.
> > 
> > So even if we support them, what is their replacement in W10 and later?
> 
> May or not help:
> 
> https://stackoverflow.com/questions/59152220/cant-get-reparse-point-information-for-the-onedrive-folder

We can add an explicit call to

  RtlSetProcessPlaceholderCompatibilityMode (PHCM_EXPOSE_PLACEHOLDERS);

and we can recognize the IO_REPARSE_TAG_FILE_PLACEHOLDER and
IO_REPARSE_TAG_CLOUD_* tags during symlink evaluation, but even then
we still have to know what the reparse point buffer actually contains.

Given that the content of reparse points with these reparse tags are
undocumented, some people using cloud services should examine these
reparse points so we can add some suitable code to Cygwin.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  6 14:22, Corinna Vinschen via Cygwin wrote:
> On Mar  5 19:54, Marcin Wisnicki via Cygwin wrote:
> > If I invoke ls or anything else that does stat inside OneDrive folder
> > it will trigger download of all files.
> > 
> > OneDrive uses placeholder files[1] to represent remote files.
> > 
> > I'm guessing reading file content in stat is to support detection of
> > actually executable files as in here[2]?
> > 
> > I think this should be disabled on non-hydrated placeholder files.
> > Running `find` or 'ls -R` and having your entire OneDrive downloaded
> > is extremely problematic.
> > 
> > I could live without executable scripts in the OneDrive folder and
> > it's easy to mark files as always offline to solve it.
> > 
> > Another idea is to skip checking files with extensions known to be
> > non-executable such as jpg (or just any extensions that is not known
> > to be executable).
> 
> Nothing of this makes sense from a POSIX library POV.  The library can
> either not handle placeholder files specially, as today, or it can
> handle them all the same way.
> 
> Given these placeholder files are actually reparse points of type
> IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.
> 
> However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
> data buffer is undocumented.  It would be helpful if somebody using
> OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.
> 
> > [2] 
> > https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548
> 
> The NtReadFile call at this point is not the problem.  It would be
> helpful to point to Cygwin's source instead of MSYS2, btw.

Oh, btw., this is from
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/c8e77b37-3909-4fe6-a4ea-2b9d423b1ee4:

  IO_REPARSE_TAG_FILE_PLACEHOLDER
  0x8015

Obsolete.
-
Used by Windows Shell for legacy placeholder files in Windows 8.1.
Server-side interpretation only, not meaningful over the wire.

So even if we support them, what is their replacement in W10 and later?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: ls/stat on OneDrive causes download of files

[Corinna Vinschen via Cygwin]

On Mar  5 19:54, Marcin Wisnicki via Cygwin wrote:
> If I invoke ls or anything else that does stat inside OneDrive folder
> it will trigger download of all files.
> 
> OneDrive uses placeholder files[1] to represent remote files.
> 
> I'm guessing reading file content in stat is to support detection of
> actually executable files as in here[2]?
> 
> I think this should be disabled on non-hydrated placeholder files.
> Running `find` or 'ls -R` and having your entire OneDrive downloaded
> is extremely problematic.
> 
> I could live without executable scripts in the OneDrive folder and
> it's easy to mark files as always offline to solve it.
> 
> Another idea is to skip checking files with extensions known to be
> non-executable such as jpg (or just any extensions that is not known
> to be executable).

Nothing of this makes sense from a POSIX library POV.  The library can
either not handle placeholder files specially, as today, or it can
handle them all the same way.

Given these placeholder files are actually reparse points of type
IO_REPARSE_TAG_FILE_PLACEHOLDER, we can handle them as symbolic links.

However, the structure of the IO_REPARSE_TAG_FILE_PLACEHOLDER reparse
data buffer is undocumented.  It would be helpful if somebody using
OneDrive would examine the content of the attached REPARSE_DATA_BUFFER.

> [2] 
> https://github.com/msys2/msys2-runtime/blob/msys2-3.4.10/winsup/cygwin/fhandler/disk_file.cc#L548

The NtReadFile call at this point is not the problem.  It would be
helpful to point to Cygwin's source instead of MSYS2, btw.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Mar  5 23:38, Dan Shelton via Cygwin wrote:
> On Sat, 24 Feb 2024 at 14:11, Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Feb 23 22:15, Dan Shelton via Cygwin wrote:
> > > HOWEVER, there is another Cygwin bug:
> > > "getent group mywingrp1" does not list any group members, even after
> > > "net localgroup mywingrp1 mywinuser44 /add", which is a POSIX
> > > violation.
> >
> > Not a bug.  Two problems:
> >
> > - Getting members of a group can be an extremly costly operation
> >   in a domain or, worse, a domain forest, or even worse, if the
> >   domain or domain forest is remote.
> >
> > - Alonmg the same lines, getting members of a group can be extremly
> >   costly in big orgs with thousands of users.  Nobody want's to clutter
> >   up space with the list of members in the "Domain Users" group.
> >
> > - Permissions to enumerate members of a group are restricted.
> >   By default only admins and group members are allow to enumerate
> >   members and this can be restricted further by domain admins.
> >
> > Therefore we dropped even trying to populate gr_mem, considering
> > that even in its original form on Unix systems, it's used only
> > to add supplementary groups.  To do this right on Windows is even
> > more costly than blindly enumerating.
> >
> > It's not a bug, it's a feature :)
> 
> Could you add an option to getent so that the full lookup can be
> requested via command line, pls?

That's not possible.  getent just calls getpwent/getgrent.

> Always editing /etc/nsswitch.conf
> forth and back is not a elegant solution, aside from race conditions
> with other users on a system

So, here we go again.

- What exactly are you trying to accomplish by enumerating the accounts?
  Maybe you won't actually need it for your task at hand.

- Why do you have to change nsswitch.conf "back and forth"?
  Just change it once and you're done.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Cygwin: pipe: Give up to use query_hdl for non-cygwin apps.

[Corinna Vinschen]

On Mar  6 03:42, Takashi Yano wrote:
> On Tue, 5 Mar 2024 17:54:19 +0100
> Corinna Vinschen wrote:
> > On Mar  5 23:47, Takashi Yano wrote:
> > > On Tue, 5 Mar 2024 11:14:46 +0100
> > > Corinna Vinschen wrote:
> > > > This doesn't affect your patch, but while looking into this, what
> > > > strikes me as weird is that fhandler_pipe::temporary_query_hdl() calls
> > > > NtQueryObject() and assembles the pipe name via swscanf() every time it
> > > > is called.
> > > > 
> > > > Wouldn't it make sense to store the name in the fhandler's
> > > > path_conv::wide_path/uni_path at creation time instead?
> > > > The wide_path member is not used at all in pipes, ostensibly.
> > > 
> > > Is the patch attached as you intended?
> > 
> > Yes, but it looks like it misses a few potential simplifications:
> > [...]
> Thanks for advice. I have revised the patch.

Looks good, thanks!


Corinna

Re: [PATCH] Cygwin: pipe: Give up to use query_hdl for non-cygwin apps.

[Corinna Vinschen]

On Mar  5 23:47, Takashi Yano wrote:
> On Tue, 5 Mar 2024 11:14:46 +0100
> Corinna Vinschen wrote:
> > This doesn't affect your patch, but while looking into this, what
> > strikes me as weird is that fhandler_pipe::temporary_query_hdl() calls
> > NtQueryObject() and assembles the pipe name via swscanf() every time it
> > is called.
> > 
> > Wouldn't it make sense to store the name in the fhandler's
> > path_conv::wide_path/uni_path at creation time instead?
> > The wide_path member is not used at all in pipes, ostensibly.
> 
> Is the patch attached as you intended?

Yes, but it looks like it misses a few potential simplifications:

> diff --git a/winsup/cygwin/fhandler/pipe.cc b/winsup/cygwin/fhandler/pipe.cc
> index c877d89d7..0611dd1c3 100644
> --- a/winsup/cygwin/fhandler/pipe.cc
> +++ b/winsup/cygwin/fhandler/pipe.cc
> @@ -93,6 +93,19 @@ fhandler_pipe::init (HANDLE f, DWORD a, mode_t mode, 
> int64_t uniq_id)
> even with FILE_SYNCHRONOUS_IO_NONALERT. */
>  set_pipe_non_blocking (get_device () == FH_PIPER ?
>  true : is_nonblocking ());
> +
> +  /* Store pipe name to path_conv pc for query_hdl check */
> +  if (get_dev () == FH_PIPEW)
> +{
> +  ULONG len;
> +  tmp_pathbuf tp;
> +  OBJECT_NAME_INFORMATION *ntfn = (OBJECT_NAME_INFORMATION *) tp.w_get 
> ();
> +  NTSTATUS status = NtQueryObject (f, ObjectNameInformation, ntfn,
> +65536, );
> +  if (NT_SUCCESS (status) && ntfn->Name.Buffer)
> + pc.set_nt_native_path (>Name);

We don't have to call NtQueryObject.  The name is created in nt_create()
and we know the unique id, so the name is

  "%S%S-%u-pipe-nt-%p", _u_ntfs, >installation_key,
GetCurrentProcessId (), unique_id);

Do you think it's cheaper to call NtQueryObject()?  If so, no worries,
but NtQueryObject() has to call into the kernel, while just creating
the name by ourselves doesn't.

> @@ -1149,6 +1162,9 @@ fhandler_pipe::temporary_query_hdl ()
>tmp_pathbuf tp;
>OBJECT_NAME_INFORMATION *ntfn = (OBJECT_NAME_INFORMATION *) tp.w_get ();
>  
> +  UNICODE_STRING *name = pc.get_nt_native_path (NULL);
> +  name->Buffer[name->Length / sizeof (WCHAR)] = L'\0';

The string returned by get_nt_native_path() is always NUL-terminated.

>/* Try process handle opened and pipe handle value cached first
>   in order to reduce overhead. */
>if (query_hdl_proc && query_hdl_value)
> @@ -1161,14 +1177,7 @@ fhandler_pipe::temporary_query_hdl ()
>status = NtQueryObject (h, ObjectNameInformation, ntfn, 65536, );
>if (!NT_SUCCESS (status) || !ntfn->Name.Buffer)
>   goto hdl_err;
> -  ntfn->Name.Buffer[ntfn->Name.Length / sizeof (WCHAR)] = L'\0';
> -  uint64_t key;
> -  DWORD pid;
> -  LONG id;
> -  if (swscanf (ntfn->Name.Buffer,
> -L"\\Device\\NamedPipe\\%llx-%u-pipe-nt-0x%x",
> -, , ) == 3 &&
> -   key == pipename_key && pid == pipename_pid && id == pipename_id)
> +  if (RtlEqualUnicodeString (name, >Name, FALSE))
>   return h;
>  hdl_err:
>CloseHandle (h);
> @@ -1178,19 +1187,9 @@ cache_err:
>query_hdl_value = NULL;
>  }
>  
> -  status = NtQueryObject (get_handle (), ObjectNameInformation, ntfn,
> -   65536, );
> -  if (!NT_SUCCESS (status) || !ntfn->Name.Buffer)
> +  if (name->Length == 0 || name->Buffer == NULL)
>  return NULL; /* Non cygwin pipe? */
> -  WCHAR name[MAX_PATH];
> -  int namelen = min (ntfn->Name.Length / sizeof (WCHAR), MAX_PATH-1);
> -  memcpy (name, ntfn->Name.Buffer, namelen * sizeof (WCHAR));
> -  name[namelen] = L'\0';
> -  if (swscanf (name, L"\\Device\\NamedPipe\\%llx-%u-pipe-nt-0x%x",
> -_key, _pid, _id) != 3)
> -return NULL; /* Non cygwin pipe? */
> -
> -  return get_query_hdl_per_process (name, ntfn); /* Since Win8 */
> +  return get_query_hdl_per_process (name->Buffer, ntfn); /* Since Win8 */

Given the name is stored in the fhandler, get_query_hdl_per_process()
doesn't need it as argument, and get_query_hdl_per_process() can just
call RtlCompareUnicodeString() instead of adding a \0 and calling
wcscmp().


Thanks,
Corinna

Re: [PATCH v3] Cygwin: pipe: Give up to use query_hdl for non-cygwin apps.

[Corinna Vinschen]

On Mar  5 23:48, Takashi Yano wrote:
> Non-cygwin app may call ReadFile() for empty pipe, which makes
> NtQueryObject() for ObjectNameInformation block in fhandler_pipe::
> get_query_hdl_per_process. Therefore, do not to try to get query_hdl
> for non-cygwin apps.
> 
> Addresses: https://github.com/msys2/msys2-runtime/issues/202
> 
> Fixes: b531d6b06eeb ("Cygwin: pipe: Introduce temporary query_hdl.")
> Reported-by: Alisa Sireneva, Johannes Schindelin 
> Reviewed-by: Corinna Vinschen 
> Signed-off-by: Takashi Yano 
> ---
>  winsup/cygwin/fhandler/pipe.cc | 57 --
>  winsup/cygwin/release/3.5.2|  4 +++
>  2 files changed, 17 insertions(+), 44 deletions(-)

Looks good, thanks!


Corinna

Re: [PATCH] Cygwin: pipe: Give up to use query_hdl for non-cygwin apps.

[Corinna Vinschen]

On Mar  5 09:06, Takashi Yano wrote:
> On Mon, 4 Mar 2024 18:38:07 +0100
> Corinna Vinschen wrote:
> > On Mar  4 16:45, ASSI wrote:
> > > Corinna Vinschen writes:
> > > > Right you are.  We always said that independent Cygwin installations
> > > > are supposed to *stay* independent.
> > > >
> > > > Keep in mind that they don't share the same shared objects in the native
> > > > NT namespace and they don't know of each other.  It's not only the
> > > > process table but also in-use FIFO stuff, pty info, etc.
> > > 
> > > What I was getting at is that a process not showing up in the process
> > > list in one Cygwin installation doesn't automatically mean it's a native
> > > Windows process, it could be a process started by an independent Cygwin
> > > installation.  So this way of checking for "native" Windows processes
> > > may or may not do what was originally intended.
> > 
> > But that was my point. A "foreign" Cygwin process from another
> > installation is not a Cygwin process.  Lots of interoperability
> > just won't work, so it's basically a native process.
> 
> Actually, I think query_hdl can be retrieved from the process
> from another installation of cygwin using NtQueryInformationProcess()
> with ProcessHandleInformation. However, I cannot imagne the case
> that the pipe is made by one cygwin installation but the reader
> process is from another installation of cygwin.
> 
> BTW, what about v2 patch itself?

It does the job with less code and less memory, which is good.
I would change the comment

  stop to try to get query_hdl for non-cygwin apps

to something like

  don't try to fetch query_hdl from non-cygwin apps

"stop trying" is a bit of a back-reference to the old code, which
is not necessary, I think.

This doesn't affect your patch, but while looking into this, what
strikes me as weird is that fhandler_pipe::temporary_query_hdl() calls
NtQueryObject() and assembles the pipe name via swscanf() every time it
is called.

Wouldn't it make sense to store the name in the fhandler's
path_conv::wide_path/uni_path at creation time instead?
The wide_path member is not used at all in pipes, ostensibly.


Thanks,
Corinna

Re: [PATCH] Cygwin: pipe: Give up to use query_hdl for non-cygwin apps.

[Corinna Vinschen]

On Mar  4 16:45, ASSI wrote:
> Corinna Vinschen writes:
> > Right you are.  We always said that independent Cygwin installations
> > are supposed to *stay* independent.
> >
> > Keep in mind that they don't share the same shared objects in the native
> > NT namespace and they don't know of each other.  It's not only the
> > process table but also in-use FIFO stuff, pty info, etc.
> 
> What I was getting at is that a process not showing up in the process
> list in one Cygwin installation doesn't automatically mean it's a native
> Windows process, it could be a process started by an independent Cygwin
> installation.  So this way of checking for "native" Windows processes
> may or may not do what was originally intended.

But that was my point. A "foreign" Cygwin process from another
installation is not a Cygwin process.  Lots of interoperability
just won't work, so it's basically a native process.


Corinna

Re: [PATCH] Cygwin: pipe: Give up to use query_hdl for non-cygwin apps.

[Corinna Vinschen]

On Mar  3 20:36, Takashi Yano wrote:
> On Sun, 03 Mar 2024 11:39:40 +0100
> ASSI wrote:
> > Takashi Yano writes:
> > >> After noticing that we enumerate all the processes (which is an expensive
> > >> operation) just to skip all of the non-Cygwin ones anyway, I wonder if it
> > >> wouldn't be smarter to go through the internal list of cygpids and take 
> > >> it
> > >> from there, skipping the `SystemProcessInformation` calls altogether.
> > >
> > > Yeah, that makes sens. I'll submit v2 patch.
> > 
> > Keep in mind that there may be different independent Cygwin
> > installations running on the same nachine.
> 
> That's possible. But how can we know that is a process in another
> installaion of cygwin?
> 
> If it is difficult, I think it is not so nonsense to treat it as
> same as non-cygwin process.

Right you are.  We always said that independent Cygwin installations
are supposed to *stay* independent.

Keep in mind that they don't share the same shared objects in the native
NT namespace and they don't know of each other.  It's not only the
process table but also in-use FIFO stuff, pty info, etc.


Corinna

Re: UNIX nobody/nogroup mapping to which Windows SID/account?

[Corinna Vinschen via Cygwin]

On Mar  3 14:45, Martin Wege via Cygwin wrote:
> Hello,
> 
> How can we map UNIX "nobody"/"nogroup" to Win32 SIDs/accounts? Cygwin
> has entries for "nobody" in /etc/passwd and "nogroup" in /etc/group,
> but these accounts have SIDs returned by /usr/bin/getent passwd and
> /usr/bin/getent group which LookupAccountSidA() does not recognise.
> 
> So what is the correct Win32 solution?

I urge you to read the entire thread starting at

https://sourceware.org/legacy-ml/cygwin/2016-06/msg00347.html

There's a *LOT* of information in there in terms of discussing and
creating the nobody/nogroup mapping.

Bottom line is, there's no nobody account equivalent on Windows and no
resolvable SID/Name pair.  Thus, we decided to use the SID S-1-0-65534
mapped to uid/gid 65534 for this purpose.  This doesn't matter to native
Windows, it's just some foreign SID.  But it's resolvable inside Cygwin:

  $ getent passwd S-1-0-65534
  no+body:*:65534:65534:U-no\body,S-1-0-65534:/:/sbin/nologin
  $  getent group S-1-0-65534
  no+body:S-1-0-65534:65534:


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: openat(O_DIRECT) supported in Cygwin?

[Corinna Vinschen via Cygwin]

On Feb 27 17:46, Martin Wege via Cygwin wrote:
> Hello,
> 
> Does Cygwin support openat(O_DIRECT)? Which effect does O_DIRECT in
> Cygwin have, and which underlying Windows API is used?

https://cygwin.com/cgit/newlib-cygwin/tree/winsup/cygwin/fhandler/base.cc#n587


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH 2/2] Cygwin: remove ENOSHARE and ECASECLASH from _sys_errlist[]

[Corinna Vinschen]

On Feb 27 17:26, Christian Franke wrote:
> Hi Corinna,
> 
> Corinna Vinschen wrote:
> > On Feb 27 13:18, Christian Franke wrote:
> > > ...
> > > 
> > > diff --git a/winsup/cygwin/errno.cc b/winsup/cygwin/errno.cc
> > > index 7d58e62ec..d8c057e51 100644
> > > --- a/winsup/cygwin/errno.cc
> > > +++ b/winsup/cygwin/errno.cc
> > > @@ -167,8 +167,8 @@ const char *_sys_errlist[] =
> > >   /* ESTALE 133 */  "Stale NFS file handle",
> > >   /* ENOTSUP 134 */ "Not supported",
> > >   /* ENOMEDIUM 135 */   "No medium found",
> > > -/* ENOSHARE 136 */ "No such host or network path",
> > > -/* ECASECLASH 137 */   "Filename exists with different case",
> > > +   NULL, /* Was ENOSHARE 136, no longer used. */
> > > +   NULL, /* Was ECASECLASH 137, no longer used. */
> > In terms of politenness, wouldn't it be better to define them as
> > empty strings?  This may be one crash less in already existing
> > binaries...
> 
> Indeed, I missed that case. Patch attached.
> 
> Christian
> 

> From 151da4ef76f84cd0343e6f49aa23de398ca73d1c Mon Sep 17 00:00:00 2001
> From: Christian Franke 
> Date: Tue, 27 Feb 2024 17:21:45 +0100
> Subject: [PATCH 2/2] Cygwin: set ENOSHARE and ECASECLASH _sys_errlist[]
>  entries to empty
> 
> These errno values are no longer used by Cygwin.  Change the entries
> to empty strings instead of NULL to avoid crashes in existing
> binaries directly accessing the table.  Enhance strerror_worker()
> such that empty strings also result in "Unknown error ..." messages.
> Also add a static_assert check for the _sys_errlist[] size.
> 
> Signed-off-by: Christian Franke 
> ---
>  winsup/cygwin/errno.cc | 12 +---
>  1 file changed, 9 insertions(+), 3 deletions(-)

Pushed.

Thanks,
Corinna

Re: How can I get the group SID for a file in cmd.exe (powershell is not an option)?

[Corinna Vinschen via Cygwin]

On Feb 27 15:41, Cedric Blancher via Cygwin wrote:
> Good afternoon!
> 
> How can I get the group SID for a file in cmd.exe (powershell is not an 
> option)?
> Is it possible to add an option to ls -l to list the user/group SIDs too?

No, but you can use Cygwin tools:

 getent -w group $(ls -gn foo | awk '{print $3}') | awk -F: '{print $4}'


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH 2/2] Cygwin: remove ENOSHARE and ECASECLASH from _sys_errlist[]

[Corinna Vinschen]

Hi Christian,

On Feb 27 13:18, Christian Franke wrote:
> From f495fb0e7c2bd3a42f16f81af18c64ffaba9a860 Mon Sep 17 00:00:00 2001
> From: Christian Franke 
> Date: Tue, 27 Feb 2024 13:05:36 +0100
> Subject: [PATCH 2/2] Cygwin: remove ENOSHARE and ECASECLASH from
>  _sys_errlist[]
> 
> These errno values are no longer used by Cygwin.  Also add a
> static_assert check for _sys_errlist[] size.
> 
> Signed-off-by: Christian Franke 
> ---
>  winsup/cygwin/errno.cc | 6 --
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/winsup/cygwin/errno.cc b/winsup/cygwin/errno.cc
> index 7d58e62ec..d8c057e51 100644
> --- a/winsup/cygwin/errno.cc
> +++ b/winsup/cygwin/errno.cc
> @@ -167,8 +167,8 @@ const char *_sys_errlist[] =
>  /* ESTALE 133 */   "Stale NFS file handle",
>  /* ENOTSUP 134 */  "Not supported",
>  /* ENOMEDIUM 135 */"No medium found",
> -/* ENOSHARE 136 */ "No such host or network path",
> -/* ECASECLASH 137 */   "Filename exists with different case",
> +   NULL, /* Was ENOSHARE 136, no longer used. */
> +   NULL, /* Was ECASECLASH 137, no longer used. */

In terms of politenness, wouldn't it be better to define them as
empty strings?  This may be one crash less in already existing
binaries...


Corinna

Re: passwd/group/grent fixes for Cygwin 3.5.1?

[Corinna Vinschen via Cygwin]

On Feb 27 04:21, Dan Shelton via Cygwin wrote:
> On Mon, 26 Feb 2024 at 10:18, Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Feb 26 04:44, Dan Shelton via Cygwin wrote:
> > > Hello!
> > >
> > > Can these 3 commits please be backported to the Cygwin 3.5.1 branch?
> > >
> > > 4 days agoCorinna VinschenCygwin: getgrent/getpwent: avoid local
> > > enumeration...commit | commitdiff | tree
> > > 4 days agoCorinna VinschenCygwin: getgrent: fix local SAM enumeration
> > > on domain...commit | commitdiff | tree
> > > 5 days agoCorinna VinschenCygwin: passwd/group: drop Capability
> > > SIDscommit | commitdiff | tree
> >
> > Did changing db_enum in /etc/nsswitch.conf help on your test machine?
> 
> Yes, just adding "local" to db_enum worked, but it is not much of a

Thanks, good to know.

> solution since I need Administrator rights on the target machine,
> which is not going to happen.

If you're not admin, why do you have to enumerate accounts?  This
is usually not an action required by standard applications.

> Why are local groups excluded by default?

Because enumerating local group means to enumerate the entire AD
group list on DCs, which is not feasible in some scenarios.  And
there is a way to configure this by changing /etc/nsswitch.conf.
If you need this on machines you're not admin of, why not just
ask the admin to enable this?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: cygwin1.dll 3.5.0-1 appears to cause spurious "error 127" with make 4.4.1

[Corinna Vinschen via Cygwin]

On Feb 26 20:08, Dimitry Andric via Cygwin wrote:
> On 26 Feb 2024, at 20:03, Corinna Vinschen  wrote:
> > 
> > On Feb 26 17:34, Dimitry Andric via Cygwin wrote:
> >> Hi,
> >> 
> >> After a recent upgrade of a Cygwin installation, including cygwin1.dll
> >> (see https://cygwin.com/pipermail/cygwin/2024-February/255308.html) to
> >> 3.5.0-1, I now get spurious "error 127" messages from (Cygwin's copy of)
> >> GNU make 4.4.1-2, when it starts external processes and those external
> >> processes exit with a zero exit code.
> >> 
> >> For example, a very simple Makefile:
> >> 
> >> all:
> >> cmd /c echo done
> >> 
> >> Running this a few times in a Cygwin bash prompt, gives:
> >> [...]
> >>  make: *** [Makefile:2: all] Error 127
> > 
> > This is probably what has been fixed for 3.5.1 already in patch
> > aa73e1152426 ("Cygwin: console: Fix exit code for non-cygwin process.")
> > 
> > For testing, please install the most recent test release
> > cygwin-3.6.0-0.62.gc2f6c0415501 via the Cygwin installer and try again.
> 
> Yes, I tried that and it fixes it, thanks!

Great, thanks for your feedback!


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: cygwin1.dll 3.5.0-1 appears to cause spurious "error 127" with make 4.4.1

[Corinna Vinschen via Cygwin]

On Feb 26 17:34, Dimitry Andric via Cygwin wrote:
> Hi,
> 
> After a recent upgrade of a Cygwin installation, including cygwin1.dll
> (see https://cygwin.com/pipermail/cygwin/2024-February/255308.html) to
> 3.5.0-1, I now get spurious "error 127" messages from (Cygwin's copy of)
> GNU make 4.4.1-2, when it starts external processes and those external
> processes exit with a zero exit code.
> 
> For example, a very simple Makefile:
> 
> all:
>   cmd /c echo done
> 
> Running this a few times in a Cygwin bash prompt, gives:
> [...]
>   make: *** [Makefile:2: all] Error 127

This is probably what has been fixed for 3.5.1 already in patch
aa73e1152426 ("Cygwin: console: Fix exit code for non-cygwin process.")

For testing, please install the most recent test release
cygwin-3.6.0-0.62.gc2f6c0415501 via the Cygwin installer and try again.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH 1/4] Cygwin: introduce constexpr errmap_size and errmap[] consistency checks

[Corinna Vinschen]

On Feb 26 15:21, Christian Franke wrote:
> From 947daa02b0b64131626c2ecedb74ca6893aab6c6 Mon Sep 17 00:00:00 2001
> From: Christian Franke 
> Date: Mon, 26 Feb 2024 13:37:33 +0100
> Subject: [PATCH 1/4] Cygwin: introduce constexpr errmap_size and errmap[]
>  consistency checks
> 
> Use constexpr instead of const for errmap[] to allow static_assert
> checks on its values.
> 
> Signed-off-by: Christian Franke 
> ---
>  winsup/cygwin/errno.cc|  2 +-
>  winsup/cygwin/local_includes/errmap.h | 11 ++-
>  2 files changed, 11 insertions(+), 2 deletions(-)

:+1:
Patchset pushed.


Thanks,
Corinna

Re: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to ENODEV

[Corinna Vinschen]

On Feb 26 12:14, Christian Franke wrote:
> Corinna Vinschen wrote:
> > Why so many?  I used winerror.h to populate the list not too long ago,
> > so I wonder why it suddenly has so many more error codes?
> 
> "Required for mozilla-central." - 850 insertions:
> https://sourceforge.net/p/mingw-w64/mingw-w64/ci/ddeb05a
> 
> Most or all would possible never occur with the NTDLL/Win32 API subset used
> by Cygwin.
> 
> Includes interesting codes like "ERROR_NO_WORK_DONE" :-)

LOL.  The only hint I found on that one is in Wine ChangeLog:

 "FormatMessage() now reports ERROR_NO_WORK_DONE error for empty string."


Corinna

Re: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to ENODEV

[Corinna Vinschen]

On Feb 25 10:12, Christian Franke wrote:
> Corinna Vinschen wrote:
> > So the default was EPERM at first and has been changed to EACCES
> > because it "is better for the unknown error case".
> > 
> > I'm open to ideas for an improved error mapping.
> 
> I have no better suggestion for a default errno. Adding a cygwin specific
> one (like ENMFILE, ENOSHARE and ECASECLASH added 2000-2001) is possibly not
> desired.

ENOSHARE and ECASECLASH are not used anymore, fortunately, and ENMFILE
is hopefully never returned to userspace.  It might be a good idea to
remove it from Cygwin's code as well.

> Some thoughts about minor improvements of the errmap.h file:
> - Add error number to each /* ERROR_... */ comment, e.g. /* 2:
> ERROR_FILE_NOT_FOUND */.

Ok.

> - Update /* NUMBER */ comments using current MinGW-w64's winerror.h (~850
> changes).

Why so many?  I used winerror.h to populate the list not too long ago,
so I wonder why it suddenly has so many more error codes?

> - Max errno is 143, so data type size could be reduced from int to uint8_t
> aka unsigned char. Could even add a compile time check by using C++11's
> braced initializers which do not allow narrowing conversions.

Yeah, we could do that.

> - Remove trailing entries which only map to 0.
> - Append a static_assert which checks whether array size matches the last
> mapped error number.

Yeah, not so sure about that.  I'm aware that we only map errors
below 3000 somewhere, but it's no safe bet that it stays that way.

For instance, we handle NT status codes STATUS_TRANSACTIONAL_CONFLICT
and STATUS_TRANSACTION_NOT_ENLISTED and those translate into the TxF
error range between 6800 and 6899.  We don't convert those to userspace
errno yet, but consider having to add them at one point and thus having
to add the 3000 entries from the last used one up the newly used one.

The reason to keep them is to allow us to be lazy about it.  The list
also just takes ~36K, and with the change to uint8_t it only takes
9K, so what?

> I could provide separate patches if desired.

Always welcome!


Thanks,
Corinna

Re: Will all SIDs fit into |SECURITY_MAX_SID_SIZE| bytes ? / was: Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Feb 25 22:04, Roland Mainz via Cygwin wrote:
> On Sat, Feb 24, 2024 at 7:57 PM Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Feb 24 15:38, Roland Mainz via Cygwin wrote:
> > > On Thu, Feb 22, 2024 at 8:11 PM Corinna Vinschen via Cygwin
> > >  wrote:
> > > > On Feb 22 18:38, Roland Mainz via Cygwin wrote:
> > > > > If I switch the current user's group with /usr/bin/newgrp, how can a
> > > > > (native) Win32 process use
> > > > > |GetTokenInformation(GetCurrentThreadToken(), ...)| to find out which
> > > > > group is the new "current group" (e.g. which |TokenInformationClass|
> > > > > should I use) ?
> > > >
> > > >   PSID sidbuf = (PSID) alloca (SECURITY_MAX_SID_SIZE);
> > > [snip]
> > >
> > > Win32/NT API question: All known SIDs will fit into
> > > |SECURITY_MAX_SID_SIZE| bytes, right ? I'm asking because right now
> > > the ms-nfs41-client code assumes that all SIDs use a variable amount
> > > of memory, and we always have to ask the Win32/NT API about the number
> > > of bytes to allocate. If |SECURITY_MAX_SID_SIZE| is the global maximum
> > > limit for all Windows versions, then we could simplify the code a
> > > lot...
> >
> > Yes.  ACLs are size restricted to 64K, though, but that shouldn't be
> > much of a problem usally.
> 
> Erm... why ACLs? I was asking about the memory allocation size for an SID.

I know, and I wrote "Yes".

I mentioned ACLs because ACLs consist of SIDs and if all SIDs take
SECURITY_MAX_SID_SIZE bytes... well, no worries.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: passwd/group/grent fixes for Cygwin 3.5.1?

[Corinna Vinschen via Cygwin]

On Feb 26 04:44, Dan Shelton via Cygwin wrote:
> Hello!
> 
> Can these 3 commits please be backported to the Cygwin 3.5.1 branch?
> 
> 4 days agoCorinna VinschenCygwin: getgrent/getpwent: avoid local
> enumeration...commit | commitdiff | tree
> 4 days agoCorinna VinschenCygwin: getgrent: fix local SAM enumeration
> on domain...commit | commitdiff | tree
> 5 days agoCorinna VinschenCygwin: passwd/group: drop Capability
> SIDscommit | commitdiff | tree

Did changing db_enum in /etc/nsswitch.conf help on your test machine?

Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Will all SIDs fit into |SECURITY_MAX_SID_SIZE| bytes ? / was: Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Feb 24 15:38, Roland Mainz via Cygwin wrote:
> On Thu, Feb 22, 2024 at 8:11 PM Corinna Vinschen via Cygwin
>  wrote:
> > On Feb 22 18:38, Roland Mainz via Cygwin wrote:
> > > If I switch the current user's group with /usr/bin/newgrp, how can a
> > > (native) Win32 process use
> > > |GetTokenInformation(GetCurrentThreadToken(), ...)| to find out which
> > > group is the new "current group" (e.g. which |TokenInformationClass|
> > > should I use) ?
> >
> >   PSID sidbuf = (PSID) alloca (SECURITY_MAX_SID_SIZE);
> [snip]
> 
> Win32/NT API question: All known SIDs will fit into
> |SECURITY_MAX_SID_SIZE| bytes, right ? I'm asking because right now
> the ms-nfs41-client code assumes that all SIDs use a variable amount
> of memory, and we always have to ask the Win32/NT API about the number
> of bytes to allocate. If |SECURITY_MAX_SID_SIZE| is the global maximum
> limit for all Windows versions, then we could simplify the code a
> lot...

Yes.  ACLs are size restricted to 64K, though, but that shouldn't be
much of a problem usally.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Feb 23 22:15, Dan Shelton via Cygwin wrote:
> HOWEVER, there is another Cygwin bug:
> "getent group mywingrp1" does not list any group members, even after
> "net localgroup mywingrp1 mywinuser44 /add", which is a POSIX
> violation.

Not a bug.  Two problems:

- Getting members of a group can be an extremly costly operation
  in a domain or, worse, a domain forest, or even worse, if the 
  domain or domain forest is remote.

- Alonmg the same lines, getting members of a group can be extremly
  costly in big orgs with thousands of users.  Nobody want's to clutter
  up space with the list of members in the "Domain Users" group.

- Permissions to enumerate members of a group are restricted.
  By default only admins and group members are allow to enumerate
  members and this can be restricted further by domain admins.

Therefore we dropped even trying to populate gr_mem, considering
that even in its original form on Unix systems, it's used only
to add supplementary groups.  To do this right on Windows is even
more costly than blindly enumerating.

It's not a bug, it's a feature :)


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Feb 23 19:45, Roland Mainz via Cygwin wrote:
> On Fri, Feb 23, 2024 at 4:47 PM Corinna Vinschen via Cygwin
>  wrote:
> > On Feb 23 14:03, Roland Mainz via Cygwin wrote:
> > > Do you have any idea what is going wrong in this case ?
> >
> > Not sure about that.  I'm not familiar with driver development under
> > Windows.
> 
> Me neither, I'm still new to this whole Windows kernel stuff (coming
> from SUN engineering), but as we need a NFSv4 filesystem
> client at work I'm basically forced at knifepoint to learn as fast as
> I can... ;-/
> 
> > I'd expect that you get the token of the calling thread or, in
> > this case, process as is.
> 
> I think it's the calling thread which makes the Win32 syscall, then
> the MiniRedirector driver (nfs41_driver.sys) gets that security
> context, and uses that to set the impersonation stuff when making the
> upcall to the userland part (nfsd_debug.exe), so that daemon thread
> can impersonate the caller.
> 
> > However, did you try this with a primary group SID being part of the
> > token's supplementary group list, or did you try this with some
> > arbitrary group SID?
> 
> I tried it like this:
> 1. On the Windows machine I created these two new groups:
>  snip 
> WINHOST1:~$ net localgroup cygwingrp1 /add
> WINHOST1:~$ net localgroup cygwingrp2 /add
> WINHOST1:~$ getent group cygwingrp1
> cygwingrp1:S-1-5-21-3286904461-66123-4220857270-1003:197611:
> WINHOST1:~$ getent group cygwingrp2
> cygwingrp2:S-1-5-21-3286904461-66123-4220857270-1004:197612:
>  snip 
> 
> On the Linux NFSv4 server side I added these groups too, and added
> group membership for the matching user:
>  snip 
> root@DERFWNB4966:~# groupadd -g 197611 cygwingrp1
> root@DERFWNB4966:~# groupadd -g 197612 cygwingrp2
> root@DERFWNB4966:~# usermod -a -G cygwingrp1 roland_mainz
> root@DERFWNB4966:~# usermod -a -G cygwingrp2 roland_mainz
>  snip 
> 
> After that /usr/bin/chgrp on Cygwin works on the NFSv4.1 filesystem,
> but if I do a /usr/bin/newgrp+/usr/bin/touch it will not create files
> with that new group, because nfsd_debug.exe only sees the default
> primary group, not the new primary group set by /usr/bin/newgrp.
> 
> Or is there a mistake - do I have to add the current user to the
> Windows localgroup first somehow (like usermod on Linux) ?

Yes, there's a mistake, but in some way it's not your fault.

You have to make sure that the user calling newgrp is member in the
group *and* it has to be in the current user token's TOKEN_GROUP list.

The fact that it (looks like it) works on Cygwin does not mean it
actually worked on the OS level.  See below.

> > I toyed around a bit with this in user space, and it seems I
> > misinterpreted the results when I added the newgrp(1) tool.  The primary
> > group in the token *must* be member of the token's supplementary group
> > list.
> 
> Like on UNIX, right ?

No.  On UNIX it depends if you're a privileged process with the
CAP_SETGID capability.  If so, you can set the pgrp to any arbitrary
group without being asked for credentials.  If you're non-priv'ed,
you can change to any group from the supplementary group list without
being asked for credentials, *or* you're asked for credentials (group
password).

> > The fact that it looks like it works in Cygwin to set the pgrp to
> > an arbitrary SID is apparently based on incorrect error handling.
> >
> > I will fix this in the next couple of days.
> 
> Thanks :-)

In fact I can't fix it and here's why:

Windows only allows to set the new primary group to a group which is
already in the TOKEN_GROUP list of the manipulated user token.  Even
a privileged account can't do that.  Only creating a new token from
scratch will do this.

However, there's a problem.  Consider the OpenSSH server sshd.  It
switches the user account basically like this:

  if (setgid(newgid) != 0)
ERROR;
  if (setuid(newuid) !=0)
ERROR;

Only after the setuid(2) call, Cygwin has full information to create the
new token for the process of the user just logging in.  So during the
setgid(2) call, Cygwin can only operate on the token of the service
account running sshd.

Now, if setgid(2) would fail if the group is not in the user token
of the current effective account, sshd would practically always fail.

So what Cygwin does is to fake success in setgid(2) even if switching
the pgrp in the token failed.  It just stores the information in its
internal user account representation.  Then, when setuid(2) gets called,
it will try to rectify this in the new user's token.

That's why setgid(2) always succeeds if the group exists.  We just don't
have a better way to implement the user account switch.

I just pushed a patch which lets newgrp(1) fail if the gro

Re: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to ENODEV

[Corinna Vinschen]

On Feb 23 19:14, Christian Franke wrote:
> Experiments with damaged USB flash drives and ddrescue revealed that the
> current mapping of these Win32 errors to the fallback EACCES could be
> improved.
> 
> BTW: I wonder why EACCES was selected as the fallback. Source code control
> forensics suggest that this was decided in the last millennium. A related
> comment from CGF added August 2000 persists until today :-)
> /* FIXME: what's so special about EACCESS? */

This goes back until 1997 in pre-CVS times.  There's a ChangeLog entry

  Wed Oct 29 22:43:57 1997  Geoffrey Noer  

[...]
* syscalls.cc (seterrno): on failure, set EACCES instead of EPERM
which is better for the unknown error case

So the default was EPERM at first and has been changed to EACCES
because it "is better for the unknown error case".

I'm open to ideas for an improved error mapping.

> From 8aa19c7fd13dc3790dc271dede8954539bffcd4d Mon Sep 17 00:00:00 2001
> From: Christian Franke 
> Date: Fri, 23 Feb 2024 19:01:09 +0100
> Subject: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to
>  ENODEV
> 
> If a removable (USB) device is disconnected after opening its raw
> device, R/W attempts fail with ERROR_NO_SUCH_DEVICE(433).  If the
> raw device of a partition is used, ERROR_MEDIA_CHANGED(1110) is
> returned instead.  Both are mapped to ENODEV(19) because 
> does not offer a value which better matches ERROR_MEDIA_CHANGED.
> 
> Signed-off-by: Christian Franke 
> ---
>  winsup/cygwin/local_includes/errmap.h | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Pushed.

Thanks,
Corinna

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Feb 23 14:03, Roland Mainz via Cygwin wrote:
> On Thu, Feb 22, 2024 at 8:11 PM Corinna Vinschen via Cygwin
>  wrote:
> > On Feb 22 18:38, Roland Mainz via Cygwin wrote:
> > > If I switch the current user's group with /usr/bin/newgrp, how can a
> > > (native) Win32 process use
> > > |GetTokenInformation(GetCurrentThreadToken(), ...)| to find out which
> > > group is the new "current group" (e.g. which |TokenInformationClass|
> > > should I use) ?
> >
> >   PSID sidbuf = (PSID) alloca (SECURITY_MAX_SID_SIZE);
> >   NTSTATUS status;
> >   ULONG size;
> >
> >   status = NtQueryInformationToken (hProcToken, TokenPrimaryGroup,
> > sidbuf, SECURITY_MAX_SID_SIZE,
> > );
> 
> Well, it works in the case of an "hello world" application, but if I
> stuff that into the nfsd_daemon (NFSv4.1 ms-nfs41-client client
> daemon) it always prints the default primary group, even if the
> current thread should impersonate another user - or in this case even
> the same user, but a different primary group (e.g. see
> https://github.com/kofemann/ms-nfs41-client/blob/master/sys/nfs41_driver.c#L1367).
> 
> Do you have any idea what is going wrong in this case ?

Not sure about that.  I'm not familiar with driver development under
Windows.  I'd expect that you get the token of the calling thread or, in
this case, process as is.

However, did you try this with a primary group SID being part of the
token's supplementary group list, or did you try this with some
arbitrary group SID?

I toyed around a bit with this in user space, and it seems I
misinterpreted the results when I added the newgrp(1) tool.  The primary
group in the token *must* be member of the token's supplementary group
list.

The fact that it looks like it works in Cygwin to set the pgrp to
an arbitrary SID is apparently based on incorrect error handling.

I will fix this in the next couple of days.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: BUG: Cygwin getent group refuses to list group added with net localgroup mygrp1 /add

[Corinna Vinschen via Cygwin]

On Feb 23 09:10, Dan Shelton via Cygwin wrote:
> On Tue, 20 Feb 2024 at 21:45, Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Feb 20 17:24, Dan Shelton via Cygwin wrote:
> > > On Tue, 20 Feb 2024 at 10:57, Corinna Vinschen via Cygwin
> > >  wrote:
> > > >
> > > > On Feb 20 05:13, Dan Shelton via Cygwin wrote:
> > > > > Hello!
> > > > >
> > > > > I think I found a rather nasty bug:
> > > > > 1. Add a new group with "net localgroup mygrp1 /add"
> > > > > 2. Then list all groups with "getent group", the new group mygrp1 will
> > > > > not show up. "getent group mygrp1" will list it, but this is useless
> > > > > in my case, as I need the group listed by "getent group"
> > > >
> > > > I guess we can rule out a domain controller but is this a standalone
> > > > machine or a domain member machine?
> > >
> > > Standalone test machine, stock Windows 10 installed 2 days ago,
> > > current patches, Cygwin 3.5 from 2 days ago.
> > >
> > > >
> > > > What's the setting of your /etc/nsswitch.conf?
> > >
> > > Only comments in that file, i.e. empty
> >
> > Please set the db_enum: line to
> >
> >   db_enum: all
> >
> > and try again after exiting from the shell window.  Have a look at
> > https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-mapping-nsswitch
> > for the settings.  By default account enumeration only returns
> > the bare minimum for speed.
> >
> > Btw, there *is* a bug in that code, but it affects only domain
> > menber machines, afaics.
> 
> I saw you did a couple of commits related to "getent group". Is any of
> these commits fixing the problem that a new Win32 local group created
> with "net localgroup mygrp1 /add"  does not show up in the output of
> "getent group"?

Did you read my previous reply and tried what I wrote?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Switching groups with newgrp - how to get the new group with |GetTokenInformation()| ?

[Corinna Vinschen via Cygwin]

On Feb 22 18:38, Roland Mainz via Cygwin wrote:
> Hi!
> 
> 
> 
> If I switch the current user's group with /usr/bin/newgrp, how can a
> (native) Win32 process use
> |GetTokenInformation(GetCurrentThreadToken(), ...)| to find out which
> group is the new "current group" (e.g. which |TokenInformationClass|
> should I use) ?

  PSID sidbuf = (PSID) alloca (SECURITY_MAX_SID_SIZE);
  NTSTATUS status;
  ULONG size;

  status = NtQueryInformationToken (hProcToken, TokenPrimaryGroup,
sidbuf, SECURITY_MAX_SID_SIZE,
    );


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: binutils >= 2.41 makes .rsrc section read-only

[Corinna Vinschen via Cygwin]

On Feb 21 09:27, Corinna Vinschen via Cygwin wrote:
> On Feb 20 21:39, Jeremy Drake via Cygwin wrote:
> > On Tue, 20 Feb 2024, Corinna Vinschen wrote:
> > 
> > > On Feb 19 21:41, Jeremy Drake via Cygwin wrote:
> > > > 1) is there actually a good reason that _cygheap_start is in the .rsrc 
> > > > and
> > > > not the .cygheap section?
> > >
> > > As you know we got rid of this way to define the cygheap, but
> > > _cygheap_start was just defined this way so it's the address marking
> > > the start address of the cygheap.
> > >
> > > It was just the way it was.  It's probably just as well to move
> > >
> > >   _SYM (_cygheap_start) = .;
> > >
> > > to the begining of the .cygheap section.  Did you try?
> > 
> > I hadn't; I have now and it seems to work fine.
> 
> I'm glad to read that.
> 
> > I basically assumed there
> > was some reason I wasn't aware of for putting it in the .rsrc section, so
> > focused more on trying to figure out how to get binutils to clear the
> > readonly section flag.  When I couldn't figure that out, I figured I'd ask
> > here both about the "institutional knowledge" if there was a known reason
> > why it needed to be in the .rsrc section and because I figured some people
> > knowledgable in binutils pe stuff might also read this and be able to give
> > me another idea on making the .rsrc section read/write again if it did
> > need to stay in that section.
> 
> I think there was some historical reason to put in .rsrc, or rather,
> into the section preceeding .cygheap, but I don't remember what it was.
> Might have had something to do with stripping the cygwin DLL or some
> such...

Actually, searching the history shows commit ce95c6407ef6a from
2005-06-01, which moves the definition of __cygheap_start from
the .cygheap section to the .idata section.  However, the commit
message only contains the hint "cygwin.sc: Modernize. [...]"
There's no other reason given ¯\_(ツ)_/¯


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: binutils >= 2.41 makes .rsrc section read-only

[Corinna Vinschen via Cygwin]

On Feb 20 21:39, Jeremy Drake via Cygwin wrote:
> On Tue, 20 Feb 2024, Corinna Vinschen wrote:
> 
> > On Feb 19 21:41, Jeremy Drake via Cygwin wrote:
> > > 1) is there actually a good reason that _cygheap_start is in the .rsrc and
> > > not the .cygheap section?
> >
> > As you know we got rid of this way to define the cygheap, but
> > _cygheap_start was just defined this way so it's the address marking
> > the start address of the cygheap.
> >
> > It was just the way it was.  It's probably just as well to move
> >
> >   _SYM (_cygheap_start) = .;
> >
> > to the begining of the .cygheap section.  Did you try?
> 
> I hadn't; I have now and it seems to work fine.

I'm glad to read that.

> I basically assumed there
> was some reason I wasn't aware of for putting it in the .rsrc section, so
> focused more on trying to figure out how to get binutils to clear the
> readonly section flag.  When I couldn't figure that out, I figured I'd ask
> here both about the "institutional knowledge" if there was a known reason
> why it needed to be in the .rsrc section and because I figured some people
> knowledgable in binutils pe stuff might also read this and be able to give
> me another idea on making the .rsrc section read/write again if it did
> need to stay in that section.

I think there was some historical reason to put in .rsrc, or rather,
into the section preceeding .cygheap, but I don't remember what it was.
Might have had something to do with stripping the cygwin DLL or some
such...


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 20 10:33, Kevin Ushey via Cygwin wrote:
> Hi Corinna,
> 
> I just updated 
> https://gist.github.com/kevinushey/cdbd15cdf22e5cdcd094b0ad80347dce
> with that output (windbg-output-2.txt); let me know if that gives you
> what you need.

Thank you!  Not sure I'll follow up on this, ARM64 assembler still
puzzles me.  However, this way I can at least take a look.


Thanks again,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: BUG: Cygwin getent group refuses to list group added with net localgroup mygrp1 /add

[Corinna Vinschen via Cygwin]

On Feb 20 17:24, Dan Shelton via Cygwin wrote:
> On Tue, 20 Feb 2024 at 10:57, Corinna Vinschen via Cygwin
>  wrote:
> >
> > On Feb 20 05:13, Dan Shelton via Cygwin wrote:
> > > Hello!
> > >
> > > I think I found a rather nasty bug:
> > > 1. Add a new group with "net localgroup mygrp1 /add"
> > > 2. Then list all groups with "getent group", the new group mygrp1 will
> > > not show up. "getent group mygrp1" will list it, but this is useless
> > > in my case, as I need the group listed by "getent group"
> >
> > I guess we can rule out a domain controller but is this a standalone
> > machine or a domain member machine?
> 
> Standalone test machine, stock Windows 10 installed 2 days ago,
> current patches, Cygwin 3.5 from 2 days ago.
> 
> >
> > What's the setting of your /etc/nsswitch.conf?
> 
> Only comments in that file, i.e. empty

Please set the db_enum: line to

  db_enum: all

and try again after exiting from the shell window.  Have a look at
https://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-mapping-nsswitch
for the settings.  By default account enumeration only returns
the bare minimum for speed.

Btw, there *is* a bug in that code, but it affects only domain
menber machines, afaics.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: BUG: gettimeofday() didn't handle first argument being NULL

[Corinna Vinschen via Cygwin]

On Feb 20 19:15, 109224573 via Cygwin wrote:
> this code works on Linux, but not msys2:
> 
> #include
> #include
> int main() {
>  struct timezone tz;
>  puts("a");
>  gettimeofday(NULL, tz);
>  puts("b");
> }
> 
> 
> I guess the reason would be
> winsup/cygwin/times.cc

Thanks for the bug report.  Fixed in git.  The test release
cygwin-3.6.0-0.46.g70653fd8f1db containing this patch will be
available shortly.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: binutils >= 2.41 makes .rsrc section read-only

[Corinna Vinschen via Cygwin]

On Feb 19 21:41, Jeremy Drake via Cygwin wrote:
> This is probably the right thing to do, but breaks building msys2-runtime
> (read: cygwin) 3.3, because the _cygheap_start symbol is actually in the
> .rsrc section and code very early attempts to memset _cygheap_start.
> Couple of questions:
> 
> 1) is there actually a good reason that _cygheap_start is in the .rsrc and
> not the .cygheap section?

As you know we got rid of this way to define the cygheap, but
_cygheap_start was just defined this way so it's the address marking
the start address of the cygheap.

It was just the way it was.  It's probably just as well to move

  _SYM (_cygheap_start) = .;

to the begining of the .cygheap section.  Did you try?


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: BUG: Cygwin getent group refuses to list group added with net localgroup mygrp1 /add

[Corinna Vinschen via Cygwin]

On Feb 20 05:13, Dan Shelton via Cygwin wrote:
> Hello!
> 
> I think I found a rather nasty bug:
> 1. Add a new group with "net localgroup mygrp1 /add"
> 2. Then list all groups with "getent group", the new group mygrp1 will
> not show up. "getent group mygrp1" will list it, but this is useless
> in my case, as I need the group listed by "getent group"

I guess we can rule out a domain controller but is this a standalone
machine or a domain member machine?

What's the setting of your /etc/nsswitch.conf?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

Hi Kevin,

On Feb 15 20:13, Corinna Vinschen via Cygwin wrote:
> On Feb 15 09:46, Kevin Ushey via Cygwin wrote:
> > https://gist.github.com/kevinushey/cdbd15cdf22e5cdcd094b0ad80347dce.
> [...]
>   0001`802b7054 db030094 bl  ntdll!#RtlpReferenceCurrentDirectory 
> (1802b7fc0)

I'm not familiar with ARM assembler, but a quick search showed that the
`bl' opcode is basically a subroutine call.

So ntdll!#RtlpReferenceCurrentDirectory is not the address of the struct
pointer, as I surmised, but the address of the function (probably)
fetching the address of the struct pointer.

Would you mind to send the assembler output for this function as well?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Group name madness: "CurrentSession" vs. "LogonSessionId_0_111590" vs. "NT AUTHORITY\LogonSessionId_0_111590" ...

[Corinna Vinschen via Cygwin]

On Feb 19 08:11, Roland Mainz via Cygwin wrote:
> Hi!
> 
> 
> 
> While implementing chown/chgrp support in the ms-nfs41-client (Windows
> NFSv4.1 client filesystem driver) I encountered an issue with group
> names.
> 
> I tried a simple $ chgrp CurrentSession myfile #, which returned
> "Permission denied".

Works for me:

  $ chgrp CurrentSession x.c
  $ ls -l x.c
  -rw-r--r-- 1 corinna CurrentSession 906 Jan 23 21:40 x.c
  $ icacls x.c
  x.c VINSCHEN\corinna:(R,W,D,WDAC,WO)
  NT AUTHORITY\LogonSessionId_0_2027255:(R)
  Everyone:(R)

  Successfully processed 1 files; Failed processing 0 files
  $

> OK, /usr/bin/id -a shows that the current user is a member of this
> group, so I added a new group called "CurrentSession" to the NFSv4.1
> server side ([1]).

Just don't do that.  This group is a *temporary* group which is created
from your login session ID.  Your login session is basically a pair of
two 32 bit values, generated via NtAllocateLocallyUniqueId().  From
there, Windows creates a SID

 S-1-5-5--

These days, when you call LookupAccountSidW() on this SID, you get the
info

  Domain: NT AUTHORITY
  Namme:  LogonSessionId_0_2027255

This SID is used inside your user token to connect your token with the
login session for which this token has been created.  It can be used for
authorization purposes, but it's not meant to be used as a SID in file
ownership, even though it's possible (basically any SID can be added
to files).

So, what's that about "CurrentSession" in Cygwin?

The reason is that this is historically what Interix did.  Keep in
mind that Interix (as well as Cygwin, but that doesn't matter here)
exist since Windows NT 3.5 times.

The above naming mechanism didn't exist at the time.  If you fetched the
Logon SID from the token and called LookupAccountSidW() on it, the call
failed, i. e., there was no name attached to the SID.

So... given that a Logon SID is unique to the logon session, given that
the SID has no unambiguous name attached, and given that the Logon SID
makes *no sense at all* outside its session, the Interix guys decided to
solve this little problem like this:

- If the inspected token contains the Logon SID equivalent to the
  current user session, call it "CurrentSession" with unique gid 4095.

- If the inspected token contains a Logon SID which is not attached
  to the current Logon Session we're running in, call it "OtherSession"
  with the non-unique gid 4094.

This is the model Cygwin follows as well.  The Logon SIDs shouldn't
really concern you in terms of file access.  From the user perspective
they are barely informative at all.  They *could* make sense in terms
of temporary files, but I don't know of any useful example.

> But that did not work either, /usr/bin/chgrp # still failed.

I can't tell you why it didn't work for you while it works out of the
box for me.  However, there isn't much sense in there to attach these
SIDs to a file.  Logout, login again, and you will have a new Logon SID
anyway ¯\_(ツ)_/¯


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin sqlite locking debug options still available?

[Corinna Vinschen via Cygwin]

On Feb 17 14:38, Martin Wege via Cygwin wrote:
> On Tue, Feb 6, 2024 at 12:31 PM Martin Wege  wrote:
> >
> > Hello!
> >
> > Is the debug mode for sqlite locking mode as described in
> > https://cygwin.cygwin.narkive.com/nkjMGQga/test-sqlite3-3-7-17-1-1-7-19-locking-feature
> > soehow still available?
> >
> > We sometimes get sqlite locking errors, and like to investigate what
> > and why this fails
> 
> Corinna?

Sorry, I'm not the sqlite maintainer.

Also, I skimmed the above mail thread and I don't see any mention of a
specific debug mode in the above mail thread.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 15 09:46, Kevin Ushey via Cygwin wrote:
> Thanks -- I've put the associated WinDbg output up at
> https://gist.github.com/kevinushey/cdbd15cdf22e5cdcd094b0ad80347dce.
> (Sharing it externally just because it's relatively large.)

Thank you!

> The important thing to note is that #RtlGetCurrentDirectory_U appears
> to be valid ARM assembly, but not x86_64 assembly. My hypothesis here
> is that the stub is used to allow emulated x86_64 processes to call
> back into native ARM code... which would further confirm that
> disabling the find_fast_cwd_pointer checks for ARM is the correct
> choice.

I guess so, but I'm quite puzzled how the emulator is supposed
to know that it just left the x86_64 assembler code.  Magic!!!

Checking the ARM code, it might even be possible to come up with
a real solution, given this line in the first few bytes of
ntdll!#RtlGetCurrentDirectory_U:

  0001`802b7054 db030094 bl  ntdll!#RtlpReferenceCurrentDirectory 
(1802b7fc0)

This seems to fetch the address of the pointer we're interested in.
It's unlikely (but not impossible) that the structure differs from the
same struct as on x86_64.

We may have a chance there, but I would feel better if I could debug my
code by myself.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 14 13:49, Kevin Ushey via Cygwin wrote:
> Thanks for your patience. Here's what I've got for the assembly around
> get_dir. I added a bit of debug logging just so I could get the
> function addresses:

First of all, thanks for taking the time to debug this further!

> C:\cygwin\bin>cygpath
> get_dir = 0x7FFB85E251B0
> rcall = 0x7FFB85E251CB
> 
> And here's what WinDbg reports:
> 
> ntdll!EXP+#RtlGetCurrentDirectory_U:
> 7ffb`85e251b0 488bc4 mov rax, rsp
> 7ffb`85e251b3 48895820   mov qword ptr [rax+20h], rbx
> 7ffb`85e251b7 55 pushrbp
> 7ffb`85e251b8 5d pop rbp
> 7ffb`85e251b9 e9721e2b00 jmp ntdll!#RtlGetCurrentDirectory_U
> (7ffb860d7030)
> 7ffb`85e251be cc int 3
> 7ffb`85e251bf cc int 3
> ntdll!EXP+#RtlGetCurrentPeb:
> 7ffb`85e251c0 488bc4 mov rax, rsp
> 7ffb`85e251c3 48895820   mov qword ptr [rax+20h], rbx
> 7ffb`85e251c7 55 pushrbp
> 7ffb`85e251c8 5d pop rbp
> 7ffb`85e251c9 e9e2e82400 jmp ntdll!#RtlGetCurrentPeb (7ffb86073ab0)
> 7ffb`85e251ce cc int 3
> 7ffb`85e251cf cc int 3
> 
> I'm not sure what the "EXP+#" prefix here means, but it appears to
> just be a stub that calls into the real implementation now?

Yes, that seems to be the case, same EXP+#for RtlGetCurrentPeb.

> So, if I'm understanding correctly:
> 
> 1. Cygwin was expecting to find a 'call' instruction somewhere
> following (the procedure address for) RtlGetCurrentDirectory_U;
> 2. The expected 'call' instruction no longer exists; however, by
> chance, there is a 'jmp' later on that includes '0xe8' in the bytes of
> the address to be jumped to;

That's it.  Chances are high that the above ntdll code was always more
or less the same and find_fast_cwd_pointer() failed all the time.  Only,
it never found the "e8" and so nothing bad happened.

So, as long as we don't know how to fix this correctly, my patch
4e77fa9b8bf4 ("Cygwin: find_fast_cwd: don't run assembler checking code
on ARM64") seems the right thing to do.

What annoys me is that I don't have access to ARM64 myself.  I tried
to install Windows for ARM64 on a QEmu emulator, but the VM always
failed to boot into Windows, it just sat there and used up CPU.
I even contemplated installing an Azure ARM64 VM, but I always shy
away from cloud services at the point they ask you for your size of
shoe and your social security number.

Anyway...

> For reference, here's what I see on my Intel Windows 11 machine, where
> all works as normal (showing up to the "call" instruction)

I wonder if you would be willing to grant us a view into the
ntdll!#RtlGetCurrentDirectory_U function jumped to from
ntdll!EXP+#RtlGetCurrentDirectory_U.  Per your above assembler output,
that would be at 7ffb860d7030.  Would you mind to post the WinDBG
assembler output of that function as well, even if just for curiosity?


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 14 10:52, Corinna Vinschen via Cygwin wrote:
> On Feb 13 15:48, Kevin Ushey via Cygwin wrote:
> > Here's a bit more information from a debug build of cygwin; here I'm
> > just trying to launch cygpath.exe:
> > 
> > (gdb) f 1
> > #1  0x7ffa0123ba1f in find_fast_cwd_pointer () at
> > ../../../../winsup/cygwin/path.cc:4526
> > 4526  const uint8_t *lock = (const uint8_t *)
> > 
> > (gdb) bt
> > #0  memmem (haystack=, hs_len=,
> > needle=0x7ffa0142e531 <_C_collate_locale+59857>, ne_len=4) at
> > ../../../newlib/libc/string/memmem.c:161
> > #1  0x7ffa0123ba1f in find_fast_cwd_pointer () at
> > ../../../../winsup/cygwin/path.cc:4526
> > [...]
> 
> Ok, so we know now which memmem call fails, so the next question is,
> why?  The haystack address is unfortunately still optimized out in
> memmem, so it looks like the newlib subdir is still optimized.
> But it's pretty certainly a wrong haystack address.  This address has
> been fetched from
> 
>   const uint8_t *rcall = (const uint8_t *) memchr (get_dir, 0xe8, 80);
>   ...
>   const uint8_t *use_cwd = rcall + 5 + peek32 (rcall + 1);
> 
> Chances are high, that the call instruction found by rcall is bogus.

Can you run this in GDB?  I'd love to see the assembler code starting at
address `get_dir'...


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 13 15:48, Kevin Ushey via Cygwin wrote:
> Here's a bit more information from a debug build of cygwin; here I'm
> just trying to launch cygpath.exe:
> 
> (gdb) f 1
> #1  0x7ffa0123ba1f in find_fast_cwd_pointer () at
> ../../../../winsup/cygwin/path.cc:4526
> 4526  const uint8_t *lock = (const uint8_t *)
> 
> (gdb) bt
> #0  memmem (haystack=, hs_len=,
> needle=0x7ffa0142e531 <_C_collate_locale+59857>, ne_len=4) at
> ../../../newlib/libc/string/memmem.c:161
> #1  0x7ffa0123ba1f in find_fast_cwd_pointer () at
> ../../../../winsup/cygwin/path.cc:4526
> [...]

Ok, so we know now which memmem call fails, so the next question is,
why?  The haystack address is unfortunately still optimized out in
memmem, so it looks like the newlib subdir is still optimized.
But it's pretty certainly a wrong haystack address.  This address has
been fetched from

  const uint8_t *rcall = (const uint8_t *) memchr (get_dir, 0xe8, 80);
  ...
  const uint8_t *use_cwd = rcall + 5 + peek32 (rcall + 1);

Chances are high, that the call instruction found by rcall is bogus.

> Diving in further is a bit beyond my level of experience with Windows
> internals.

It would be great if you could run this under GDB and check the
assembler code inside the ntdll.dll this code is scanning.  You
don't really need knowledge of Windows internals, it's basically
just really dumb scanning of assembler code for certain instructions,
and the expected assembler code is x86_64.

> I wonder if it'd be
> worth having cygwin respect an environment variable like
> CYGWIN_USE_FAST_CWD_POINTER, so that users could opt out of this
> particular optimization if it happens to break on new versions of
> Windows, as I have seemingly encountered.

My latest patch already does it, no env var required.

> As an aside, I had to make a couple of small patches to side-step gcc
> warnings (converted to errors) during build; I tried building with "-g
> -Og" flags.

I'm usually building with -ggdb only, or with -ggdb -O2.

I just added -Og, and checked the below code.  I'm not so sure about the
usefulness of -Og, given that all three warnings are false positives...

> $ git diff
> diff --git a/winsup/cygwin/fhandler/fifo.cc b/winsup/cygwin/fhandler/fifo.cc
> index efea508ae..f288e874a 100644
> --- a/winsup/cygwin/fhandler/fifo.cc
> +++ b/winsup/cygwin/fhandler/fifo.cc
> @@ -669,7 +669,7 @@ fhandler_fifo::create_shmem (bool only_open)
>  {
>HANDLE sect;
>OBJECT_ATTRIBUTES attr;
> -  NTSTATUS status;
> +  NTSTATUS status = 0;
>LARGE_INTEGER size = { .QuadPart = sizeof (fifo_shmem_t) };
>SIZE_T viewsize = sizeof (fifo_shmem_t);
>PVOID addr = NULL;

My gcc (11.3.0) did not show this, even with -Og.  However, the status
var always gets set, see lines 685 - 695.

> diff --git a/winsup/cygwin/tty.cc b/winsup/cygwin/tty.cc
> index bf7c6010f..2cd4ae6ed 100644
> --- a/winsup/cygwin/tty.cc
> +++ b/winsup/cygwin/tty.cc
> @@ -323,7 +323,7 @@ tty::wait_fwd ()
>   thread when the last data is transfered. */
>const ULONGLONG sleep_in_nat_pipe = 16;
>const ULONGLONG time_to_wait = sleep_in_nat_pipe * 2 + 1/* margine */;
> -  ULONGLONG elapsed;
> +  ULONGLONG elapsed = 0;
>while (fwd_not_empty
>  || (elapsed = GetTickCount64 () - fwd_last_time) < time_to_wait)
>  {

fwd_not_empty is false for a start, so GetTickCount64() is called at
least once.

> diff --git a/winsup/utils/kill.cc b/winsup/utils/kill.cc
> index fb45e4c9d..bcabcd47c 100644
> --- a/winsup/utils/kill.cc
> +++ b/winsup/utils/kill.cc
> @@ -73,7 +73,7 @@ print_version ()
>  static const char *
>  strsigno (int signo)
>  {
> -  static char sigbuf[8];
> +  static char sigbuf[32];
> 
>if (signo > 0 && signo < SIGRTMIN)
>  return sys_sigabbrev[signo];

We're calling snprintf here for a reason, and we know that the
number has never more than 2 digits.  Oh well.


Thanks,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 13 10:01, Kevin Ushey via Cygwin wrote:
> On Tue, Feb 13, 2024 at 8:25 AM Corinna Vinschen wrote:
> > On Feb 13 11:09, Corinna Vinschen via Cygwin wrote:
> > > Other than that, the only thing you really could do at this point is to
> > > check Cygwin's find_fast_cwd_pointer() function and go through this
> > > function step by step, diving into the assembler code this function
> > > inspects trying to find out which of the memmem calls fails and how to
> > > fix it.  And send a patch eventually.
> > > [...]
> > > One thing we can do is to skip the find_fast_cwd_pointer() code entirely
> > > when running on AArch64, but it's a bit annoying, given this wasn't
> > > necessary in previous AArch64 builds.
> >
> > I've just pushed a (temporary?) patch to do just that.  This will be
> > in the test build cygwin-3.6.0-0.37.g4e77fa9b8bf4 which will be
> > installable via Cygwin's installer in an hour or two.
> 
> Thank you for taking a look so quickly -- I can confirm your patch
> fixes things for me; the installer now runs to completion and the
> Cygwin64 Terminal + other installed tools all function correctly.

Thanks for confirming.  However, if you see *any* chance to check
Cygwin's find_fast_cwd_pointer() pointer function and try to find
out why it crashes, that would be very helpful.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 13 11:09, Corinna Vinschen via Cygwin wrote:
> On Feb 12 14:38, Kevin Ushey via Cygwin wrote:
> > For reference, I first bumped into this when using Git Bash as bundled
> > with Git for Windows, but it sounds like the underlying issue may be
> > in Cygwin. See https://github.com/git-for-windows/git/issues/4808 for
> > more details.
> 
> There is, however, the stacktrace from your above issue report on
> github, which makes more sense, and, incidentally, occurs in code
> following cygwin_split_path in the .text segment:
> 
> (gdb) bt
> #0  0x0002101eeaf3 in memmem () from [...]
> #1  0x000210095d91 in cwdstuff::override_win32_cwd(bool, unsigned int) ()
>from [...]
> #2  0x00021009642e in cwdstuff::set(path_conv*, char const*) () from [...]
> #3  0x000210047025 in dll_crt0_1(void*) () from [...]
> #4  [...]
> 
> This points to something going wrong during startup, while trying
> to evaluate the global pointer to the hidden ntdll.dll struct we
> called, for want of an official expression, FAST_CWD structure.
> 
> If we can rely on frame 1, a call to memmem crashes, that would
> mean the crash occurs in find_fast_cwd_pointer().
> 
> > Does any of this sound familiar? Is there anything else I can do to
> > get more information here; e.g. are there builds of Cygwin with debug
> > symbols published somewhere, or should I try producing my own debug
> > build?
> 
> Well, you could have mentioned that this is on AArch64.  Fortunately
> you did in the github case, but next time, please tell us here, too.
> 
> I assume you can't run any Cygwin binary, even from outside the
> installer?
> 
> Other than that, the only thing you really could do at this point is to
> check Cygwin's find_fast_cwd_pointer() function and go through this
> function step by step, diving into the assembler code this function
> inspects trying to find out which of the memmem calls fails and how to
> fix it.  And send a patch eventually.
> 
> However, there isn't much point there as long as this is an insider
> build.  In the past, the code often changed and was uncritical in the
> next official release.
> 
> One thing we can do is to skip the find_fast_cwd_pointer() code entirely
> when running on AArch64, but it's a bit annoying, given this wasn't
> necessary in previous AArch64 builds.

I've just pushed a (temporary?) patch to do just that.  This will be
in the test build cygwin-3.6.0-0.37.g4e77fa9b8bf4 which will be
installable via Cygwin's installer in an hour or two.

Please try if this works for you and report back.  You can use this
DLL as workaround for the time being then.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Win32 account SID lookup if user and group have the same name?

[Corinna Vinschen via Cygwin]

On Feb 13 02:12, Dan Shelton via Cygwin wrote:
> Hello!
> 
> How can I get the SID for either user XOR group if both have the same
> name? Remember Cygwin, as Unix and Linux, allow that a user "danfsh"
> can have his own group "danfsh". That works fine there, but on Win32
> there is only LookupAccountSidA() which has no way to distinguish
> whether you want a user SID or a group SID.

User and group "danfsh" are NOT a user and a group.  Rather, there is
actually only one user account "danfsh" and Windows fills in your user
token with the *same* SID as primary group.

The weird thing in Windows is that it has no strict concept of groups.
The user token contains primary and supplementary group info, but the OS
can fill them with basically any SID it can lay its hands on.

And at one point (I'm deliberately vague here since I don't know exactly
when this occured), Microsoft started to fill in the primary group of
some local account with the user SID, rather than with the "None"(*)
group as is usually the default for local accounts.

(*) That's the local domain SID with RID 513. The name of the group
is actually localized.

> How can I get the SID for group "danfsh", or the SID for user "danfsh"?

It's the same SID and it's your user SID.  There can't be a group with
the same name as a user account in the same user DB.  Each account in
the local domain or in an AD domain has to have a unique account name.


HTH,
Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: Cygwin installer hangs when running post-install scripts

[Corinna Vinschen via Cygwin]

On Feb 12 14:38, Kevin Ushey via Cygwin wrote:
> Hello,
> 
> I'm seeing an issue when attempting to install Cygwin where the
> installer hangs while trying to run postinstall scripts (more
> specifically, /etc/postinstall/0p_000_autorebase.dash). When the hang
> occurs, I see a 'dash.exe' process chewing up 100% of a CPU. If I
> attach to the process with WinDbg, I see:
> 
> (940.978): Access violation - code c005 (first chance)
> First chance exceptions are reported before any exception handling.
> This exception may be expected and handled.
> cygwin1!memmem+0xcb:
> 7ffa`490cbb1b 410fb60424  movzx   eax,byte ptr [r12]
> ds:7ffa`552651f7=??
> 0:000> k
>  #   Arch   Child-SP  RetAddr   Call Site
> 00AMD64 0007`c930 7ffa`48f8028f cygwin1!memmem+0xcb
> 01AMD64 0007`ca90 7ffa`48f80a49
> cygwin1!cygwin_split_path+0x3a0
> 02AMD64 0007`cb20 7ffa`48f37071
> cygwin1!cygwin_split_path+0xb5a
> 03AMD64 0007`cc50 7ffa`48f35e08
> cygwin1!cygwin_dll_init+0x26b
> 04AMD64 0007`cd80 7ffa`48f35e86 cygwin1!_assert+0x23f0
> 05AMD64 0007`cdd0 ` cygwin1!_assert+0x246e
> 
> I'm not sure how much I can trust the stack trace here, though.

I don't think this stacktrace makes any sense. cygwin_split_path
is a function only exported for historical reasons and not used
internally.  Dash doesn't call it either.

> For reference, I first bumped into this when using Git Bash as bundled
> with Git for Windows, but it sounds like the underlying issue may be
> in Cygwin. See https://github.com/git-for-windows/git/issues/4808 for
> more details.

There is, however, the stacktrace from your above issue report on
github, which makes more sense, and, incidentally, occurs in code
following cygwin_split_path in the .text segment:

(gdb) bt
#0  0x0002101eeaf3 in memmem () from [...]
#1  0x000210095d91 in cwdstuff::override_win32_cwd(bool, unsigned int) ()
   from [...]
#2  0x00021009642e in cwdstuff::set(path_conv*, char const*) () from [...]
#3  0x000210047025 in dll_crt0_1(void*) () from [...]
#4  [...]

This points to something going wrong during startup, while trying
to evaluate the global pointer to the hidden ntdll.dll struct we
called, for want of an official expression, FAST_CWD structure.

If we can rely on frame 1, a call to memmem crashes, that would
mean the crash occurs in find_fast_cwd_pointer().

> Does any of this sound familiar? Is there anything else I can do to
> get more information here; e.g. are there builds of Cygwin with debug
> symbols published somewhere, or should I try producing my own debug
> build?

Well, you could have mentioned that this is on AArch64.  Fortunately
you did in the github case, but next time, please tell us here, too.

I assume you can't run any Cygwin binary, even from outside the
installer?

Other than that, the only thing you really could do at this point is to
check Cygwin's find_fast_cwd_pointer() function and go through this
function step by step, diving into the assembler code this function
inspects trying to find out which of the memmem calls fails and how to
fix it.  And send a patch eventually.

However, there isn't much point there as long as this is an insider
build.  In the past, the code often changed and was uncritical in the
next official release.

One thing we can do is to skip the find_fast_cwd_pointer() code entirely
when running on AArch64, but it's a bit annoying, given this wasn't
necessary in previous AArch64 builds.


Corinna

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple