Re: [PATCH cygport] Add customization support for announce command

[Christian Franke via Cygwin-apps]

Adam Dinwoodie via Cygwin-apps wrote:

On Tue, Apr 30, 2024 at 12:27:35PM +0200, Christian Franke via Cygwin-apps 
wrote:

Jon Turney wrote:

On 10/03/2024 16:33, Christian Franke via Cygwin-apps wrote:

+    /bin/bash -c "cd ${top} || exit 1
+${HOMEPAGE+HOMEPAGE=${HOMEPAGE@Q}}
+P=${P@Q}; PF=${PF@Q}; PN=${PN@Q}; PR=${PR@Q}; PV=(${PV[*]@Q})
+${SMTP_SENDER+SMTP_SENDER=${SMTP_SENDER@Q}}
+${SMTP_SERVER+SMTP_SERVER=${SMTP_SERVER@Q}}
+${SMTP_SERVER_PORT+SMTP_SERVER_PORT=${SMTP_SERVER_PORT@Q}}
+${SMTP_ENCRYPTION+SMTP_ENCRYPTION=${SMTP_ENCRYPTION@Q}}
+${SMTP_USER+SMTP_USER=${SMTP_USER@Q}}
+${SMTP_PASS+SMTP_PASS=${SMTP_PASS@Q}}
+${cmd}
+" $0 ${msg} || error "Command '\${${cmdvar}} ${msg}'
(cwd=${top}) failed"
+}

Sorry I didn't notice this before, and I am terrible at writing shell,
but perhaps you could share the reasoning behind writing this as above,
and not as, e.g.

(cd ${top} && env BLAH ${cmd})

avoiding all the verbiage in the description of ANNOUNCE_EDITOR about it
being fed into 'bash -c' (and hence getting evaluated twice??) rather
than just run?



None of the mentioned variables are exported to the environment by cygport.
I wanted to keep this fact in the subshell. Therefore the assignments are
added to the script instead of passing via env(ironment). The latter won't
even work with the PV variable because arrays could not be exported.

Variables would not be evaluated twice. For example in the rare case that
someone uses something like

SMTP_SERVER="smtp.$(hostname -d)"

in cygport.conf, this would immediately expand to
SMTP_SERVER="smtp.some.domain". The above

${SMTP_SERVER+SMTP_SERVER=${SMTP_SERVER@Q}}

would expand to

SMTP_SERVER=${SMTP_SERVER@Q}

and then to

SMTP_SERVER='smtp.some.domain'

(The @Q bash extension ensures proper quoting).

Using a subshell created by ( ... ) would achieve the behaviour you're
after, without requiring nearly so much quote handling.  The code Jon
has pulled out could be rewritten as below; using ( ... ) would mean
that everything happens in a subshell and the exports don't affect the
rest of the environment.

```
(
cd "$top"
export HOMEPAGE P PF PN PR PV SMTP_SENDER SMTP_SERVER SMTP_SERVER_PORT 
SMTP_ENCRYPTION SMTP_USER SMTP_PASS


This unnecessarily exports all variables (except PV, see below) to the 
subcommands run by $cmd.




"$cmd" "$msg" || error "Command $cmd $msg (cwd=${top}) failed"


This would limit $cmd to simple commands instead of multiline scripts. 
This reduces flexibility and some of the examples I provided in my 
original post would no longer work:

https://sourceware.org/pipermail/cygwin-apps/2024-February/043501.html



)
```

I've removed the array handling for $PV, as it's not an array; possibly
you've confused it with $PVP?



No. PV it is initialized as a regular shell variable but is later 
changed to an array by appending the PVP array.


/bin/cygport:
...
declare PV=${VERSION}
...
    PV=$(echo ${PF} | sed -e "s/${PN}\-\(.*\)\-${PR}$/\1/");
...
declare -r  PV=(${PV} ${PVP[*]});
...

$ git blame bin/cygport.in | grep 'declare -r  PV='
deb528a88 (Yaakov Selkowitz   2009-12-31 08:05:52 + 397) declare -r  
PV=(${PV} ${PVP[*]});


Bash silently ignores 'export PV'.


   In any case, there is no way to pass an
array to "$cmd" unless "$cmd" is itself a Bash function, as there's no
standard way to store anything other than strings in environment
variables.


That's why I use 'PV=(${PV[*]@Q})' as a prefix of the configured $cmd 
string instead of passing any new environment to $cmd.




I've also removed the `|| return 1` part, since cygport runs with `set
-e` enabled so you only want to catch non-zero return codes if you want
specific error handling.


There is no 'return 1' is my patch.



Finally, I've also added "$msg" to the arguments to "$cmd"; that seems
to be missing and seems to be critical to this working at all!


$msg is not missing in my patch but passed to the launched /bin/bash as $1.



Alternatively, if you really wanted to avoid the export statement, the
below will achieve the same thing; the only use of the subshell at this
point is to avoid the `cd` changing the working directory for the rest
of the script.

```
(
cd "$top"
HOMEPAGE="$HOMEPAGE" P="$P" PF="$PF" PN="$PN" PR="$PR" PV="$PV" SMTP_SENDER="$SMTP_SENDER" SMTP_SERVER="$SMTP_SERVER" 
SMTP_SERVER_PORT="$SMTP_SERVER_PORT" SMTP_ENCRYPTION="$SMTP_ENCRYPTION" SMTP_USER="$SMTP_USER" SMTP_PASS="$SMTP_PASS" "$cmd" "$msg" || error "Command $cmd $msg 
(cwd=${top}) failed"
)
```


Same problem with missing flexibility for $cmd as above.

Re: [PATCH cygport] Add check of SPDX expression provided by LICENSE variable

[Christian Franke via Cygwin-apps]

Brian Inglis via Cygwin-apps wrote:

On 2024-04-30 15:07, Christian Franke via Cygwin-apps wrote:

Brian Inglis via Cygwin-apps wrote:

On 2024-04-30 11:45, Christian Franke via Cygwin-apps wrote:
The new script uses the SPDX webpages to create the license file. I 
didn't find a usable single license list at https://github.com/spdx


As usual, it is easier if you clearly state the purpose of the file 
you want, and its desired properties, like data content, format, etc.



What about:
https://spdx.github.io/license-list-data/


This is apparently a draft version of 
https://spdx.org/licenses/index.html which is used by the script to 
generate the local license file.


Strip out the table entries and create what you want with a command or 
script.


The spdx-check script from the patch optionally (-m, -u) downloads 
https://spdx.org/licenses/index.html and creates the local spdx-licenses 
file intended to distribute with cygport. The file is grep'able.and 
reduced to the bare minimum for this use case.






and everything under:
https://github.com/spdx/license-list-data



I didn't find a single file which lists the licenses there.


GH does not always make access easy, ...


... including that github.com is still unreachable via IPv6 without 
NAT64 (except for downloads from raw.githubusercontent.com) ...



... with its limited online displays and fixed display orders, and 
searches return a lot of junk, without easy access to better searching 
in context, but try:


https://github.com/spdx/license-list-data/blob/main/licenses.md

which also has xrefs to the text files; also there are:

https://github.com/spdx/license-list-data/blob/main/json/licenses.json 

https://github.com/spdx/license-list-data/blob/main/json/exceptions.json 



which can be easily processed using `jq`.



Indeed, thanks. I obviously missed these files when I wrote the 
spdx-check script some month ago.


The current file format used by the script could then be created with:

url="https://raw.githubusercontent.com/spdx/license-list-data/main/json;

wget -O - "$url/licenses.json" \
| jq -j '
    .licenses[] | (
  if .isDeprecatedLicenseId then "!" else "" end,
  .licenseId,
  "\n"
    )'

wget -O - "$url/exceptions.json" \
| jq -j '
    .exceptions[] | (
  if .isDeprecatedLicenseId then "!&" else "&" end,
  .licenseExceptionId,
  "\n"
    )'

This adds these license ids not yet mentioned at 
https://spdx.org/licenses/index.html:

AMD-newlib, BSD-2-clause-first-lines, Catharon, HPND-UC-export-US,
MIT-Khronos-old, NCL, OAR, Sun-PPP-2000, pkgconf, threeparttable, xzoom

I could provide a new patch with an updated script if desired.

Re: [PATCH cygport] Add check of SPDX expression provided by LICENSE variable

[Christian Franke via Cygwin-apps]

Brian Inglis via Cygwin-apps wrote:

On 2024-04-30 11:45, Christian Franke via Cygwin-apps wrote:
...

Attached.
The new script uses the SPDX webpages to create the license file. I 
didn't find a usable single license list at https://github.com/spdx


What about:

https://spdx.github.io/license-list-data/



This is apparently a draft version of 
https://spdx.org/licenses/index.html which is used by the script to 
generate the local license file.




and everything under:

https://github.com/spdx/license-list-data


I didn't find a single file which lists the licenses there.

[PATCH cygport] Add check of SPDX expression provided by LICENSE variable

[Christian Franke via Cygwin-apps]

Jon Turney via Cygwin-apps wrote (thread "[PATCH cygport] Add 
repro-finish command"):

...
PS: I have a local script which checks SPDX Identifiers and 
expressions. Any interest to add this to cygport and then check 
LICENSE settings?


Oh, yes please. That sounds like a good idea.



Attached.

The new script uses the SPDX webpages to create the license file. I 
didn't find a usable single license list at https://github.com/spdx


The data/spdx-licenses file is not included in the patch. It could be 
generated from the source dir with:


$ tools/spdx-check -f data/spdx-licenses -m
...
data/spdx-licenses: created

$ sha1sum data/spdx-licenses
80a19d6891d08bf34113464464ee12308374c792 *data/spdx-licenses

The changes to the meson files are guessed. I didn't test the meson 
build yet.


--
Regards,
Christian

From 61f75757fa8e9118207cc09cf4a621aac8a4da78 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 30 Apr 2024 19:28:01 +0200
Subject: [PATCH] Add check of SPDX expression provided by LICENSE variable

The new script 'tools/spdx-checks' checks a SPDX license expression.
License identifiers are provided by the new file 'spdx-licenses'
which could be created by the script from the related SPDX webpages.
---
 bin/cygport.in|  17 
 data/meson.build  |   1 +
 tools/meson.build |   1 +
 tools/spdx-check  | 198 ++
 4 files changed, 217 insertions(+)
 create mode 100644 tools/spdx-check

diff --git a/bin/cygport.in b/bin/cygport.in
index 15bd559e..3166beba 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -41,6 +41,7 @@ declare -r  _cygport_version=@VERSION@;
 declare -r _privdatadir=@pkgdatadir@;
 declare -r _privclassdir=@cygclassdir@;
 declare -r _privlibdir=@cygpartdir@;
+declare -r _privtoolsdir=@pkgdatadir@/tools;
 declare -r _privgnuconfigdir=@gnuconfigdir@;
 declare -r _privsysconfdir=@sysconfdir@;
 
@@ -489,6 +490,22 @@ do
fi
 done
 
+if [ "${LICENSE+y}" = "y" ]
+then
+   if ! _out=$(${_privtoolsdir}/spdx-check -f 
${_privdatadir}/spdx-licenses "${LICENSE}" 2>&1)
+   then
+   warning "LICENSE='${LICENSE}' is invalid:"
+   echo "${_out}"
+   elif [ "${_out:+y}" = "y" ]
+   then
+   warning "LICENSE='${LICENSE}' has warnings:"
+   echo "${_out}"
+   else
+   inform "LICENSE='${LICENSE}' is valid"
+   fi
+   unset _out
+fi
+
 for restrict in ${RESTRICT//,/ }
 do
declare _CYGPORT_RESTRICT_${restrict//-/_}_=1
diff --git a/data/meson.build b/data/meson.build
index 51c6a5fd..e83a90fe 100644
--- a/data/meson.build
+++ b/data/meson.build
@@ -2,6 +2,7 @@ datadocs = files('cygport.conf', 'mirrors')
 
 install_data('mirrors',
  'sample.cygport',
+ 'spdx-licenses',
  install_dir: pkgdatadir)
 
 install_data('gnuconfig/config.guess',
diff --git a/tools/meson.build b/tools/meson.build
index acd83926..96d8d19e 100644
--- a/tools/meson.build
+++ b/tools/meson.build
@@ -1,6 +1,7 @@
 tools = files(
 'deb2targz',
 'pkgrip',
+'spdx-check',
 'sysrootize'
 )
 
diff --git a/tools/spdx-check b/tools/spdx-check
new file mode 100644
index ..bffcaae0
--- /dev/null
+++ b/tools/spdx-check
@@ -0,0 +1,198 @@
+#! /bin/bash
+#######
+#
+# spdx-check - check SPDX license expression
+#
+# Copyright (C) 2024 Christian Franke
+#
+# SPDX-License-Identifier: BSD-3-Clause
+#
+
+
+set -e -o pipefail
+myname=$0
+
+# SPDX license list web pages
+spdx_url_lic="https://spdx.org/licenses/index.html;
+spdx_url_exc="https://spdx.org/licenses/exceptions-index.html;
+
+# Default license file
+def_spdx_file="$(dirname "$myname")/spdx-licenses"
+
+usage()
+{
+  cat <&2
+  exit 1
+}
+
+warning()
+{
+  echo "Warning:" "$@" >&2
+}
+
+check_spdx_id()
+{
+  local id=$1
+  local m m_id
+
+  if ! [ -f "$spdx_file" ]; then
+warning "Missing '$spdx_file' - SPDX identifier '$1' not checked"
+return 0
+  fi
+
+  # SPDX identifiers are case insensitive but the correct case is recommended
+  m=$(grep -Ei -m 1 "^!?&?${id//+/\\+}\$" "$spdx_file" 2>/dev/null) \
+|| error "Unknown SPDX identifier '$id'"
+
+  # TODO: Distinguish licenses and exceptions
+  m_id=${m#!}; m_id=${m_id#&}
+
+  [ "$m_id" = "$id" ] || warning "It is recommended to use '$m_id' instead of 
'$id'"
+  [ "$m" = "${m#!}" ] || warning "SPDX identifier '$m_id' is deprecated"
+}
+
+check_spdx_expr()
+{
+  local x=$1
+  local f s t
+
+  # Insert spaces around tokens to simplify parsing
+  x=" $x "; x=${x//(/ ( }; x=${

Re: [PATCH cygport] Add customization support for announce command

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 10/03/2024 16:33, Christian Franke via Cygwin-apps wrote:

Jon Turney wrote:

On 23/02/2024 11:23, Christian Franke via Cygwin-apps wrote:

Christian Franke wrote:
The email generated by the cygport announce command is useful, but 
actual use cases are somewhat limited due to the hard-coded email 
submission.


The attached patch adds more flexibility. The patch is on top of 
the "Use correct wording if only one package is announced" patch.


Slightly changed patch attached. Also adjusted to new version of 
"Use correct wording if only one package is announced" patch.




[...]

Thanks for this.


Possible (better?) alternative names for the new settings:
ANNOUNCEMENT_EDITOR
ANNOUNCEMENT_MAILER


Hmmm... I think "ANNOUNCE_EDITOR" and "ANNOUNCE_MAILER" would be
the best for clarity and conciseness.


New patch attached. Is still on top of "Use correct wording ..." patch.

I also added HOMEPAGE to the propagated variables as this should be 
included in an announcement.


Thanks.


+    /bin/bash -c "cd ${top} || exit 1
+${HOMEPAGE+HOMEPAGE=${HOMEPAGE@Q}}
+P=${P@Q}; PF=${PF@Q}; PN=${PN@Q}; PR=${PR@Q}; PV=(${PV[*]@Q})
+${SMTP_SENDER+SMTP_SENDER=${SMTP_SENDER@Q}}
+${SMTP_SERVER+SMTP_SERVER=${SMTP_SERVER@Q}}
+${SMTP_SERVER_PORT+SMTP_SERVER_PORT=${SMTP_SERVER_PORT@Q}}
+${SMTP_ENCRYPTION+SMTP_ENCRYPTION=${SMTP_ENCRYPTION@Q}}
+${SMTP_USER+SMTP_USER=${SMTP_USER@Q}}
+${SMTP_PASS+SMTP_PASS=${SMTP_PASS@Q}}
+${cmd}
+" $0 ${msg} || error "Command '\${${cmdvar}} ${msg}' 
(cwd=${top}) failed"

+}


Sorry I didn't notice this before, and I am terrible at writing shell, 
but perhaps you could share the reasoning behind writing this as 
above, and not as, e.g.


(cd ${top} && env BLAH ${cmd})

avoiding all the verbiage in the description of ANNOUNCE_EDITOR about 
it being fed into 'bash -c' (and hence getting evaluated twice??) 
rather than just run?





None of the mentioned variables are exported to the environment by 
cygport. I wanted to keep this fact in the subshell. Therefore the 
assignments are added to the script instead of passing via 
env(ironment). The latter won't even work with the PV variable because 
arrays could not be exported.


Variables would not be evaluated twice. For example in the rare case 
that someone uses something like


SMTP_SERVER="smtp.$(hostname -d)"

in cygport.conf, this would immediately expand to 
SMTP_SERVER="smtp.some.domain". The above


${SMTP_SERVER+SMTP_SERVER=${SMTP_SERVER@Q}}

would expand to

SMTP_SERVER=${SMTP_SERVER@Q}

and then to

SMTP_SERVER='smtp.some.domain'

(The @Q bash extension ensures proper quoting).

Re: [PATCH cygport] Increase _FORTIFY_SOURCE level from 2 to 3 in CFLAGS

[Christian Franke via Cygwin-apps]

ASSI via Cygwin-apps wrote:

Christian Franke via Cygwin-apps writes:

_FORTIFY_SOURCE=3 is supported by Cygwin 3.5.0 headers and Cygwin gcc
13.2.1 test release.

Silently falls back to level 2 if level 3 is unsupported (older
headers or gcc) or to level 0 if unsupported at all (C++, clang).

Well, if only that was the case…

--8<---cut here---start->8---
  from /usr/include/w32api/windows.h:9,
  from 
/mnt/share/cygpkgs/libarchive/libarchive.x86_64/src/libarchive-3.7.4/test_utils/test_common.h:88,
  from 
/mnt/share/cygpkgs/libarchive/libarchive.x86_64/src/libarchive-3.7.4/tar/test/test.h:38,
  from 
/mnt/share/cygpkgs/libarchive/libarchive.x86_64/src/libarchive-3.7.4/tar/test/test_extract_tar_lrz.c:25:
/usr/include/w32api/_mingw_mac.h:319:8: warning: #warning Using 
_FORTIFY_SOURCE=2 (level 3 requires __builtin_dynamic_object_size support) 
[-Wcpp]
   319 | #  warning Using _FORTIFY_SOURCE=2 (level 3 requires 
__builtin_dynamic_object_size support)
--8<---cut here---end--->8---

Can't we conditiohnalize this to depend on the actual compiler support?


This is a bogus warning. Sorry, my bad.

In my contribution of _FORTIFY_SOURCE support to MinGW-w64 from 2019, I 
didn't realize that these warnings also appear if only Win32 API 
includes (windows.h, ...) are used. The related internal macros have 
only an effect if MinGW-w64 runtime includes (stdio.h, string.h, ...) 
are used.


Meantime this has been fixed upstream:
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/f8e088e

--
Regards,
Christian

Re: Cygwin a bit slow

[Christian Franke via Cygwin]

J M via Cygwin wrote:

...

Specifically for this problem, I have investigated the problem and can be
related to pipes and antivirus.

Specifically
while true
do
echo ABC | grep AAA
done

It makes the cpu of that antivirus go up.


This is as expected because malware scanners hook into Win32 API's 
CreateProcess*() calls which are also used by the fork()/exec() 
emulation of Cygwin. Each run of 'grep' above uses at least two 
CreateProcess*() calls.


This quick test shows how many 'date' commands could be run per second:

$ while :; do date +%s; done | uniq -c
...
 65 1712742865 <== Windows Defender off
 66 1712742866
 66 1712742867
 64 1712742868
 61 1712742869
 51 1712742870 <== Windows Defender turned on
 51 1712742871
 49 1712742872
 45 1712742873
 53 1712742874
 54 1712742875
...

The above could even slow down to 1-2 per second with certain malware 
scanners if expensive heuristics (which may also generate a lot of false 
positives, BTW) is enabled. So one problem is the lousy performance of 
CreateProcess*() calls. This is not Cygwin-specific but affects typical 
Cygwin use cases like shell scripts.


Using bash builtins in the above example speeds it up to ~21000/second 
on the same very old box:


$ while :; do printf '%(%s)T\n'; done | uniq -c

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: util-linux-2.39.3-1: libblkid returns invalid physical_sector_size

[Christian Franke via Cygwin]

Christian Franke via Cygwin wrote:

,,,
BTW, according to the Linux kernel sources, BLKPBSZGET etc return 
'unsigned int' and not 'unsigned long' since first appearance in 
2.6.32-rc3 (2009?):


https://elixir.bootlin.com/linux/v2.6.32-rc3/source/block/ioctl.c#L276
https://elixir.bootlin.com/linux/v2.6.32-rc3/source/block/compat_ioctl.c#L743 


https://elixir.bootlin.com/linux/v6.8.2/source/block/ioctl.c#L533

So I don't understand why the mentioned code would be correct for Linux.



It is likely an upstream regression from an 1+ year old commit. I filed 
a GH issue:

https://github.com/util-linux/util-linux/issues/2904


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: util-linux-2.39.3-1: libblkid returns invalid physical_sector_size

[Christian Franke via Cygwin]

Bruce Jerrick via Cygwin wrote:

Downgrading to util-linux-2.33.3-3 does not help. The related code
differs, but has the same problem.


I take that back. The above should read "util-linux-2.33.1-3".



But it was OK in util-linux-2.33.1-3 .


Yes, this is correct. I possibly downgraded util-linux, but forgot 
libblkid1.



--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: util-linux-2.39.3-1: libblkid returns invalid physical_sector_size

[Christian Franke via Cygwin]

Hi Mark,

Mark Geisert via Cygwin wrote:

Hi Christian,

On 3/31/2024 1:11 AM, Christian Franke via Cygwin wrote:

Testcase:

# cygcheck -f /sbin/fdisk.exe
util-linux-2.39.3-1

# /sbin/fdisk.exe -l /dev/sdd
Disk /dev/sdd: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 34359738880 bytes
I/O size (minimum/optimal): 34359738880 bytes / 34359738880 bytes

[...valuable investigation and patch suggestion elided...]

Your suggested patch looks fine to me.  I have added it to the patch 
deck for a new util-linux 2.39.3-2, which has just been uploaded.  The 
patch allows fdisk.exe to report the three correct values in my 
limited testing.

Thanks for the report and the patch!


You're welcome.

BTW, according to the Linux kernel sources, BLKPBSZGET etc return 
'unsigned int' and not 'unsigned long' since first appearance in 
2.6.32-rc3 (2009?):


https://elixir.bootlin.com/linux/v2.6.32-rc3/source/block/ioctl.c#L276
https://elixir.bootlin.com/linux/v2.6.32-rc3/source/block/compat_ioctl.c#L743
https://elixir.bootlin.com/linux/v6.8.2/source/block/ioctl.c#L533

So I don't understand why the mentioned code would be correct for Linux.


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

util-linux-2.39.3-1: libblkid returns invalid physical_sector_size

[Christian Franke via Cygwin]

Testcase:

# cygcheck -f /sbin/fdisk.exe
util-linux-2.39.3-1

# /sbin/fdisk.exe -l /dev/sdd
Disk /dev/sdd: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 34359738880 bytes
I/O size (minimum/optimal): 34359738880 bytes / 34359738880 bytes
Disklabel type: dos
Disk identifier: 0x0ac1a23

Device Boot Start   End   Sectors   Size Id Type
/dev/sdd1    2048 976769023 976766976 465.8G  7 HPFS/NTFS/exFAT

Partition 1 does not start on physical sector boundary.

# printf '0x%016x\n' 34359738880
0x00080200


The problem is that libblkid expects the results of BLKIOMIN, BLKIOOPT 
and BLKPBSZGET as 64 bit 'unsigned long' but Cygwin only returns 32 bit 
'int':


- util-linux-2.39.3/libblkid/src/topology/ioctl.c:
...
static const struct topology_val {

    long  ioc;

    /* functions to set probing result */
    int (*set_ulong)(blkid_probe, unsigned long);
    int (*set_int)(blkid_probe, int);
    int (*set_u64)(blkid_probe, uint64_t);

} topology_vals[] = {
    { BLKALIGNOFF, NULL, blkid_topology_set_alignment_offset },
    { BLKIOMIN, blkid_topology_set_minimum_io_size },
    { BLKIOOPT, blkid_topology_set_optimal_io_size },
    { BLKPBSZGET, blkid_topology_set_physical_sector_size },
#ifdef BLKGETDISKSEQ
    { BLKGETDISKSEQ, .set_u64 = blkid_topology_set_diskseq },
#endif
    /* we read BLKSSZGET in topology.c */
};
...

- util-linux-2.39.3/libblkid/src/topology/topology.c:
...
struct blkid_struct_topology {
    unsigned long    alignment_offset;
    unsigned long    minimum_io_size;
    unsigned long    optimal_io_size;
    unsigned long    logical_sector_size;
    unsigned long    physical_sector_size;
    unsigned long   dax;
    uint64_t    diskseq;
};
...
int blkid_topology_set_physical_sector_size(blkid_probe pr, unsigned 
long val)

...

- newlib-cygwin/winsup/cygwin/fhandler/floppy.cc:
...
    case BLKIOMIN:
  debug_printf ("BLKIOMIN");
  *(int *)buf = (int) bytes_per_sector;
  break;
    case BLKIOOPT:
  debug_printf ("BLKIOOPT");
  *(int *)buf = (int) bytes_per_sector;
  break;
    case BLKPBSZGET:
  debug_printf ("BLKPBSZGET");
  *(int *)buf = (int) bytes_per_sector;
  break;
...


A quick fix which only works on LE platforms:

- util-linux-2.39.3/libblkid/src/topology/ioctl.c:
...
static int probe_ioctl_tp(blkid_probe pr,
...
        union {
            unsigned long ul;
            int i;
            uint64_t u64;
-        } data;
+        } data = { 0 };
...


Downgrading to util-linux-2.33.3-3 does not help. The related code 
differs, but has the same problem.


The fdisk variant in busybox-1.36.1-2 is not affected.

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Christian Franke via Cygwin]

Corinna Vinschen via Cygwin wrote:

On Mar 23 10:57, Christian Franke via Cygwin wrote:

On Mar 21 16:15, Corinna Vinschen via Cygwin wrote:

On Mar 21 09:58, Christian Franke via Cygwin wrote:

..

I just gave it a try on W11. The results are even more funny than I
anticipated:

...

So, yeah, with your observations especially on older W10 versions and
with 8.1 doing the same thing, I guess we can safely drop the extra call
to RtlGetNtVersionNumbers now.

Or:
Keep RtlGetVersionNumbers, compare result with RtlGetVersion and emit debug
output if both differ. This would allow to diagnose a possible future
reappearance of the issue.

Good idea! Do you want to create a patch?



I tried, but without success. Unfortunately debug_printf() does not work 
because wincap.init() is called before get_cygwin_startup_info() which 
calls strace.activate().



--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Christian Franke via Cygwin]

On Mar 21 16:15, Corinna Vinschen via Cygwin wrote:

On Mar 21 09:58, Christian Franke via Cygwin wrote:

..

I just gave it a try on W11. The results are even more funny than I
anticipated:

...

So, yeah, with your observations especially on older W10 versions and
with 8.1 doing the same thing, I guess we can safely drop the extra call
to RtlGetNtVersionNumbers now.


Or:
Keep RtlGetVersionNumbers, compare result with RtlGetVersion and emit 
debug output if both differ. This would allow to diagnose a possible 
future reappearance of the issue.




   After such a long time, I don't know
on which version of Windows we observed the problem.


Some developer preview?

Meantime I also tested with Win10 1607, 1703, 1709, 1803 VM snapshots 
with same result.


--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [ITP] afflib 3.7.20-1

[Christian Franke via Cygwin-apps]

Christian Franke wrote:

marco atzeri wrote:

On Wed, Mar 6, 2024 at 11:26 PM Christian Franke via Cygwin-apps  wrote:

...
Should I also rename libtsk to libtsk19 in the planned sleutkit-*-2
package which will add afflib support ?

yes please


The original package is only a few days old and has possibly only a
small but experienced audience, so I expect not much worries if the
change will be explained in the announcement.

not worries at all if you use

libtsk19_OBSOLETES=libtsk

see
https://cygwin.github.io/cygport/pkg_pkg_cygpart.html#PKG_OBSOLETES


Thanks for the info. I will do this that way when libafflib0 package 
is available.




Just for info: This worked as expected. Setup automatically selected 
"Uninstall" for libtsk.

sleuthkit 4.12.1-2

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* sleuthkit-4.12.1-2
* sleuthkit-perl-4.12.1-2
* libtsk19-4.12.1-2
* libtsk-devel-4.12.1-2

The Sleuth Kit (TSK) is a collection of command line tools for disk
images.  It allows to analyze volume and filesystem data, examine disk
layout, recover deleted files, etc.  Many filesystems are supported,
including FAT, exFAT, NTFS, Ext2/3/4, HFS and UFS.  Raw, split raw and
AFF files are accepted as input images.

The sleuthkit-perl package contains the perl scripts 'mactime' and 'sorter'.

https://www.sleuthkit.org/sleuthkit/

Changes since sleuthkit-4.12.1-1:

- Support for image files in AFF (Advanced Forensic Format) has been 
enabled.


- The sleuthkit package does no longer require perl because the perl 
scripts have been moved to the new package sleuthkit-perl.


- The runtime package libtsk has been renamed to libtsk19.

The source package supports reproducible builds except for libtsk.dll.a
from the libtsk-devel package.

Regards,
Christian Franke

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
<https://cygwin.com/mailman/options/cygwin-announce>, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
<https://sourceware.org/lists.html#unsubscribe>.

New: afflib 3.7.20-1

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* afflib-tools-3.7.20-1
* libafflib0-3.7.20-1
* libafflib-devel-3.7.20-1

The Advanced Forensic Format (AFF) is a file format for storing computer
forensic information.  It supports metadata, compression, encryption and
signing.

The afflib-tools package contains tools to examine, convert, compare, copy,
encrypt, decrypt and sign AFF files.

https://github.com/sshock/AFFLIBv3

The source package supports reproducible builds except for libafflib.dll.a
from the libafflib-devel package.

Regards,
Christian Franke

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
<https://cygwin.com/mailman/options/cygwin-announce>, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
<https://sourceware.org/lists.html#unsubscribe>.

Re: [ITP] afflib 3.7.20-1

[Christian Franke via Cygwin-apps]

On Wed, 6 Mar 2024 23:26:05 +0100, Christian Franke wrote:

Jon Turney wrote:
...


be added only when needed for new not backward compatible releases. 
The upstream afflib project is mostly idling, so I don't expect any 
new major lib versions in the near future.


If course, I could rename it to libafflib0 if desired.


As far as I know, there is no cost for doing this, and it saves grief 
if upstream ever bumps the soversion.


Also, it's probably best to explicitly list the filename with 
soversion in the CONTENTS, so that if upstream ever does change the 
soversion, it will be detected as a packaging failure, rather than 
producing a package with a mismatch between the soversion in it's 
name and in it's contents.


Good point, new cygport file is attached.


Any further issues with this ITP?

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Christian Franke via Cygwin]

Corinna Vinschen via Cygwin wrote:

On Mar 20 12:39, Christian Franke via Cygwin wrote:

Corinna Vinschen via Cygwin wrote:

You have to create an application with an application manifest not
supporting your OS.

For Cygwin apps, this occured when you built, say, an executable under
Windows 8.1 before Windows 10 support was added to the Cygwin toolchain:
the manifest linked to the Cygwin executable didn't yet contain a GUID
entry for Windows 10 support.

In this case, RtlGetVersion returns an OS version 6.3 even when running
under the 10.0 kernel.  This behaviour exists back 'til Windows Vista.

Could not reproduce the latter on Win10. I tested with recent Win10 and
Win11 and also found a Win10 1511 (and Slackware 1.1.2, Win3.1, OS/2, ...)
in my VM image museum.

Regardless of the exe manifest, RtlGetVersion and RtlGetNtVersionNumbers
return the correct versions:
10.0.22621 (Win11 22H2)
10.0.19045 (Win10 22H2)
10.0.10586 (Win10 1511)

Without a manifest, GetVersionEx returns:
6.2.9200 (Win8)

Please check on commit 48511f3d3847c.  It was a real, existing problem
at the time.  I wouldn't have added the RtlGetNtVersionNumbers call
just for fun.



Of course.
I learned about the existence of RtlGetNtVersionNumbers via this commit 
and was curious, which Windows versions were affected but found none. 
This suggests that only some early Win10 versions were affected.


--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

bmake 20240314-1

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* bmake-20240314-1
* bmake-extras-20240314-1
* bmake-extras-python-20240314-1

bmake is a portable version of the NetBSD make(1) utility. It is similar
to GNU make, even though the syntax for the advanced features supported
in Makefiles is very different.

The bmake-extras package contains many additional *.mk files and some
support scripts which are only required if used in Makefiles.

The bmake-extras-python package contains a faster alternative to the
'meta2deps.sh' script.

https://www.crufty.net/help/sjg/bmake.htm

This is an update to the latest upstream release.

Changes since bmake-20240301-1:

- The variables .MAKE.OS and HOST_OS are now set to "Cygwin" instead of 
"CYGWIN_NT-X.Y-BUILD".


- Added sys/Cygwin.mk file.

- The error message "sed: can't read /etc/group" does no longer occur.

- The above changes are included in the upstream source release.

- The doc files for the *.mk files are now included in /usr/share/doc/bmake.

- The bmake-extras package does no longer require python because the 
'meta2deps.py' script has been moved to the new package bmake-extras-python.


The source package supports reproducible builds.

Regards,
Christian Franke

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
<https://cygwin.com/mailman/options/cygwin-announce>, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
<https://sourceware.org/lists.html#unsubscribe>.

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Christian Franke via Cygwin]

Corinna Vinschen via Cygwin wrote:

On Mar 19 09:18, Bill Stewart via Cygwin wrote:

On Tue, Mar 19, 2024 at 9:01 AM Richard Campbell wrote:

On Tue, Mar 19, 2024 at 9:04 AM Bill Stewart via Cygwin 

wrote:


Can you please clarify the circumstances under which the RtlGetVersion
function "may not return the correct values"?

"Originally, using RtlGetVersion instead of GetVersionEx was supposed to
fix the fact that GetVersionInfo returns the wrong kernel version if the
executable has been built with an old manifest (or none at all), starting
with Windows 8.1.  Either this never really worked as desired and our
testing was flawed, or this has been changed again with Windows 10, so
that RtlGetVersion does the kernel faking twist as well.  Since we're
only reading the value in the first process in a process tree. the entire
process tree is running with a wrong OS version information in that case.

Fortunately, the (undocumented) RtlGetNtVersionNumbers function is not
affected by this nonsense, so we simply override the OS version info
fields with the correct values now."

https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=48511f3d3847c35352d09cded56e25f0c1b22bc9


Interesting. I have not yet been able to find a scenario where the
RtlGetVersion function gets "manifested" like GetVersionEx.

I wrote a small Win32 console utility for displaying and testing OS
information (requires Windows Vista/Server 2008 or later):

https://github.com/Bill-Stewart/osinfo

It uses RtlGetVersion, and this function works correctly for me in all
current Windows versions (Windows 10, Server 2016, Windows 11, Server 2019,
Server 2022, etc.).

I'm not sure of the exact scenario that led to the "RtlGetVersion is
subject to manifesting" conclusion, but I can't reproduce it.

You have to create an application with an application manifest not
supporting your OS.

For Cygwin apps, this occured when you built, say, an executable under
Windows 8.1 before Windows 10 support was added to the Cygwin toolchain:
the manifest linked to the Cygwin executable didn't yet contain a GUID
entry for Windows 10 support.

In this case, RtlGetVersion returns an OS version 6.3 even when running
under the 10.0 kernel.  This behaviour exists back 'til Windows Vista.


Could not reproduce the latter on Win10. I tested with recent Win10 and 
Win11 and also found a Win10 1511 (and Slackware 1.1.2, Win3.1, OS/2, 
...) in my VM image museum.


Regardless of the exe manifest, RtlGetVersion and RtlGetNtVersionNumbers 
return the correct versions:

10.0.22621 (Win11 22H2)
10.0.19045 (Win10 22H2)
10.0.10586 (Win10 1511)

Without a manifest, GetVersionEx returns:
6.2.9200 (Win8)

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: GetVersionEx() depreciated, what should be used instead for Windows 7/8/10?

[Christian Franke via Cygwin]

Cedric Blancher via Cygwin wrote:

How does cygwin uname -s work, i.e. how does it get the OS version,
revision and build number ('10.0-19045')?
I tried to replicate it via GetVersionEx(), but that is deprecated.



AFAICS, MS does not offer an official way to retrieve the actual Windows 
version via the Win32 API layer. GetVersionEx() is unable to detect 
Windows versions which are more recent than the compatibility level 
advertised in the application manifest.


Cygwin uses the undocumented RtlGetNtVersionNumbers() from ntdll.dll 
because even RtlGetVersion() may not return the correct values. See 
function wincapc::init()

https://cygwin.com/git/?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/wincap.cc

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

New: mandoc 1.14.6-1

[Christian Franke]

The following package has been uploaded to the Cygwin distribution:

* mandoc-1.14.6-1

mandoc is a suite of tools compiling mdoc, the roff macro language of
choice for BSD manual pages, and man, the predominant historical language
for UNIX manuals.  It is small, self-contained, and quite fast.  The main
component of the toolset is the mandoc utility program, based on the
libmandoc validating compiler, to format output for UTF-8 and ASCII
terminals, HTML 5, PostScript, and PDF.

https://mandoc.bsd.lv/

The source package supports reproducible builds.

Regards,
Christian Franke

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
<https://cygwin.com/mailman/options/cygwin-announce>, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
<https://sourceware.org/lists.html#unsubscribe>.

[ITP] mandoc 1.14.6-1

[Christian Franke via Cygwin-apps]

I would like to contribute mandoc. Also present in Debian, Fedora, 
Ubuntu, ... and as the default man page formatter on *BSD.


Useful to check man pages for compatibility with *BSD systems.

The build is reproducible without the need to export SOURCE_DATE_EPOCH.

SUMMARY="BSD mandoc compiler toolset"

DESCRIPTION="\
mandoc is a suite of tools compiling mdoc, the roff macro language of
choice for BSD manual pages, and man, the predominant historical language
for UNIX manuals.  It is small, self-contained, and quite fast.  The main
component of the toolset is the mandoc utility program, based on the
libmandoc validating compiler, to format output for UTF-8 and ASCII
terminals, HTML 5, PostScript, and PDF."

mandoc-1.14.6-1.tar.xz:
usr/bin/demandoc.exe
usr/bin/mandoc.exe
usr/bin/mapropos -> mandoc
usr/bin/mman -> mandoc
usr/bin/msoelim.exe
usr/bin/mwhatis -> mandoc
usr/sbin/mandocdb -> ../bin/mandoc
usr/share/doc/mandoc/*
usr/share/man/man1/demandoc.1.gz
usr/share/man/man1/mandoc.1.gz
usr/share/man/man1/mapropos.1.gz
usr/share/man/man1/mman.1.gz
usr/share/man/man1/msoelim.1.gz
usr/share/man/man1/mwhatis.1.gz
usr/share/man/man5/mandoc.conf.5.gz
usr/share/man/man5/mandoc.db.5.gz
usr/share/man/man7/mandoc_char.7.gz
usr/share/man/man7/mandoc_eqn.7.gz
usr/share/man/man7/mandoc_man.7.gz
usr/share/man/man7/mandoc_mdoc.7.gz
usr/share/man/man7/mandoc_roff.7.gz
usr/share/man/man7/mandoc_tbl.7.gz
usr/share/man/man8/mandocdb.8.gz

--
Regards,
Christian

# cygport script for mandoc
NAME=mandoc
VERSION=1.14.6
RELEASE=1
SOURCE_DATE="2024-03-11 18:00:00 UTC"

SUMMARY="BSD mandoc compiler toolset"

DESCRIPTION="\
mandoc is a suite of tools compiling mdoc, the roff macro language of
choice for BSD manual pages, and man, the predominant historical language
for UNIX manuals.  It is small, self-contained, and quite fast.  The main
component of the toolset is the mandoc utility program, based on the
libmandoc validating compiler, to format output for UTF-8 and ASCII
terminals, HTML 5, PostScript, and PDF."

LICENSE="ISC"
CATEGORY="Text"
REQUIRES="" # zlib0
BUILD_REQUIRES="binutils gcc-core perl_base" # make

HOMEPAGE="https://mandoc.bsd.lv/;
SRC_URI="https://mandoc.bsd.lv/snapshots/${P}.tar.gz;

SOURCE_DATE_EPOCH=$(date -d "${SOURCE_DATE}" +%s) # 'export' is not needed

src_compile() {
cd ${B}
lndirs

cat <<-EOF > configure.local
OSNAME="Cygwin"

PREFIX="/usr"
MANDIR="/usr/share/man"
MANPATH_BASE="/usr/share/man"
MANPATH_DEFAULT="/usr/share/man:/usr/local/share/man"

HAVE_WCHAR=1
CFLAGS="${CFLAGS}"
LDFLAGS="${LDFLAGS}"
LN="ln -sf"

BINM_APROPOS="mapropos" # "apropos"
BINM_MAKEWHATIS="mandocdb"  # "makewhatis"
BINM_MAN="mman" # "man"
BINM_SOELIM="msoelim"   # "soelim"
BINM_WHATIS="mwhatis"   # "whatis"
MANM_EQN="mandoc_eqn"   # "eqn"
MANM_MANCONF="mandoc.conf"  # "man.conf"
MANM_MAN="mandoc_man"   # "man"
MANM_MDOC="mandoc_mdoc" # "mdoc"
MANM_ROFF="mandoc_roff" # "roff"
MANM_TBL="mandoc_tbl"   # "tbl"
EOF

# No cygconf because ./configure is not generated
./configure

cygmake
}

src_test() {
cd ${B}/regress
./regress.pl . ascii tag man html markdown lint

# Unicode chars >= U+1 do not work
# U+1D6C1 (Mathematical Bold Nabla) is output as U+D6C1 (Hangul 
Syllable Hyot)
./regress.pl . utf8 ||
inform "The above failure of the 'nabla' testcase could be 
safely ignored"
}

[PATCH cygport] Add repro-finish command

[Christian Franke via Cygwin-apps]

Thanks for accepting the repro-check patch. A minor enhancement is attached.

The function is in pkg_pkg.cygpart instead of pkg_cleanup.cygpart 
because then it is easier to keep it in sync with the other __repro_* 
functions.


PS: I have a local script which checks SPDX Identifiers and expressions. 
Any interest to add this to cygport and then check LICENSE settings?


--
Regards,
Christian

From b08796262308cf1b3a2c063349d024a5ccfd2455 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 11 Mar 2024 12:12:32 +0100
Subject: [PATCH] Add repro-finish command

This command removes the temporary directory used by repro-check.
---
 README  |  7 ---
 bin/cygport.in  |  4 
 lib/help.cygpart|  1 +
 lib/pkg_pkg.cygpart | 24 +++-
 4 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/README b/README
index 3c9e4d4a..a0897a4f 100644
--- a/README
+++ b/README
@@ -163,9 +163,10 @@ Other COMMANDs are meant primarily for maintainers:
 diff - write a patch file capturing changes to source in the working 
directory
 stage- as upload, but don't request processing of uploaded packages
 announce - compose and send a package announcement
-repro-build - rebuild from created source package to temp directory
-repro-diff  - check whether packages from original and rebuild differ
-repro-check - run repro-build and repro-diff
+repro-build  - rebuild from created source package to temp directory
+repro-diff   - check whether packages from original and rebuild differ
+repro-check  - run repro-build and repro-diff
+repro-finish - delete the temp directory used for rebuild
 
 The standard arguments --help or --version may also be passed to cygport.
 
diff --git a/bin/cygport.in b/bin/cygport.in
index df38a8b5..15bd559e 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -801,6 +801,10 @@ do
__pkg_repro_diff
_status=$?
;;
+   repro-finish)
+   __pkg_repro_finish
+   _status=$?
+   ;;
help)
__show_help;
exit 0;
diff --git a/lib/help.cygpart b/lib/help.cygpart
index d28fd7bb..ff03fb5f 100644
--- a/lib/help.cygpart
+++ b/lib/help.cygpart
@@ -59,6 +59,7 @@ __show_help() {
  repro-build   rebuild from created source package to temp 
directory
  repro-diffcheck whether packages from original and 
rebuild differ
  repro-check   run repro-build and repro-diff
+ repro-finish  delete the temp directory used for rebuild
 
See the included README file for further documentation.
 
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 25b80906..07313c66 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -1048,6 +1048,28 @@ __pkg_repro_diff() {
inform "Rebuild produced identical packages"
 }
 
+__pkg_repro_finish() {
+   local t_spkgdir=${T}/${spkgdir##*/}
+
+   cd ${top}
+
+   __step "Removing rebuild directory in 2 seconds..."
+   inform "Rebuild dir: ${t_spkgdir}"
+   if ! [ -d ${t_spkgdir} ]
+   then
+   inform "Rebuild directory does not exist"
+   return 0
+   fi
+
+   sleep 2 || exit $?
+
+   __step "Removing rebuild directory NOW."
+   rm -rf ${t_spkgdir}
+
+   __step "Finished."
+}
+
 # protect functions
 readonly -f __pkg_binpkg __pkg_diff __gpg_sign __pkg_srcpkg __pkg_dist \
-__pkg_repro_build __pkg_repro_diff __squeeze_whitespace __tar
+__pkg_repro_build __pkg_repro_diff __pkg_repro_finish \
+__squeeze_whitespace __tar
-- 
2.43.0

Re: [PATCH cygport] Add customization support for announce command

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 23/02/2024 11:23, Christian Franke via Cygwin-apps wrote:

Christian Franke wrote:
The email generated by the cygport announce command is useful, but 
actual use cases are somewhat limited due to the hard-coded email 
submission.


The attached patch adds more flexibility. The patch is on top of the 
"Use correct wording if only one package is announced" patch.


Slightly changed patch attached. Also adjusted to new version of "Use 
correct wording if only one package is announced" patch.




[...]

Thanks for this.


Possible (better?) alternative names for the new settings:
ANNOUNCEMENT_EDITOR
ANNOUNCEMENT_MAILER


Hmmm... I think "ANNOUNCE_EDITOR" and "ANNOUNCE_MAILER" would be
the best for clarity and conciseness.


New patch attached. Is still on top of "Use correct wording ..." patch.

I also added HOMEPAGE to the propagated variables as this should be 
included in an announcement.







-From: ${SMTP_SENDER}
-To: cygwin-annou...@cygwin.com
+${SMTP_SENDER:+From: ${SMTP_SENDER}
+}To: cygwin-annou...@cygwin.com
 Date: $(date -R --date=${msgat})
-Message-Id: <$(date "+%Y%m%d%H%M%S.$$" --date=${msgat})-1-$(echo 
${SMTP_SENDER} | sed 's|.*<\(.*\)>.*|\1|')>
+Message-Id: <$(date "+%Y%m%d%H%M%S.$$" --date=${msgat})-1-$(echo 
${SMTP_SENDER:-cygport} | sed 's|.*<\(.*\)>.*|\1|')>

 Subject: ${NAME} ${PVR}


Can you also explain what this is doing in the commit message, since 
it's not immediately apparent.





If the mail infrastructure always replaces the "From:" line or the 
default one is sufficient, then there is no need to generate one. 
SMTP_SENDER could be left alone then. I added a related comment to 
cygport.conf


From 335cbde3c6c2450051cc739cee60a555b236843e Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Sun, 10 Mar 2024 17:28:09 +0100
Subject: [PATCH] Add customization support for announce command

Two new configuration settings allow to override the launch of
a text editor (ANNOUNCE_EDITOR) and the builtin email submission
(ANNOUNCE_MAILER).  Don't create a "From:" header line if
SMTP_SENDER is undefined or empty.
---
 data/cygport.conf  | 27 +++-
 lib/pkg_upload.cygpart | 57 +-
 2 files changed, 77 insertions(+), 7 deletions(-)

diff --git a/data/cygport.conf b/data/cygport.conf
index 34ccd291..3da744d9 100644
--- a/data/cygport.conf
+++ b/data/cygport.conf
@@ -101,10 +101,35 @@
 #PAGER=
 
 
+#v* Configuration/ANNOUNCE_EDITOR
+#  DESCRIPTION
+#  Shell command string to process the email message created by cygport's
+#  announce command before sending the email.  If undefined, a text editor
+#  will be run, see EDITOR setting above.  If empty, nothing will be run.
+#  If not empty, '/bin/bash' will be launched with the command string passed
+#  with '-c' option and the path of the temporary email message file as '$1'.
+#  The working directory of the shell will be the directory of the cygport
+#  file.  The specified command string will be prepended by shell assignments
+#  of the cygport variables HOMEPAGE, P, PF, PN, PR and PV and all SMTP_*
+#  settings described below.
+#ANNOUNCE_EDITOR=
+
+#v* Configuration/ANNOUNCE_MAILER
+#  DESCRIPTION
+#  Shell command string to process the email message created by cygport's
+#  announce command after editing.  If undefined, the email will be sent
+#  using the builtin perl-based SMTP support.  If empty, nothing will be run.
+#  If not empty, the command string will be handled similar to ANNOUNCE_EDITOR
+#  described above.
+#ANNOUNCE_MAILER=
+
+
 #v* Configuration/SMTP_SENDER
 #  DESCRIPTION
 #  Name and email address, in the form of "First Last " to be used
-#  by cygport's announcement command.
+#  by cygport's announcement command.  If undefined or empty, no "From:" email
+#  header line will be generated.  The local mail tool or the mail provider may
+#  unconditionally replace this header line or only the "" part.
 #  NOTE
 #  Many webmail services do not allow using arbitrary sender address in SMTP
 #  mail, or may first require registering other email addresses as authorized
diff --git a/lib/pkg_upload.cygpart b/lib/pkg_upload.cygpart
index 37bc2d63..9ced1fb5 100644
--- a/lib/pkg_upload.cygpart
+++ b/lib/pkg_upload.cygpart
@@ -168,6 +168,34 @@ EOF
echo "Upload complete."
 }
 
+__pkg_announce_run_cmd_on_msg() {
+   local cmdvar=$1
+   local msg=$2
+   local cmd
+
+   eval cmd="\${${cmdvar}}"
+
+   if [ "${cmd:+y}" != "y" ]
+   then
+   inform "\${${cmdvar}} is empty"
+   return 0
+   fi
+   echo
+   inform "Launching '\${${cmdvar}} ${msg}'"
+
+   /bin/bash -c "cd ${top} || exit 1
+${HOMEPAGE+HOMEPAGE=${HOMEPAGE@Q}}
+P=${P@Q}; PF=${PF@Q}; PN=${PN@Q}; PR=

Re: [PATCH cygport] dodoc: Skip a file if a compressed version already exists

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 01/03/2024 13:13, Christian Franke via Cygwin-apps wrote:
It IMO makes sense to compress large and rarely viewed doc files like 
change logs. This seems to be common practice on Debian etc.


With current cygport, the following results in ChangeLog and 
ChangeLog.gz in the docdir:


src_install()
{
   ...
   dodoc ChangeLog
   gzip -9 -n "${D}/usr/share/doc/${PN}/ChangeLog"
}


Uh, I don't quite see how this patch will change the behavior of this 
fragment.




Yes, it actually doesn't change the behavior of this fragment itself.


Even more confusing, why isn't this already doing what you want? 
Unless you specify -k/--keep to gzip, the input file is removed, right?


Yes - but after this src_install() the file will be re-added by 
__predoc() unless _CYGPORT_RESTRICT_postinst_doc_ is set. The patch 
avoids this because __predoc() also uses dodoc().

Re: [PATCH cygport] Add more checks of SOURCE_DATE_EPOCH

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 26/02/2024 19:53, Christian Franke via Cygwin-apps wrote:



Would it not make more sense to just re-export it if set?


If the cygport file decides to set but not export it, there is 
possibly no need to do it. An example is smartmontools.cygport which 
passes the unexported variable as a parameter to configure.


Ok, but exporting it is harmless there, right?


I'm not aware of any corner cases where exporting would break something. 
But leaving the decision to the user would allow to handle such cases. 
It would also allow to check whether it makes a difference and if yes, 
which files are affected.






(so that commands like "SOURCE_DATE_EPOCH=something cygport foo" 
work as expected?)




Would make no difference as the 'VAR=val CMD...' syntax already 
exports the variable to the CMD:


$ unset FOO; FOO=bar sh -c 'sh -c "sh -c printenv\ FOO"'
bar


Ah, right.

So you seem to be saying that the only situation where it's set but 
not exported is where it's set in the cygport.


So we're just making people (need to remember to) explicitly write 
"export SOURCE_DATE_EPOCH" in their cygport where needed?


Exactly.

New: bmake 20240301-1

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* bmake-20240301-1
* bmake-extras-20240301-1

bmake is a portable version of the NetBSD make(1) utility. It is similar
to GNU make, even though the syntax for the advanced features supported
in Makefiles is very different.

The bmake-extras package contains many additional *.mk files and some
support scripts which are only required if used in Makefiles.

https://www.crufty.net/help/sjg/bmake.htm

The source package supports reproducible builds.

Regards,
ChristianFranke

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
.

[ITP] bmake 20240301-1

[Christian Franke via Cygwin-apps]

I would like to contribute bmake. Also present in Debian, Fedora, 
FreeBSD, Ubuntu, ...


I occasionally use it to check whether Makefiles are compatible with 
non-GNU versions of make.


SUMMARY="Portable version of the NetBSD 'make' utility"

DESCRIPTION="\
bmake is a portable version of the NetBSD make(1) utility.  It is similar
to GNU make, even though the syntax for the advanced features supported
in Makefiles is very different."

bmake_extras_SUMMARY="${SUMMARY} (additional support files)"

bmake_extras_DESCRIPTION="${DESCRIPTION}
This package contains many additional *.mk files and some support scripts
which are only required if used in Makefiles."

bmake-20240301-1.tar.xz:
usr/bin/bmake.exe
usr/share/doc/bmake/*
usr/share/man/man1/bmake.1.gz
usr/share/bmake/mk/host-target.mk
usr/share/bmake/mk/suffixes.mk
usr/share/bmake/mk/sys.mk
usr/share/bmake/mk/sys.*.mk
usr/share/bmake/mk/sys/Generic.mk

bmake-extras-20240301-1.tar.xz:
usr/share/bmake/mk/*.mk
usr/share/bmake/mk/install-sh
usr/share/bmake/mk/meta2deps.py
usr/share/bmake/mk/meta2deps.sh
usr/share/bmake/mk/mkopt.sh
usr/share/bmake/mk/stage-install.sh
usr/share/bmake/mk/sys/*.mk

--
Regards,
Christian

# cygport script for bmake
NAME=bmake
VERSION=20240301
RELEASE=1
SOURCE_DATE="2024-03-09 12:00:00 UTC"

SUMMARY="Portable version of the NetBSD 'make' utility"

DESCRIPTION="\
bmake is a portable version of the NetBSD make(1) utility.  It is similar
to GNU make, even though the syntax for the advanced features supported
in Makefiles is very different."

LICENSE="BSD-3-Clause"
CATEGORY="Devel"

PKG_NAMES="bmake bmake-extras"

bmake_extras_SUMMARY="${SUMMARY} (additional support files)"
bmake_extras_DESCRIPTION="${DESCRIPTION}  
This package contains many additional *.mk files and some support scripts
which are only required if used in Makefiles."

REQUIRES=""
bmake_extras_REQUIRES="bmake" # bash python3
BUILD_REQUIRES="binutils gcc-core" # "make" not required

HOMEPAGE="https://www.crufty.net/help/sjg/bmake.htm;
SRC_URI="https://www.crufty.net/ftp/pub/sjg/bmake-${PV}.tar.gz
 https://www.crufty.net/ftp/pub/sjg/bmake-${PV}.tar.gz.asc;
SRC_DIR="bmake"

bmake_CONTENTS="
usr/bin
usr/share/doc
usr/share/man
usr/share/${PN}/mk/host-target.mk
usr/share/${PN}/mk/suffixes.mk
usr/share/${PN}/mk/sys.mk
usr/share/${PN}/mk/sys.*.mk
usr/share/${PN}/mk/sys/Generic.mk
"
bmake_extras_CONTENTS="
--exclude=usr/share/${PN}/mk/host-target.mk
--exclude=usr/share/${PN}/mk/suffixes.mk
--exclude=usr/share/${PN}/mk/sys.mk
--exclude=usr/share/${PN}/mk/sys.*.mk
--exclude=usr/share/${PN}/mk/sys/Generic.mk
usr/share/${PN}
"

SOURCE_DATE_EPOCH=$(date -d "$SOURCE_DATE" +%s) # 'export' not needed

src_compile() {
cd ${B}
${S}/boot-strap --prefix=/usr \
--with-default-sys-path=/usr/share/${PN}/mk \
--skip-test op=build
}

src_test() {
cd ${B}
${S}/boot-strap op=test

if false; then # BROKEN_TESTS (all harmless)
TESTS='export opt-chdir opt-keep-going-indirect' \
${S}/boot-strap op=test
fi
}

src_install() {
cd ${B}
# Don't use 'install -s' for bmake.exe.
STRIP_FLAG="" \
${S}/boot-strap --prefix=/usr \
--install-destdir=${D} \
--skip-test op=install
}
--- origsrc/bmake/Makefile  2024-03-09 12:00:00.0 +
+++ src/bmake/Makefile  2024-03-09 12:00:00.0 +
@@ -90,7 +90,7 @@ OS := ${.MAKE.OS:U${uname -s:L:sh}}
 # are we 4.4BSD ?
 isBSD44:=${BSD44_LIST:M${OS}}
 
-.if ${isBSD44} == "" && ${OS:NDarwin:NLinux} != ""
+.if ${isBSD44} == "" && ${OS:NCYGWIN*:NDarwin:NLinux} != ""
 MANTARGET= cat
 INSTALL?=${srcdir}/install-sh
 .if ${MACHINE} == "sun386"
@@ -192,7 +192,7 @@ CONFIGURE_ARGS += --without-makefile
 AUTOCONF_GENERATED_MAKEFILE = Makefile.config
 .include 
 .endif
-SHARE_MK ?= ${SHAREDIR}/mk
+SHARE_MK ?= ${DEFAULT_SYS_PATH}
 MKSRC = ${srcdir}/mk
 INSTALL ?= ${srcdir}/install-sh
 
--- origsrc/bmake/boot-strap2024-03-09 12:00:00.0 +
+++ src/bmake/boot-strap2024-03-09 12:00:00.0 +
@@ -69,6 +69,9 @@
 #  This is useful when $prefix/ is shared by multiple
 #  machines.
 #
+#  --skip-test
+#  Don't run test after build or before install.
+#
 #  Flags relevant when installing:
 #
 #  -DWITHOUT_INSTALL_MK
@@ -212,6 +215,7 @@ srcdir=$Mydir
 mksrc=$Mydir/mk
 objdir=
 quiet=:
+skip_test=false
 
 ${SKIP_RC:+:} source_rc .bmake-boot-strap.rc . "$Mydir/.." "$HOME"
 
@@ -249,6 +253,7 @@ do
 INSTALL_BIN=$HOST_TARGET/bin;;
--install-destdir=*) INSTALL_DESTDIR=`get_optarg "$1"`;;
--install-prefix=*) INSTALL_PREFIX=`get_optarg "$1"`;;
+   --skip-test) skip_test=true;;
-DWITH*) INSTALL_ARGS="$INSTALL_ARGS $1";;
-s|--src) 

Re: [ITP] afflib 3.7.20-1

[Christian Franke via Cygwin-apps]

marco atzeri wrote:

On Wed, Mar 6, 2024 at 11:26 PM Christian Franke via Cygwin-apps  wrote:

Jon Turney wrote:

On 06/03/2024 15:39, Christian Franke via Cygwin-apps wrote:

Jon Turney wrote:

Thanks!


libafflib_CONTENTS="
 usr/bin/cygafflib-*.dll

Any reason why this package doesn't include the soversion, i.e. why
not libafflib0?


Libtsk and libafflib are my first library packages, so I'm not sure
what the policy is. My recent package libtsk has been accepted
without soversion, so I omitted it also here. I assumed that the
soversion will

I'm going to suggest that was an oversight in the review.

Should I also rename libtsk to libtsk19 in the planned sleutkit-*-2
package which will add afflib support ?

yes please


The original package is only a few days old and has possibly only a
small but experienced audience, so I expect not much worries if the
change will be explained in the announcement.

not worries at all if you use

libtsk19_OBSOLETES=libtsk

see
https://cygwin.github.io/cygport/pkg_pkg_cygpart.html#PKG_OBSOLETES


Thanks for the info. I will do this that way when libafflib0 package is 
available.


--
Regards,
Christian

Re: [ITP] afflib 3.7.20-1

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 06/03/2024 15:39, Christian Franke via Cygwin-apps wrote:

Jon Turney wrote:


Thanks!



libafflib_CONTENTS="
usr/bin/cygafflib-*.dll


Any reason why this package doesn't include the soversion, i.e. why 
not libafflib0?




Libtsk and libafflib are my first library packages, so I'm not sure 
what the policy is. My recent package libtsk has been accepted 
without soversion, so I omitted it also here. I assumed that the 
soversion will 


I'm going to suggest that was an oversight in the review.


Should I also rename libtsk to libtsk19 in the planned sleutkit-*-2 
package which will add afflib support ?
The original package is only a few days old and has possibly only a 
small but experienced audience, so I expect not much worries if the 
change will be explained in the announcement.





be added only when needed for new not backward compatible releases. 
The upstream afflib project is mostly idling, so I don't expect any 
new major lib versions in the near future.


If course, I could rename it to libafflib0 if desired.


As far as I know, there is no cost for doing this, and it saves grief 
if upstream ever bumps the soversion.


Also, it's probably best to explicitly list the filename with 
soversion in the CONTENTS, so that if upstream ever does change the 
soversion, it will be detected as a packaging failure, rather than 
producing a package with a mismatch between the soversion in it's name 
and in it's contents.


Good point, new cygport file is attached.




(Cygport should perhaps and detect and warn about apparently 
soversioned libraries that aren't in appropriately named packages, 
but...)




... this is possibly a good idea.

# cygport script for afflib
NAME=afflib
VERSION=3.7.20
RELEASE=1
SOURCE_DATE="2024-03-06 22:00:00 UTC"

SUMMARY="Library and tools for the Advanced Forensic Format"

DESCRIPTION="\
The Advanced Forensic Format (AFF) is a file format for storing computer
forensic information.  It supports metadata, compression, encryption and
signing."

LICENSE="BSD-4-clause"

PKG_NAMES="afflib-tools libafflib0 libafflib-devel"

afflib_tools_SUMMARY="${SUMMARY} (tools)"
afflib_tools_DESCRIPTION="${DESCRIPTION}  
This package contains tools to examine, convert, compare, copy, encrypt,
decrypt and sign AFF files."

libafflib0_SUMMARY="${SUMMARY} (runtime)"
libafflib0_DESCRIPTION="${DESCRIPTION}  
This package contains the runtime library for afflib."

libafflib_devel_SUMMARY="${SUMMARY} (development)"
libafflib_devel_DESCRIPTION="${DESCRIPTION}  
This package contains the development files for libafflib0."

CATEGORY="Devel Libs Utils"
afflib_tools_CATEGORY="Utils"
libafflib0_CATEGORY="Libs"
libafflib_devel_CATEGORY="Devel Libs"

afflib_tools_REQUIRES="" # libafflib0 libexpat1 libgcc1 libreadline7 libssl3 
libstdc++6
libafflib0_REQUIRES="" # libcurl4 libexpat1 libgcc1 libssl3 libstdc++6 zlib0
libafflib_devel_REQUIRES="" # libafflib0 libssl-devel pkg-config

BUILD_REQUIRES="
  binutils gcc-g++ gzip libcurl-devel libexpat-devel libreadline-devel
  libssl-devel zlib-devel
" # make

HOMEPAGE="https://github.com/sshock/AFFLIBv3;
SRC_URI="https://codeload.github.com/sshock/AFFLIBv3/tar.gz/refs/tags/v${PV}#/${P}.tar.gz;
SRC_DIR="AFFLIBv3-${PV}"

afflib_tools_CONTENTS="
--exclude=usr/bin/cygafflib-0.dll
usr/bin
usr/share
"
libafflib0_CONTENTS="
usr/bin/cygafflib-0.dll
"
libafflib_devel_CONTENTS="
usr/include/afflib
usr/lib
"

DIFF_EXCLUDES="lzma443"

export SOURCE_DATE_EPOCH=$(date -d "$SOURCE_DATE" +%s)

src_compile() {
cd ${S}
cygautoreconf

cd ${B}
cygconf --enable-shared --disable-static \
--enable-qemu --enable-s3 --enable-threading \
--with-curl --with-expat --with-gnu-ld \
--disable-fuse --disable-python

# configure sets _FORTIFY_SOURCE=2 which would override level 3
# libtool requires '-no-undefined'
cygmake CFLAGS="${CFLAGS}" CXXFLAGS="${CXXFLAGS}" \
LDFLAGS="${LDFLAGS}${LDFLAGS:+ }-no-undefined"
}

src_install() {
cd ${B}
cyginstall

cd ${S}
dodoc doc/*.txt

cd ${D}
# affuse only prints that FUSE is not supported
rm -v usr/bin/affuse.exe usr/share/man/man1/affuse.1

gzip -9nv usr/share/doc/${PN}/*.txt
}

Re: [ITP] afflib 3.7.20-1

[Christian Franke via Cygwin-apps]

Jon Turney wrote:


Thanks!



libafflib_CONTENTS="
usr/bin/cygafflib-*.dll


Any reason why this package doesn't include the soversion, i.e. why 
not libafflib0?




Libtsk and libafflib are my first library packages, so I'm not sure what 
the policy is. My recent package libtsk has been accepted without 
soversion, so I omitted it also here. I assumed that the soversion will 
be added only when needed for new not backward compatible releases. The 
upstream afflib project is mostly idling, so I don't expect any new 
major lib versions in the near future.


If course, I could rename it to libafflib0 if desired.


rm -v usr/bin/affuse.exe usr/share/man/man1/affuse.1 # 
--disable-fuse


I guess this comment means something to someone.  But it doesn't tell 
me anything...




:-)

Long form: If (Linux-)FUSE is unavailable, the affuse tool only prints 
that this is the case and exits then. There is no need to distribute it.

[ITP] afflib 3.7.20-1

[Christian Franke via Cygwin-apps]

I would like to contribute afflib. Also present in Debian, Fedora, 
Ubuntu, ... but package naming differs:


Debian/Ubuntu: afflib-tools, libafflib0v5, libafflib-dev
Fedora: afftools, afflib, afflib-devel

It is indented to enable afflib format support for the next version of 
the sleuthkit package.


SUMMARY="Library and tools for the Advanced Forensic Format"

DESCRIPTION="\
The Advanced Forensic Format (AFF) is a file format for storing computer
forensic information.  It supports metadata, compression, encryption and
signing."

afflib_tools_DESCRIPTION="...
This package contains tools to examine, convert, compare, copy, encrypt,
decrypt and sign AFF files."

afflib-tools-3.7.20-1.tar.xz:
usr/bin/affcat.exe
usr/bin/affcompare.exe
usr/bin/affconvert.exe
usr/bin/affcopy.exe
usr/bin/affcrypto.exe
usr/bin/affdiskprint.exe
usr/bin/affinfo.exe
usr/bin/affix.exe
usr/bin/affrecover.exe
usr/bin/affsegment.exe
usr/bin/affsign.exe
usr/bin/affstats.exe
usr/bin/affverify.exe
usr/bin/affxml.exe
usr/share/doc/afflib/*
usr/share/man/man1/aff*.1.gz

libafflib-3.7.20-1.tar.xz:
usr/bin/cygafflib-0.dll

libafflib-devel-3.7.20-1.tar.xz:
usr/include/afflib/*.h
usr/lib/libafflib.dll.a
usr/lib/pkgconfig/afflib.pc

--
Regards,
Christian

# cygport script for afflib
NAME=afflib
VERSION=3.7.20
RELEASE=1
SOURCE_DATE="2024-03-06 13:00:00 UTC"

SUMMARY="Library and tools for the Advanced Forensic Format"

DESCRIPTION="\
The Advanced Forensic Format (AFF) is a file format for storing computer
forensic information.  It supports metadata, compression, encryption and
signing."

LICENSE="BSD-4-clause"

PKG_NAMES="afflib-tools libafflib libafflib-devel"

afflib_tools_SUMMARY="${SUMMARY} (tools)"
afflib_tools_DESCRIPTION="${DESCRIPTION}  
This package contains tools to examine, convert, compare, copy, encrypt,
decrypt and sign AFF files."

libafflib_SUMMARY="${SUMMARY} (runtime)"
libafflib_DESCRIPTION="${DESCRIPTION}  
This package contains the runtime library for afflib."

libafflib_devel_SUMMARY="${SUMMARY} (development)"
libafflib_devel_DESCRIPTION="${DESCRIPTION}  
This package contains the development files for libafflib."

CATEGORY="Devel Libs Utils"
afflib_tools_CATEGORY="Utils"
libafflib_CATEGORY="Libs"
libafflib_devel_CATEGORY="Devel Libs"

afflib_tools_REQUIRES="" # libafflib libexpat1 libgcc1 libreadline7 libssl3 
libstdc++6
libafflib_REQUIRES="" # libcurl4 libexpat1 libgcc1 libssl3 libstdc++6 zlib0
libafflib_devel_REQUIRES="" # libafflib libssl-devel pkg-config

BUILD_REQUIRES="
  binutils gcc-g++ gzip libcurl-devel libexpat-devel libreadline-devel
  libssl-devel zlib-devel
" # make

HOMEPAGE="https://github.com/sshock/AFFLIBv3;
SRC_URI="https://codeload.github.com/sshock/AFFLIBv3/tar.gz/refs/tags/v${PV}#/${P}.tar.gz;
SRC_DIR="AFFLIBv3-${PV}"

afflib_tools_CONTENTS="
--exclude=usr/bin/cygafflib-*.dll
usr/bin
usr/share
"
libafflib_CONTENTS="
usr/bin/cygafflib-*.dll
"
libafflib_devel_CONTENTS="
usr/include/afflib
usr/lib
"

DIFF_EXCLUDES="lzma443"

export SOURCE_DATE_EPOCH=$(date -d "$SOURCE_DATE" +%s)

src_compile() {
cd ${S}
cygautoreconf

cd ${B}
cygconf --enable-shared --disable-static \
--enable-s3 --enable-threading --enable-qemu \
--with-curl --with-gnu-ld --with-expat \
--disable-fuse --disable-python

# configure sets _FORTIFY_SOURCE=2 which would override level 3
# libtool requires '-no-undefined'
cygmake CFLAGS="${CFLAGS}" CXXFLAGS="${CXXFLAGS}" \
LDFLAGS="${LDFLAGS}${LDFLAGS:+ }-no-undefined"
}

src_install() {
cd ${B}
cyginstall

cd ${S}
dodoc doc/*.txt

cd ${D}
rm -v usr/bin/affuse.exe usr/share/man/man1/affuse.1 # --disable-fuse
gzip -9nv usr/share/doc/${PN}/*.txt
}
--- origsrc/AFFLIBv3-3.7.20/lib/Makefile.am 2024-03-06 13:00:00.0 
+
+++ src/AFFLIBv3-3.7.20/lib/Makefile.am 2024-03-06 13:00:00.0 +
@@ -76,7 +76,7 @@ install-exec-hook:
  then echo $(libdir) already installed ; \
  else echo installing $(libdir) in $(DESTDIR)/etc/ld.so.conf ; \
   echo $(libdir) >> $(DESTDIR)/etc/ld.so.conf ; \
- PATH=$(PATH):/sbin; \
+ PATH="$(PATH):/sbin"; \
  ldconfig; \
   fi ; \
  echo "*" ;\
--- origsrc/AFFLIBv3-3.7.20/tools/affcopy.cpp   2024-03-06 13:00:00.0 
+
+++ src/AFFLIBv3-3.7.20/tools/affcopy.cpp   2024-03-06 13:00:00.0 
+
@@ -146,7 +146,7 @@ void unlink_outfiles(vector outf
 if(failure) exit(1);
 }
 
-#if !defined( __BSD_VISIBLE) && !defined(isnumber)
+#if (!defined(__BSD_VISIBLE) || defined(__CYGWIN__)) && !defined(isnumber)
 #define isnumber(x) isdigit(x)
 #endif
 

New: sleuthkit-4.12.1-1

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* sleuthkit-4.12.1-1
* libtsk-4.12.1-1
* libtsk-devel-4.12.1-1

The Sleuth Kit (TSK) is a collection of command line tools for disk
images.  It allows to analyze volume and filesystem data, examine disk
layout,recover deleted files, etc.  Many volume and filesystem formats
are supported.

https://www.sleuthkit.org/sleuthkit/

The source package supports reproducible builds except for libtsk.dll.a
from the libtsk-devel package.

Regards,
Christian Franke

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
<https://cygwin.com/mailman/options/cygwin-announce>, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
<https://sourceware.org/lists.html#unsubscribe>.

Re: [ITP] sleuthkit 4.12.1

[Christian Franke via Cygwin-apps]

Hi Marco,

Marco Atzeri via Cygwin-apps wrote:

On 02/03/2024 13:05, Christian Franke via Cygwin-apps wrote:
I would like to contribute sleuthkit. Also present in Debian, Fedora, 
Ubuntu, ...


SUMMARY="Tools for analysis of volume and filesystem data"

DESCRIPTION="The Sleuth Kit (TSK) is a collection of command line tools
for disk images.  It allows to analyze volume and filesystem data,
examine disk layout, recover deleted files, etc.  Many partition and
filesystem formats are supported."

libtsk_SUMMARY="${SUMMARY} (runtime)"

libtsk_devel_SUMMARY="${SUMMARY} (development)"


I'm not sure about the LICENSE string:

LICENSE="CPL-1.0 AND GPL-2.0-or-later"

The license/README.md file mentions a bunch of licenses, see comment 
in cygport file. CPL-1.0 is the main license, one separate tool uses 
GPL-2.0-or-later.



The source package supports reproducible builds except for 
libtsk-devel (timestamps in *.a files).


Hi Christian,

usually we do no distribute static library


Didn't know, sorry. Makes plenty of sense, at least to prevent that 
other packages accidentally link to the static lib.





Any reason here ?


No, static lib removed.



except that GTG

$ git diff |grep "^+"
+++ b/cygwin-pkg-maint
+sleuthkit    Christian Franke



Thanks,
Christian

[ITP] sleuthkit 4.12.1

[Christian Franke via Cygwin-apps]

I would like to contribute sleuthkit. Also present in Debian, Fedora, 
Ubuntu, ...


SUMMARY="Tools for analysis of volume and filesystem data"

DESCRIPTION="The Sleuth Kit (TSK) is a collection of command line tools
for disk images.  It allows to analyze volume and filesystem data,
examine disk layout, recover deleted files, etc.  Many partition and
filesystem formats are supported."

libtsk_SUMMARY="${SUMMARY} (runtime)"

libtsk_devel_SUMMARY="${SUMMARY} (development)"


I'm not sure about the LICENSE string:

LICENSE="CPL-1.0 AND GPL-2.0-or-later"

The license/README.md file mentions a bunch of licenses, see comment in 
cygport file. CPL-1.0 is the main license, one separate tool uses 
GPL-2.0-or-later.



The source package supports reproducible builds except for libtsk-devel 
(timestamps in *.a files).


Abbreviated list of files:

sleuthkit-4.12.1-1.tar.xz:
usr/bin/blkcalc.exe
usr/bin/blkcat.exe
usr/bin/blkls.exe
usr/bin/blkstat.exe
usr/bin/fcat.exe
usr/bin/ffind.exe
usr/bin/fiwalk.exe
usr/bin/fls.exe
usr/bin/fsstat.exe
usr/bin/hfind.exe
usr/bin/icat.exe
usr/bin/ifind.exe
usr/bin/ils.exe
usr/bin/img_cat.exe
usr/bin/img_stat.exe
usr/bin/istat.exe
usr/bin/jcat.exe
usr/bin/jls.exe
usr/bin/jpeg_extract.exe
usr/bin/mactime
usr/bin/mmcat.exe
usr/bin/mmls.exe
usr/bin/mmstat.exe
usr/bin/pstat.exe
usr/bin/sigfind.exe
usr/bin/sorter
usr/bin/srch_strings.exe
usr/bin/tsk_comparedir.exe
usr/bin/tsk_gettimes.exe
usr/bin/tsk_imageinfo.exe
usr/bin/tsk_loaddb.exe
usr/bin/tsk_recover.exe
usr/bin/usnjls.exe
usr/share/doc/sleuthkit/*
usr/share/man/man1/*
usr/share/tsk/sorter/*.sort

libtsk-4.12.1-1.tar.xz:
usr/bin/cygtsk-19.dll

libtsk-devel-4.12.1-1.tar.xz:
usr/include/tsk/*
usr/lib/libtsk.a
usr/lib/libtsk.dll.a
usr/lib/pkgconfig/tsk.pc
usr/share/doc/sleuthkit/samples/*.cpp



Real world use case: Check whether the SSD TRIM command actually works:

$ # Create a test file (> ~3*256B to prevent resident file)
$ printf 'Line %s\n' {0001..0100} > trim_check

$ # Get full path of file
$ cygpath -am trim_check
D:/tmp/trim_check

$ # Find raw device of partition
$ grep D: /proc/partitions # or: ls -l /dev/disk/by-drive/d
    8    20 629145944 sdb4   D:\

$ # Find inode (here: $MFT index) of file
$ ifind -n /tmp/trim_check /dev/sdb4
339065

$ # Find cluster(s) used by inode
$ istat /dev/sdb4 339065
...
Name: TRIM_C~1
...
Name: trim_check
...
Type: $DATA (128-4)   Name: N/A   Non-Resident size: 1000  init_size: 1000
7876740

$ # Read cluster (assumes 4KiB cluster size, could be checked with fsstat)
$ dd if=/dev/sdb4 bs=4096 count=1 skip=7876740 iflag=direct status=none 
| cat -A

Line 0001$
Line 0002$
...

$ # Remove file, flush buffers and wait
$ rm trim_check; sync; sleep 10

$ # Re-read cluster
$ dd if=/dev/sdb4 bs=4096 count=1 skip=7876740 iflag=direct status=none | od
000 00 00 00 00 00 00 00 00
*
001

$ echo "TRIM works!"
TRIM works!


--
Regards,
Christian




# cygport script for sleuthkit
NAME=sleuthkit
VERSION=4.12.1
RELEASE=1
SOURCE_DATE="2024-03-02 11:00:00 UTC"

SUMMARY="Tools for analysis of volume and filesystem data"

DESCRIPTION="The Sleuth Kit (TSK) is a collection of command line tools
for disk images.  It allows to analyze volume and filesystem data,
examine disk layout, recover deleted files, etc.  Many partition and
filesystem formats are supported."

LICENSE="CPL-1.0 AND GPL-2.0-or-later"
# Licenses mentioned in licenses/README.md:
# Apache-2.0 # case-uco/*, win32/rejistry++/* (code not used)
# BSD-3-Clause   # samples/*, tsk/fs/lzvn.c
# CPL-1.0# The Sleuth Kit (TSK) license
# GPL-2.0-or-later   # tools/srchtools/srch_strings.c
# GPL-3.0-or-later   # m4/ax_pthread.m4 (... WITH Autoconf-exception-3.0)
# IPL-1.0# The Coroner's Toolkit (TCT) license
# ISC# tools/fiwalk/src/base64.*
# "public domain"# tools/fiwalk/*, tsk/base/sha1.c
# MIT# tsk/auto/guid.cpp
# RSA-MD # tsk/base/md5c.c
# "Unicode"  # tsk/base/tsk_unicode.*

CATEGORY="Utils"
PKG_NAMES="sleuthkit libtsk libtsk-devel"

REQUIRES="" # libgcc1 libstdc++6 libtsk perl-DateTime-TimeZone perl_base
libtsk_REQUIRES="" # libgcc1 libsqlite3_0 libstdc++6 zlib0
libtsk_devel_REQUIRES="" # libtsk pkg-config
BUILD_REQUIRES="
  binutils cygwin-devel gcc-g++ gzip libsqlite3-devel perl_base zlib-devel
" # make

HOMEPAGE="https://www.sleuthkit.org/sleuthkit/;
SRC_URI="https://github.com/sleuthkit/sleuthkit/releases/download/${P}/${P}.tar.gz;

libtsk_SUMMARY="${SUMMARY} (runtime)"
libtsk_DESCRIPTION="${DESCRIPTION}  
This package contains the runtime library for sleuthkit."

libtsk_devel_SUMMARY="${SUMMARY} (development)"
libtsk_devel_DESCRIPTION="${DESCRIPTION}  
This package contains the development files for libtsk."

sleuthkit_CONTENTS="
--exclude=usr/bin/cygtsk-*.dll
--exclude=usr/share/doc/${PN}/samples
usr/bin
usr/share
"
libtsk_CONTENTS="

Re: [PATCH cygport] Add repro-check command

[Christian Franke via Cygwin-apps]

Christian Franke wrote:
This could be used to check whether a package is possibly 
reproducible. Then it could make sense to add a reasonable 
SOURCE_DATE_EPOCH value to the cygport file.


Example:

$ export SOURCE_DATE_EPOCH=$(date +%s)

$ cygport project.cygport all repro-check
...
*** Info: Build reproducibility test succeeded

$ TZ=UTC cygport project.cygport repro-check
...
*** Info: Build reproducibility test succeeded

$ unset SOURCE_DATE_EPOCH

$ cygport project.cygport repro-check
...
*** ERROR: Build reproducibility test failed



An enhanced version of the patch is attached. The build and diff could 
now be run also individually and the diff report includes individual 
files from the packages.


As a side effect, this enables another use case: Check whether changes 
to cygport only change the expected files.


$ cygport project.cygport all repro-check
...
*** Info: Rebuild produced identical packages

$ editor project.cygport
... Change some comments ...

$ cygport project.cygport all repro-diff
...
Differing files found:
! dist/project/project-1.2-3-src.tar.xz
! spkg/project-1.2-3.src/project.cygport

*** ERROR: Rebuild differs from original

--
Regards,
Christian

From 152a21dfad4c786cff1712e6aa1c33f2db0b6a75 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 1 Mar 2024 19:46:50 +0100
Subject: [PATCH] Add repro-build, repro-diff und repro-check commands

These commands check for reproducibility of distribution packages.
The repro-build command unpacks the source package from the dist
directory to the temp directory and performs a nested rebuild of
the packages there.
The repro-diff command compares original and rebuild packages.
If different, a report about individual differing files in dist,
inst and spkg directories is printed.
The repro-check command combines both commands.
---
 README  |  3 +++
 bin/cygport.in  | 17 +
 lib/help.cygpart|  3 +++
 lib/pkg_pkg.cygpart | 58 -
 4 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/README b/README
index fd16df6b..3c9e4d4a 100644
--- a/README
+++ b/README
@@ -163,6 +163,9 @@ Other COMMANDs are meant primarily for maintainers:
 diff - write a patch file capturing changes to source in the working 
directory
 stage- as upload, but don't request processing of uploaded packages
 announce - compose and send a package announcement
+repro-build - rebuild from created source package to temp directory
+repro-diff  - check whether packages from original and rebuild differ
+repro-check - run repro-build and repro-diff
 
 The standard arguments --help or --version may also be passed to cygport.
 
diff --git a/bin/cygport.in b/bin/cygport.in
index 5fc89eaf..a2c2b5a3 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -29,6 +29,10 @@ set -e;
 #
 

 
+# Preserve original environment for repro-check command
+declare -r _cygport_orig_env=$(export)
+declare -r _cygport_orig_pwd=$(pwd)
+
 # for regexes, sort, etc.
 export LC_COLLATE=C
 
@@ -784,6 +788,19 @@ do
test ${PIPESTATUS[0]} -eq 0
_status=$?;
;;
+   repro-build)
+   __pkg_repro_build
+   _status=$?
+   ;;
+   repro-diff)
+   __pkg_repro_diff
+   _status=$?
+   ;;
+   repro-check)
+   __pkg_repro_build && \
+   __pkg_repro_diff
+   _status=$?
+   ;;
help)
__show_help;
exit 0;
diff --git a/lib/help.cygpart b/lib/help.cygpart
index a7f30f7a..d28fd7bb 100644
--- a/lib/help.cygpart
+++ b/lib/help.cygpart
@@ -56,6 +56,9 @@ __show_help() {
  finishdelete the working directory
  all   run prep, compile, install and package
  all-test  run prep, compile, install and package-test
+ repro-build   rebuild from created source package to temp 
directory
+ repro-diffcheck whether packages from original and 
rebuild differ
+ repro-check   run repro-build and repro-diff
 
See the included README file for further documentation.
 
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 756a687c..3c531c0e 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -992,6 +992,62 @@ _EOF
fi
 }
 
+__pkg_repro_build() {
+   local srcpkg=${distdir}/${PN}/${PF}-src.tar.${TAR_COMPRESSION_EXT}
+   local t_spkgdir=${T}/${spkgdir##*/}
+   local t_workdir=${t_spkgdir}/${PF}.${ARCH}
+   local t_cygport="cygport ${cygportfile} finish all"
+   local rc
+
+   __stage "Reb

[PATCH cygport] dodoc: Skip a file if a compressed version already exists

[Christian Franke via Cygwin-apps]

It IMO makes sense to compress large and rarely viewed doc files like 
change logs. This seems to be common practice on Debian etc.


With current cygport, the following results in ChangeLog and 
ChangeLog.gz in the docdir:


src_install()
{
  ...
  dodoc ChangeLog
  gzip -9 -n "${D}/usr/share/doc/${PN}/ChangeLog"
}

The attached patch fixes this and also adds some missing documentation.

--
Regards,
Christian

From 1934651b93cda92207429ac91b964cff220c76d5 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 1 Mar 2024 13:56:45 +0100
Subject: [PATCH] dodoc: Skip a file if a compressed version already exists

This prevents that __prepdoc() also adds the uncompressed version
of a default doc file if src_install() installed the compressed
version.  Also add missing documentation about the handling of
FILE.md, FILE.rst and FILE.txt.
---
 lib/src_install.cygpart | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/lib/src_install.cygpart b/lib/src_install.cygpart
index 481457dc..7aca673c 100644
--- a/lib/src_install.cygpart
+++ b/lib/src_install.cygpart
@@ -162,9 +162,12 @@ docinto() {
 #  DESCRIPTION
 #  Installs the given files or directories into $D/usr/share/doc/PN/, or a
 #  subdirectory thereof specified by the previous call to docinto.
+#  If a FILE does not exist, FILE.md, FILE.rst and FILE.txt are also
+#  considered.  A FILE is skipped if the destination file or a compressed
+#  version (.bz2, .gz, .xz, .zstd) of it already exists.
 #
 dodoc() {
-   local docdir d f i x
+   local docdir d e f i x
 
case "${_docinto_dir}" in
'') docdir=/usr/share/doc/${PN} ;;
@@ -191,6 +194,10 @@ dodoc() {
do
if [ -s "${i}${x}" -a ! -f 
"${D}${docdir}/${i}${x}" ]
then
+   for e in bz2 gz xz zst
+   do
+   ! [ -f 
"${D}${docdir}/${i}${x}.${e}" ] || continue 2
+   done
__doinstall 0644 "${i}${x}" ${docdir} 
|| error "dodoc ${i} failed"
break
fi
-- 
2.43.0

Re: Setup.exe suggestions

[Christian Franke via Cygwin]

Hi,

Lavrentiev, Anton wrote

...
For new installations, it'd be great to have the most suitable (in most cases, 
the most
recent) version shown as the first selection in all those pulldown version 
lists in
the "New" columns.  The logical order would be "the preferred (latest) version",
"the version previous to that" ... "test version(s)", and then all the actions 
like
"Uninstall"(grayed out), "Skip" (checked) etc coming last, so that one doesn't 
have to
hover different distances when selecting the packages.


Doubleclick in "New" column toggles between "Skip" or "Keep" and the 
"preferred version". Ctrl+I selects the latter.and moves to the next row.




...
For existing installations, in the "Unneeded" view it'd be best to have the 
packages'
"New" dispositions as "Uninstall" rather than "Keep", which they presently have 
showing
by default.


Changing a view should never change the current package selection.



   So that one does not have to go through (sometimes quite many) packages in
that view to mark them all.


Press and hold Ctrl+U to select "Uninstall" for all. Then change to 
another view and back to "Unneeded" to see more possible unneeded packages.




   If someone wants to retain some, it's easier to change
those few back to "Keep".


Ctrl+K = "Keep", Ctrl+R = "Reinstall".

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH cygport] Set TZ=UTC if SOURCE_DATE_EPOCH is used

[Christian Franke via Cygwin-apps]

Christian Franke wrote:
Further tests of 'repro-check' patch revealed that the "origsrc/*" 
timestamps in patch files contain the local timezone offset.




This would be no longer needed for patch files if "Modify origsrc 
timestamp in patch files if SOURCE_DATE_EPOCH is used" would be applied.


This TZ=UTC patch could possibly be dropped then.


The attached patch is on top of the "Add more checks of 
SOURCE_DATE_EPOCH" patch. It could also be applied independently but 
then requires conflict resolution due to the then missing 'else' branch.

[PATCH cygport] Modify origsrc timestamp in patch files if SOURCE_DATE_EPOCH is used

[Christian Franke via Cygwin-apps]

Found during testing of 'repro-check' patch with getent-2.18.90-5 source 
package.


This patch also removes the requirement to set TZ=UTC before patches are 
generated.


--
Regards,
Christian

From 342ff5113499a83b2ffda441ddc80b4952b400f8 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Wed, 28 Feb 2024 16:46:51 +0100
Subject: [PATCH] Modify origsrc timestamp in patch files if SOURCE_DATE_EPOCH
 is used

Also the timestamp of a file in origsrc directory may be newer than
SOURCE_DATE_EPOCH if modified after unpacking e.g. by a src_*_hook.
---
 lib/pkg_pkg.cygpart | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 756a687c..414ec4b4 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -508,9 +508,9 @@ __pkg_diff() {
 
if [ -n "${SOURCE_DATE_EPOCH}" ]
then
-   # Ensure that the timestamp comment in the generated patch file 
is reproducible
+   # Ensure that the timestamp comments in the patch files are 
reproducible
source_date=$(date -d @"${SOURCE_DATE_EPOCH}" -u +'%Y-%m-%d 
%H:%M:%S.0 +')
-   sed -b -e  "s|^\(+++ [^\t]*\t\).*\$|\1${source_date}|" \
+   sed -E -b -e "s/^((---|\+\+\+) [^\t]*\t).*\$/\1${source_date}/" 
\
-i ${optional_patchfiles} ${patchdir}/${src_patchfile}
fi
 
-- 
2.43.0

Re: [PATCH 2/2] Cygwin: remove ENOSHARE and ECASECLASH from _sys_errlist[]

[Christian Franke]

Hi Corinna,

Corinna Vinschen wrote:

On Feb 27 13:18, Christian Franke wrote:

...

diff --git a/winsup/cygwin/errno.cc b/winsup/cygwin/errno.cc
index 7d58e62ec..d8c057e51 100644
--- a/winsup/cygwin/errno.cc
+++ b/winsup/cygwin/errno.cc
@@ -167,8 +167,8 @@ const char *_sys_errlist[] =
  /* ESTALE 133 */"Stale NFS file handle",
  /* ENOTSUP 134 */   "Not supported",
  /* ENOMEDIUM 135 */ "No medium found",
-/* ENOSHARE 136 */   "No such host or network path",
-/* ECASECLASH 137 */ "Filename exists with different case",
+ NULL, /* Was ENOSHARE 136, no longer used. */
+ NULL, /* Was ECASECLASH 137, no longer used. */

In terms of politenness, wouldn't it be better to define them as
empty strings?  This may be one crash less in already existing
binaries...


Indeed, I missed that case. Patch attached.

Christian

From 151da4ef76f84cd0343e6f49aa23de398ca73d1c Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 27 Feb 2024 17:21:45 +0100
Subject: [PATCH 2/2] Cygwin: set ENOSHARE and ECASECLASH _sys_errlist[]
 entries to empty

These errno values are no longer used by Cygwin.  Change the entries
to empty strings instead of NULL to avoid crashes in existing
binaries directly accessing the table.  Enhance strerror_worker()
such that empty strings also result in "Unknown error ..." messages.
Also add a static_assert check for the _sys_errlist[] size.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/errno.cc | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/winsup/cygwin/errno.cc b/winsup/cygwin/errno.cc
index 7d58e62ec..004a4021e 100644
--- a/winsup/cygwin/errno.cc
+++ b/winsup/cygwin/errno.cc
@@ -167,8 +167,8 @@ const char *_sys_errlist[] =
 /* ESTALE 133 */ "Stale NFS file handle",
 /* ENOTSUP 134 */"Not supported",
 /* ENOMEDIUM 135 */  "No medium found",
-/* ENOSHARE 136 */   "No such host or network path",
-/* ECASECLASH 137 */ "Filename exists with different case",
+ "", /* Was ENOSHARE 136, no longer used. */
+ "", /* Was ECASECLASH 137, no longer used. */
 /* EILSEQ 138 */ "Invalid or incomplete multibyte or wide character",
 /* EOVERFLOW 139 */  "Value too large for defined data type",
 /* ECANCELED 140 */  "Operation canceled",
@@ -177,6 +177,8 @@ const char *_sys_errlist[] =
 /* ESTRPIPE 143 */   "Streams pipe error"
 };
 
+static_assert(143 + 1 == sizeof (_sys_errlist) / sizeof (_sys_errlist[0]));
+
 int NO_COPY_INIT _sys_nerr = sizeof (_sys_errlist) / sizeof (_sys_errlist[0]);
 };
 
@@ -228,7 +230,11 @@ strerror_worker (int errnum)
 {
   char *res;
   if (errnum >= 0 && errnum < _sys_nerr)
-res = (char *) _sys_errlist [errnum];
+{
+  res = (char *) _sys_errlist [errnum];
+  if (res && !*res)
+   res = NULL;
+}
   else
 res = NULL;
   return res;
-- 
2.43.0

[PATCH 2/2] Cygwin: remove ENOSHARE and ECASECLASH from _sys_errlist[]

[Christian Franke]

From f495fb0e7c2bd3a42f16f81af18c64ffaba9a860 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 27 Feb 2024 13:05:36 +0100
Subject: [PATCH 2/2] Cygwin: remove ENOSHARE and ECASECLASH from
 _sys_errlist[]

These errno values are no longer used by Cygwin.  Also add a
static_assert check for _sys_errlist[] size.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/errno.cc | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/errno.cc b/winsup/cygwin/errno.cc
index 7d58e62ec..d8c057e51 100644
--- a/winsup/cygwin/errno.cc
+++ b/winsup/cygwin/errno.cc
@@ -167,8 +167,8 @@ const char *_sys_errlist[] =
 /* ESTALE 133 */ "Stale NFS file handle",
 /* ENOTSUP 134 */"Not supported",
 /* ENOMEDIUM 135 */  "No medium found",
-/* ENOSHARE 136 */   "No such host or network path",
-/* ECASECLASH 137 */ "Filename exists with different case",
+ NULL, /* Was ENOSHARE 136, no longer used. */
+ NULL, /* Was ECASECLASH 137, no longer used. */
 /* EILSEQ 138 */ "Invalid or incomplete multibyte or wide character",
 /* EOVERFLOW 139 */  "Value too large for defined data type",
 /* ECANCELED 140 */  "Operation canceled",
@@ -177,6 +177,8 @@ const char *_sys_errlist[] =
 /* ESTRPIPE 143 */   "Streams pipe error"
 };
 
+static_assert(143 + 1 == sizeof (_sys_errlist) / sizeof (_sys_errlist[0]));
+
 int NO_COPY_INIT _sys_nerr = sizeof (_sys_errlist) / sizeof (_sys_errlist[0]);
 };
 
-- 
2.43.0

[PATCH 1/2] Cygwin: add compile warning if ENOSHARE or ECASECLASH is used

[Christian Franke]

A suggestion for a first (possibly polite) step to get rid of ENOSHARE 
or ECASECLASH. Would also work with clang.


The internally used ENMFILE is not included yet. In theory, it may be 
returned to outside world as it still appears in errmap[].


--
Regards,
Christian

From f3ad1912a9c7fa4cd28ade5dc7c95134be759615 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 27 Feb 2024 13:03:08 +0100
Subject: [PATCH 1/2] Cygwin: add compile warning if ENOSHARE or ECASECLASH is
 used

These errno values are no longer used by Cygwin.

Signed-off-by: Christian Franke 
---
 newlib/libc/include/sys/errno.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/newlib/libc/include/sys/errno.h b/newlib/libc/include/sys/errno.h
index f1509712e..7199db0d2 100644
--- a/newlib/libc/include/sys/errno.h
+++ b/newlib/libc/include/sys/errno.h
@@ -176,9 +176,9 @@ extern __IMPORT char *program_invocation_short_name;
 #ifdef __LINUX_ERRNO_EXTENSIONS__
 #define ENOMEDIUM 135   /* No medium (in tape drive) */
 #endif
-#ifdef __CYGWIN__
-#define ENOSHARE 136/* No such host or network path */
-#define ECASECLASH 137  /* Filename exists with different case */
+#if defined(__CYGWIN__) && !defined(__INSIDE_CYGWIN__)
+#define ENOSHARE (_Pragma("GCC warning \"'ENOSHARE' is no longer used by 
Cygwin\"") 136)
+#define ECASECLASH (_Pragma("GCC warning \"'ECASECLASH' is no longer used by 
Cygwin\"") 137)
 #endif
 #define EILSEQ 138 /* Illegal byte sequence */
 #define EOVERFLOW 139  /* Value too large for defined data type */
-- 
2.43.0

Re: [PATCH cygport] Add more checks of SOURCE_DATE_EPOCH

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 16/02/2024 12:29, Christian Franke via Cygwin-apps wrote:
Fail if it is out of range. Warn if it lies in the future. Inform 
whether it is set or set but not exported.


What is the valid range here?


The range accepted by 'date -d @EPOCH ...', later used to adjust the 
patch timestamps.


The test could also be removed as date interestingly accepts a (too) 
wide range of values :-)


$ date -d @$((1<<55))
Sun Jun 13 08:26:08 CEST 1141709097

$ date -d @$((1<<56))
date: time ‘72057594037927936’ is out of range

$ date -d @$((-(1<<55)))
Sun Jul 20 18:27:20 LMT -1141705158

$ date -d @$((-(1<<56)))
date: time ‘-72057594037927936’ is out of range



Would it not make more sense to just re-export it if set?


If the cygport file decides to set but not export it, there is possibly 
no need to do it. An example is smartmontools.cygport which passes the 
unexported variable as a parameter to configure.



(so that commands like "SOURCE_DATE_EPOCH=something cygport foo" work 
as expected?)




Would make no difference as the 'VAR=val CMD...' syntax already exports 
the variable to the CMD:


$ unset FOO; FOO=bar sh -c 'sh -c "sh -c printenv\ FOO"'
bar

[PATCH 4/4] Cygwin: errmap[]: update comments using current winerror.h

[Christian Franke]

From 66a9c2d35a836017e52c8ce73b76b2070c3b6a95 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 26 Feb 2024 15:15:18 +0100
Subject: [PATCH 4/4] Cygwin: errmap[]: update comments using current
 winerror.h

Signed-off-by: Christian Franke 
---
 winsup/cygwin/local_includes/errmap.h | 1701 +
 1 file changed, 851 insertions(+), 850 deletions(-)

diff --git a/winsup/cygwin/local_includes/errmap.h 
b/winsup/cygwin/local_includes/errmap.h
index 496007491..354968f86 100644
--- a/winsup/cygwin/local_includes/errmap.h
+++ b/winsup/cygwin/local_includes/errmap.h
@@ -1,6 +1,7 @@
 /* This file has been autogenerated from MingW-w64's winerror.h and our
-   previous errno.cc on 2022-08-04.  More error codes can be easily added
-   to this new array manually on demand. */
+   previous errno.cc on 2022-08-04.  It has been updated on 2024-02-26
+   using winerror.h from w32api-headers-11.0.1-1.  More error codes can
+   be easily added to this new array manually on demand. */
 
 /* FIXME: Some of these choices are arbitrary! */
 constexpr uint8_t errmap[] =
@@ -176,7 +177,7 @@ constexpr uint8_t errmap[] =
   0,   /* 168 */
   0,   /* 169 */
   EBUSY,   /* 170: ERROR_BUSY */
-  0,   /* 171 */
+  0,   /* 171: ERROR_DEVICE_SUPPORT_IN_PROGRESS */
   0,   /* 172 */
   0,   /* 173: ERROR_CANCEL_VIOLATION */
   0,   /* 174: ERROR_ATOMIC_LOCKS_NOT_SUPPORTED */
@@ -230,8 +231,8 @@ constexpr uint8_t errmap[] =
   0,   /* 222: ERROR_BAD_FILE_TYPE */
   0,   /* 223: ERROR_FILE_TOO_LARGE */
   0,   /* 224: ERROR_FORMS_AUTH_REQUIRED */
-  0,   /* 225 */
-  0,   /* 226 */
+  0,   /* 225: ERROR_VIRUS_INFECTED */
+  0,   /* 226: ERROR_VIRUS_DELETED */
   0,   /* 227 */
   0,   /* 228 */
   0,   /* 229: ERROR_PIPE_LOCAL */
@@ -240,7 +241,7 @@ constexpr uint8_t errmap[] =
   EPIPE,   /* 232: ERROR_NO_DATA */
   ECOMM,   /* 233: ERROR_PIPE_NOT_CONNECTED */
   EMSGSIZE,/* 234: ERROR_MORE_DATA */
-  0,   /* 235 */
+  0,   /* 235: ERROR_NO_WORK_DONE */
   0,   /* 236 */
   0,   /* 237 */
   0,   /* 238 */
@@ -309,163 +310,163 @@ constexpr uint8_t errmap[] =
   0,   /* 301: ERROR_INVALID_OPLOCK_PROTOCOL */
   0,   /* 302: ERROR_DISK_TOO_FRAGMENTED */
   0,   /* 303: ERROR_DELETE_PENDING */
-  0,   /* 304 */
-  0,   /* 305 */
-  0,   /* 306 */
-  0,   /* 307 */
-  0,   /* 308 */
-  0,   /* 309 */
-  0,   /* 310 */
-  0,   /* 311 */
-  0,   /* 312 */
-  0,   /* 313 */
-  0,   /* 314 */
+  0,   /* 304: 
ERROR_INCOMPATIBLE_WITH_GLOBAL_SHORT_NAME_REGISTRY_SETTING */
+  0,   /* 305: ERROR_SHORT_NAMES_NOT_ENABLED_ON_VOLUME */
+  0,   /* 306: ERROR_SECURITY_STREAM_IS_INCONSISTENT */
+  0,   /* 307: ERROR_INVALID_LOCK_RANGE */
+  0,   /* 308: ERROR_IMAGE_SUBSYSTEM_NOT_PRESENT */
+  0,   /* 309: ERROR_NOTIFICATION_GUID_ALREADY_DEFINED */
+  0,   /* 310: ERROR_INVALID_EXCEPTION_HANDLER */
+  0,   /* 311: ERROR_DUPLICATE_PRIVILEGES */
+  0,   /* 312: ERROR_NO_RANGES_PROCESSED */
+  0,   /* 313: ERROR_NOT_ALLOWED_ON_SYSTEM_FILE */
+  0,   /* 314: ERROR_DISK_RESOURCES_EXHAUSTED */
   0,   /* 315: ERROR_INVALID_TOKEN */
-  0,   /* 316 */
+  0,   /* 316: ERROR_DEVICE_FEATURE_NOT_SUPPORTED */
   0,   /* 317: ERROR_MR_MID_NOT_FOUND */
   0,   /* 318: ERROR_SCOPE_NOT_FOUND */
-  0,   /* 319 */
-  0,   /* 320 */
-  0,   /* 321 */
-  0,   /* 322 */
-  0,   /* 323 */
-  0,   /* 324 */
+  0,   /* 319: ERROR_UNDEFINED_SCOPE */
+  0,   /* 320: ERROR_INVALID_CAP */
+  0,   /* 321: ERROR_DEVICE_UNREACHABLE */
+  0,   /* 322: ERROR_DEVICE_NO_RESOURCES */
+  0,   /* 323: ERROR_DATA_CHECKSUM_ERROR */
+  0,   /* 324: ERROR_INTERMIXED_KERNEL_EA_OPERATION */
   0,   /* 325 */
-  0,   /* 326 */
-  0,   /* 327 */
-  0,   /* 328 */
-  0,   /* 329 */
-  0,   /* 330 */
-  0,   /* 331 */
-  0,   /* 332 */
-  0,   /* 333 */
-  0,   /* 334

[PATCH 2/4] Cygwin: errmap[]: reduce value size from 32 to 8 bits

[Christian Franke]

From bebc11c2a9022e61cdc7544ed2cb5101a170ae92 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 26 Feb 2024 14:01:54 +0100
Subject: [PATCH 2/4] Cygwin: errmap[]: reduce value size from 32 to 8 bits

Overflow would be detected at compile time.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/local_includes/errmap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/winsup/cygwin/local_includes/errmap.h 
b/winsup/cygwin/local_includes/errmap.h
index 737c01c8b..eacf9cd1c 100644
--- a/winsup/cygwin/local_includes/errmap.h
+++ b/winsup/cygwin/local_includes/errmap.h
@@ -3,7 +3,7 @@
to this new array manually on demand. */
 
 /* FIXME: Some of these choices are arbitrary! */
-constexpr int errmap[] =
+constexpr uint8_t errmap[] =
 {
   0,   /* ERROR_SUCCESS */
   EBADRQC, /* ERROR_INVALID_FUNCTION */
-- 
2.43.0

[PATCH 1/4] Cygwin: introduce constexpr errmap_size and errmap[] consistency checks

[Christian Franke]

From 947daa02b0b64131626c2ecedb74ca6893aab6c6 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 26 Feb 2024 13:37:33 +0100
Subject: [PATCH 1/4] Cygwin: introduce constexpr errmap_size and errmap[]
 consistency checks

Use constexpr instead of const for errmap[] to allow static_assert
checks on its values.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/errno.cc|  2 +-
 winsup/cygwin/local_includes/errmap.h | 11 ++-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/errno.cc b/winsup/cygwin/errno.cc
index 1c85e9a04..7d58e62ec 100644
--- a/winsup/cygwin/errno.cc
+++ b/winsup/cygwin/errno.cc
@@ -185,7 +185,7 @@ geterrno_from_win_error (DWORD code, int deferrno)
 {
   /* A 0-value in errmap means, we don't handle this windows error
  explicitely.  Fall back to deferrno in these cases. */
-  if (code < sizeof errmap / sizeof errmap[0] && errmap[code])
+  if (code < errmap_size && errmap[code])
 {
   syscall_printf ("windows error %u == errno %d", code, errmap[code]);
   return errmap[code];
diff --git a/winsup/cygwin/local_includes/errmap.h 
b/winsup/cygwin/local_includes/errmap.h
index a0b3ff400..737c01c8b 100644
--- a/winsup/cygwin/local_includes/errmap.h
+++ b/winsup/cygwin/local_includes/errmap.h
@@ -3,7 +3,7 @@
to this new array manually on demand. */
 
 /* FIXME: Some of these choices are arbitrary! */
-static const int errmap[] =
+constexpr int errmap[] =
 {
   0,   /* ERROR_SUCCESS */
   EBADRQC, /* ERROR_INVALID_FUNCTION */
@@ -9006,3 +9006,12 @@ static const int errmap[] =
   0,   /* 8998 */
   0,   /* 8999 */
 };
+
+constexpr unsigned errmap_size = sizeof (errmap) / sizeof (errmap[0]);
+
+/* Some consistency checks. */
+static_assert (errmap_size == 8999 + 1);
+static_assert (EINTR == errmap[/* 104 */ ERROR_INVALID_AT_INTERRUPT_TIME]);
+static_assert (ENXIO == errmap[/* 1006 */ ERROR_FILE_INVALID]);
+static_assert (EAGAIN == errmap[/* 2404 */ ERROR_DEVICE_IN_USE]);
+static_assert (EIO == errmap[/* 8341 */ ERROR_DS_GENERIC_ERROR]);
-- 
2.43.0

Re: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to ENODEV

[Christian Franke]

Corinna Vinschen wrote:

On Feb 25 10:12, Christian Franke wrote:

Corinna Vinschen wrote:

So the default was EPERM at first and has been changed to EACCES
because it "is better for the unknown error case".

I'm open to ideas for an improved error mapping.

I have no better suggestion for a default errno. Adding a cygwin specific
one (like ENMFILE, ENOSHARE and ECASECLASH added 2000-2001) is possibly not
desired.

ENOSHARE and ECASECLASH are not used anymore, fortunately, and ENMFILE
is hopefully never returned to userspace.  It might be a good idea to
remove it from Cygwin's code as well.


Some thoughts about minor improvements of the errmap.h file:
- Add error number to each /* ERROR_... */ comment, e.g. /* 2:
ERROR_FILE_NOT_FOUND */.

Ok.


- Update /* NUMBER */ comments using current MinGW-w64's winerror.h (~850
changes).

Why so many?  I used winerror.h to populate the list not too long ago,
so I wonder why it suddenly has so many more error codes?


"Required for mozilla-central." - 850 insertions:
https://sourceforge.net/p/mingw-w64/mingw-w64/ci/ddeb05a

Most or all would possible never occur with the NTDLL/Win32 API subset 
used by Cygwin.


Includes interesting codes like "ERROR_NO_WORK_DONE" :-)



- Max errno is 143, so data type size could be reduced from int to uint8_t
aka unsigned char. Could even add a compile time check by using C++11's
braced initializers which do not allow narrowing conversions.

Yeah, we could do that.


- Remove trailing entries which only map to 0.
- Append a static_assert which checks whether array size matches the last
mapped error number.

Yeah, not so sure about that.  I'm aware that we only map errors
below 3000 somewhere, but it's no safe bet that it stays that way.

For instance, we handle NT status codes STATUS_TRANSACTIONAL_CONFLICT
and STATUS_TRANSACTION_NOT_ENLISTED and those translate into the TxF
error range between 6800 and 6899.  We don't convert those to userspace
errno yet, but consider having to add them at one point and thus having
to add the 3000 entries from the last used one up the newly used one.

The reason to keep them is to allow us to be lazy about it.  The list
also just takes ~36K, and with the change to uint8_t it only takes
9K, so what?


Ok.


I could provide separate patches if desired.

Always welcome!


Ok.

Thanks,
Christian

Re: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to ENODEV

[Christian Franke]

Corinna Vinschen wrote:

On Feb 23 19:14, Christian Franke wrote:

Experiments with damaged USB flash drives and ddrescue revealed that the
current mapping of these Win32 errors to the fallback EACCES could be
improved.

BTW: I wonder why EACCES was selected as the fallback. Source code control
forensics suggest that this was decided in the last millennium. A related
comment from CGF added August 2000 persists until today :-)
/* FIXME: what's so special about EACCESS? */

This goes back until 1997 in pre-CVS times.  There's a ChangeLog entry

   Wed Oct 29 22:43:57 1997  Geoffrey Noer  

 [...]
 * syscalls.cc (seterrno): on failure, set EACCES instead of EPERM
 which is better for the unknown error case

So the default was EPERM at first and has been changed to EACCES
because it "is better for the unknown error case".

I'm open to ideas for an improved error mapping.


I have no better suggestion for a default errno. Adding a cygwin 
specific one (like ENMFILE, ENOSHARE and ECASECLASH added 2000-2001) is 
possibly not desired.


Some thoughts about minor improvements of the errmap.h file:
- Add error number to each /* ERROR_... */ comment, e.g. /* 2: 
ERROR_FILE_NOT_FOUND */.
- Update /* NUMBER */ comments using current MinGW-w64's winerror.h 
(~850 changes).
- Max errno is 143, so data type size could be reduced from int to 
uint8_t aka unsigned char. Could even add a compile time check by using 
C++11's braced initializers which do not allow narrowing conversions.

- Remove trailing entries which only map to 0.
- Append a static_assert which checks whether array size matches the 
last mapped error number.


I could provide separate patches if desired.

Thanks,
Christian

[PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to ENODEV

[Christian Franke]

Experiments with damaged USB flash drives and ddrescue revealed that the 
current mapping of these Win32 errors to the fallback EACCES could be 
improved.


BTW: I wonder why EACCES was selected as the fallback. Source code 
control forensics suggest that this was decided in the last millennium. 
A related comment from CGF added August 2000 persists until today :-)

/* FIXME: what's so special about EACCESS? */

--
Regards,
Christian

From 8aa19c7fd13dc3790dc271dede8954539bffcd4d Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 23 Feb 2024 19:01:09 +0100
Subject: [PATCH] Cygwin: Map ERROR_NO_SUCH_DEVICE and ERROR_MEDIA_CHANGED to
 ENODEV

If a removable (USB) device is disconnected after opening its raw
device, R/W attempts fail with ERROR_NO_SUCH_DEVICE(433).  If the
raw device of a partition is used, ERROR_MEDIA_CHANGED(1110) is
returned instead.  Both are mapped to ENODEV(19) because 
does not offer a value which better matches ERROR_MEDIA_CHANGED.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/local_includes/errmap.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/winsup/cygwin/local_includes/errmap.h 
b/winsup/cygwin/local_includes/errmap.h
index 326b35b6c..a0b3ff400 100644
--- a/winsup/cygwin/local_includes/errmap.h
+++ b/winsup/cygwin/local_includes/errmap.h
@@ -438,7 +438,7 @@ static const int errmap[] =
   0,   /* 430 */
   0,   /* 431 */
   0,   /* 432 */
-  0,   /* 433 */
+  ENODEV,  /* ERROR_NO_SUCH_DEVICE */
   0,   /* 434 */
   0,   /* 435 */
   0,   /* 436 */
@@ -1115,7 +1115,7 @@ static const int errmap[] =
   0,   /* ERROR_DEVICE_NOT_PARTITIONED */
   0,   /* ERROR_UNABLE_TO_LOCK_MEDIA */
   0,   /* ERROR_UNABLE_TO_UNLOAD_MEDIA */
-  0,   /* ERROR_MEDIA_CHANGED */
+  ENODEV,  /* ERROR_MEDIA_CHANGED */
   EIO, /* ERROR_BUS_RESET */
   ENOMEDIUM,   /* ERROR_NO_MEDIA_IN_DRIVE */
   0,   /* ERROR_NO_UNICODE_TRANSLATION */
-- 
2.43.0

[PATCH cygport] Fix variable expansion in error message of embedded SMTP perl script

[Christian Franke via Cygwin-apps]

Harmless bug ...

--
Regards,
Christian

From b1074f4cfe549c852be7fa59d85d312c9579cf0d Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 23 Feb 2024 13:04:21 +0100
Subject: [PATCH] Fix variable expansion in error message of embedded SMTP perl
 script

---
 lib/pkg_upload.cygpart | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/pkg_upload.cygpart b/lib/pkg_upload.cygpart
index dcae8e2c..cdeae283 100644
--- a/lib/pkg_upload.cygpart
+++ b/lib/pkg_upload.cygpart
@@ -222,7 +222,7 @@ if (\$smtp_encryption eq 'tls') {
require Net::SMTP::SSL;
\$smtp->command('STARTTLS');
\$smtp->response();
-   \$smtp->code == 220 or die "$server does not support STARTTLS";
+   \$smtp->code == 220 or die "\$smtp_server does not support STARTTLS";
\$smtp = Net::SMTP::SSL->start_SSL(\$smtp) or die "STARTTLS failed";
\$smtp->hello(\$smtp_server);
 }
-- 
2.43.0

Re: [PATCH cygport] Add customization support for announce command

[Christian Franke via Cygwin-apps]

Christian Franke wrote:
The email generated by the cygport announce command is useful, but 
actual use cases are somewhat limited due to the hard-coded email 
submission.


The attached patch adds more flexibility. The patch is on top of the 
"Use correct wording if only one package is announced" patch.


Slightly changed patch attached. Also adjusted to new version of "Use 
correct wording if only one package is announced" patch.





Examples for cygport.conf settings:

ANNOUNCE_EDITOR='printf "\\nRegards,\\n$PN package maintainer\\n" >>"$1"'

ANNOUNCE_EDITOR='
n=$(wc -l <"$1") && cat >>"$1" <>> This is an update to the latest upstream release.
  >>> This is a bugfix release.
  <<< PLEASE EDIT >>>

Regards,
$PN package maintainer
EOF
vim +$((n+2)) "$1" && ! grep -E "<<<|>>>" "$1"
'

ANNOUNCE_TO_CMD='cat "$1" >/dev/clipboard'

ANNOUNCE_TO_CMD='
sed "1,/^\$/d" "$1" >$PF-announcement.txt &&
echo "Announcement placed here: $(pwd)/$PF-announcement.txt"
'

ANNOUNCE_TO_CMD='
/usr/local/sbin/custom-mailer \
  --sender="$SMTP_SENDER" \
  --smarthost="$SMTP_SERVER" \
  ...more...options... \
  cygwin-annou...@cygwin.com <"$1"
'



Possible (better?) alternative names for the new settings:
ANNOUNCEMENT_EDITOR
ANNOUNCEMENT_MAILER

From 14709f0a1ed19c7d00588fb2a1fa7273d47e00fd Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 23 Feb 2024 12:04:17 +0100
Subject: [PATCH] Add customization support for announce command

Two new configuration settings allow to override the launch of
a text editor (ANNOUNCE_EDITOR) and the builtin email submission
(ANNOUNCE_TO_CMD).
---
 data/cygport.conf  | 23 +
 lib/pkg_upload.cygpart | 56 +-
 2 files changed, 73 insertions(+), 6 deletions(-)

diff --git a/data/cygport.conf b/data/cygport.conf
index 34ccd291..48dc7bfe 100644
--- a/data/cygport.conf
+++ b/data/cygport.conf
@@ -101,6 +101,29 @@
 #PAGER=
 
 
+#v* Configuration/ANNOUNCE_EDITOR
+#  DESCRIPTION
+#  Shell command string to process the email message created by cygport's
+#  announce command before sending the email.  If undefined, a text editor
+#  will be run, see EDITOR setting above.  If empty, nothing will be run.
+#  If not empty, '/bin/bash' will be launched with the command string passed
+#  with '-c' option and the path of the temporary email message file as '$1'.
+#  The working directory of the shell will be the directory of the cygport
+#  file.  The specified command string will be prepended by shell assignments
+#  of the cygport variables P, PF, PN, PR and PV and all SMTP_* settings
+#  described below.
+#ANNOUNCE_EDITOR=
+
+#v* Configuration/ANNOUNCE_TO_CMD
+#  DESCRIPTION
+#  Shell command string to process the email message created by cygport's
+#  announce command after editing.  If undefined, the email will be sent
+#  using the builtin perl-based SMTP support.  If empty, nothing will be run.
+#  If not empty, the command string will be handled similar to ANNOUNCE_EDITOR
+#  described above.
+#ANNOUNCE_TO_CMD=
+
+
 #v* Configuration/SMTP_SENDER
 #  DESCRIPTION
 #  Name and email address, in the form of "First Last " to be used
diff --git a/lib/pkg_upload.cygpart b/lib/pkg_upload.cygpart
index 37bc2d63..d38ea8b6 100644
--- a/lib/pkg_upload.cygpart
+++ b/lib/pkg_upload.cygpart
@@ -168,6 +168,33 @@ EOF
echo "Upload complete."
 }
 
+__pkg_announce_run_cmd_on_msg() {
+   local cmdvar=$1
+   local msg=$2
+   local cmd
+
+   eval cmd="\${${cmdvar}}"
+
+   if [ "${cmd:+y}" != "y" ]
+   then
+   inform "\${${cmdvar}} is empty"
+   return 0
+   fi
+   echo
+   inform "Launching '\${${cmdvar}} ${msg}'"
+
+   /bin/bash -c "cd ${top} || exit 1
+P=${P@Q}; PF=${PF@Q}; PN=${PN@Q}; PR=${PR@Q}; PV=(${PV[*]@Q})
+${SMTP_SENDER+SMTP_SENDER=${SMTP_SENDER@Q}}
+${SMTP_SERVER+SMTP_SERVER=${SMTP_SERVER@Q}}
+${SMTP_SERVER_PORT+SMTP_SERVER_PORT=${SMTP_SERVER_PORT@Q}}
+${SMTP_ENCRYPTION+SMTP_ENCRYPTION=${SMTP_ENCRYPTION@Q}}
+${SMTP_USER+SMTP_USER=${SMTP_USER@Q}}
+${SMTP_PASS+SMTP_PASS=${SMTP_PASS@Q}}
+${cmd}
+"  $0 ${msg} || error "Command '\${${cmdvar}} ${msg}' (cwd=${top}) 
failed"
+}
+
 __pkg_announce() {
local msg=$(mktemp -t cygwin-announce-${PF}.XX)
local msgat=$(date +@%s)
@@ -178,10 +205,10 @@ __pkg_announce() {
 
cat > ${msg} <<_EOF
 From cygwin-announce-${PF} $(date '+%a %b %d %H:%M:%S %Y' --date=${msgat})
-From: ${SMTP_SENDER}
-To: cygwin-annou...@cygwin.com
+${SMTP_SENDER:+From: ${SMTP_SENDER}
+}To: cygwin-annou...@cygwin.com
 Date: $(date -R --date=${msgat})
-Message-Id: <$

Re: [PATCH cygport] Use correct wording if only one package is announced

[Christian Franke via Cygwin-apps]

Brian Inglis via Cygwin-apps wrote:

On 2024-02-21 07:25, Christian Franke via Cygwin-apps wrote:

Change variable name from $s to $has or $s_have as variable $s usually 
implies only the plural letter s or nothing; e.g.

...
+    local has="s have"
+
+    [ $pkg_count != 1 ] || has=" has"
...
+The following package${has} been uploaded to the Cygwin distribution:
...


Agree - new patch attached.

From 6da0f806a94621d0ecfcca3c63e5c46e8ab3cd32 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 23 Feb 2024 11:35:01 +0100
Subject: [PATCH] Use correct wording if only one package is announced

---
 lib/pkg_upload.cygpart | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/pkg_upload.cygpart b/lib/pkg_upload.cygpart
index dcae8e2c..37bc2d63 100644
--- a/lib/pkg_upload.cygpart
+++ b/lib/pkg_upload.cygpart
@@ -172,6 +172,9 @@ __pkg_announce() {
local msg=$(mktemp -t cygwin-announce-${PF}.XX)
local msgat=$(date +@%s)
local -i n=0
+   local s_have="s have"
+
+   [ $pkg_count != 1 ] || s_have=" has"
 
cat > ${msg} <<_EOF
 From cygwin-announce-${PF} $(date '+%a %b %d %H:%M:%S %Y' --date=${msgat})
@@ -181,7 +184,7 @@ Date: $(date -R --date=${msgat})
 Message-Id: <$(date "+%Y%m%d%H%M%S.$$" --date=${msgat})-1-$(echo 
${SMTP_SENDER} | sed 's|.*<\(.*\)>.*|\1|')>
 Subject: ${NAME} ${PVR}
 
-The following packages have been uploaded to the Cygwin distribution:
+The following package${s_have} been uploaded to the Cygwin distribution:
 
 _EOF
 
-- 
2.43.0

[PATCH cygport] Add customization support for announce command

[Christian Franke via Cygwin-apps]

The email generated by the cygport announce command is useful, but 
actual use cases are somewhat limited due to the hard-coded email 
submission.


The attached patch adds more flexibility. The patch is on top of the 
"Use correct wording if only one package is announced" patch.


Examples for cygport.conf settings:

ANNOUNCE_EDITOR='printf "\\nRegards,\\n$PN package maintainer\\n" >>"$1"'

ANNOUNCE_EDITOR='
n=$(wc -l <"$1") && cat >>"$1" <>> This is an update to the latest upstream release.
  >>> This is a bugfix release.
  <<< PLEASE EDIT >>>

Regards,
$PN package maintainer
EOF
vim +$((n+2)) "$1" && ! grep -E "<<<|>>>" "$1"
'

ANNOUNCE_TO_CMD='cat "$1" >/dev/clipboard'

ANNOUNCE_TO_CMD='
sed "1,/^\$/d" "$1" >$PF-announcement.txt &&
echo "Announcement placed here: $(pwd)/$PF-announcement.txt"
'

ANNOUNCE_TO_CMD='
/usr/local/sbin/custom-mailer \
  --sender="$SMTP_SENDER" \
  --smarthost="$SMTP_SERVER" \
  ...more...options... \
  cygwin-annou...@cygwin.com <"$1"
'

--
Regards,
Christian

From 1f13d54a40d639938fb67245eed4615be0a6e6c4 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Wed, 21 Feb 2024 15:14:53 +0100
Subject: [PATCH] Add customization support for announce command

Two new configuration settings allow to override the launch of
a text editor (ANNOUNCE_EDITOR) and the builtin email submission
(ANNOUNCE_TO_CMD).
---
 data/cygport.conf  | 23 +
 lib/pkg_upload.cygpart | 57 +-
 2 files changed, 74 insertions(+), 6 deletions(-)

diff --git a/data/cygport.conf b/data/cygport.conf
index 34ccd291..48dc7bfe 100644
--- a/data/cygport.conf
+++ b/data/cygport.conf
@@ -101,6 +101,29 @@
 #PAGER=
 
 
+#v* Configuration/ANNOUNCE_EDITOR
+#  DESCRIPTION
+#  Shell command string to process the email message created by cygport's
+#  announce command before sending the email.  If undefined, a text editor
+#  will be run, see EDITOR setting above.  If empty, nothing will be run.
+#  If not empty, '/bin/bash' will be launched with the command string passed
+#  with '-c' option and the path of the temporary email message file as '$1'.
+#  The working directory of the shell will be the directory of the cygport
+#  file.  The specified command string will be prepended by shell assignments
+#  of the cygport variables P, PF, PN, PR and PV and all SMTP_* settings
+#  described below.
+#ANNOUNCE_EDITOR=
+
+#v* Configuration/ANNOUNCE_TO_CMD
+#  DESCRIPTION
+#  Shell command string to process the email message created by cygport's
+#  announce command after editing.  If undefined, the email will be sent
+#  using the builtin perl-based SMTP support.  If empty, nothing will be run.
+#  If not empty, the command string will be handled similar to ANNOUNCE_EDITOR
+#  described above.
+#ANNOUNCE_TO_CMD=
+
+
 #v* Configuration/SMTP_SENDER
 #  DESCRIPTION
 #  Name and email address, in the form of "First Last " to be used
diff --git a/lib/pkg_upload.cygpart b/lib/pkg_upload.cygpart
index 8039ec5c..b81bf3d5 100644
--- a/lib/pkg_upload.cygpart
+++ b/lib/pkg_upload.cygpart
@@ -168,6 +168,28 @@ EOF
echo "Upload complete."
 }
 
+__pkg_announce_run_cmd_on_msg() {
+   local cmdvar=$1
+   local msg=$2
+   local cmd
+
+   eval cmd="\${${cmdvar}}"
+
+   (
+   cd ${top} && /bin/bash -c "\
+P=${P@Q}; PF=${PF@Q}; PN=${PN@Q}; PR=${PR@Q}; PV=(${PV[*]@Q})
+${SMTP_SENDER+SMTP_SENDER=${SMTP_SENDER@Q}}
+${SMTP_SERVER+SMTP_SERVER=${SMTP_SERVER@Q}}
+${SMTP_SERVER_PORT+SMTP_SERVER_PORT=${SMTP_SERVER_PORT@Q}}
+${SMTP_ENCRYPTION+SMTP_ENCRYPTION=${SMTP_ENCRYPTION@Q}}
+${SMTP_USER+SMTP_USER=${SMTP_USER@Q}}
+${SMTP_PASS+SMTP_PASS=${SMTP_PASS@Q}}
+${cmd}
+" \
+   $0 ${msg}
+   ) || error "Command '\${${cmdvar}} ${msg}' (cwd=${top}) failed"
+}
+
 __pkg_announce() {
local msg=$(mktemp -t cygwin-announce-${PF}.XX)
local msgat=$(date +@%s)
@@ -178,10 +200,10 @@ __pkg_announce() {
 
cat > ${msg} <<_EOF
 From cygwin-announce-${PF} $(date '+%a %b %d %H:%M:%S %Y' --date=${msgat})
-From: ${SMTP_SENDER}
-To: cygwin-annou...@cygwin.com
+${SMTP_SENDER:+From: ${SMTP_SENDER}
+}To: cygwin-annou...@cygwin.com
 Date: $(date -R --date=${msgat})
-Message-Id: <$(date "+%Y%m%d%H%M%S.$$" --date=${msgat})-1-$(echo 
${SMTP_SENDER} | sed 's|.*<\(.*\)>.*|\1|')>
+Message-Id: <$(date "+%Y%m%d%H%M%S.$$" --date=${msgat})-1-$(echo 
${SMTP_SENDER:-cygport} | sed 's|.*<\(.*\)>.*|\1|')>
 Subject: ${NAME} ${PVR}
 
 The following package${s} been uploaded to the Cygwin distribution:
@@ -199,7 +221,30 @@ _EOF
 ${DESCRIPTION}
 _EOF
 
-   ${EDITOR:-vi} $msg || error 

[PATCH cygport] Use correct wording if only one package is announced

[Christian Franke via Cygwin-apps]

Only cosmetic ...

--
Regards,
Christian

From f1381ebc872f3b099c257677e2b8d5bf2451bb23 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Wed, 21 Feb 2024 13:35:14 +0100
Subject: [PATCH] Use correct wording if only one package is announced

---
 lib/pkg_upload.cygpart | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/pkg_upload.cygpart b/lib/pkg_upload.cygpart
index dcae8e2c..8039ec5c 100644
--- a/lib/pkg_upload.cygpart
+++ b/lib/pkg_upload.cygpart
@@ -172,6 +172,9 @@ __pkg_announce() {
local msg=$(mktemp -t cygwin-announce-${PF}.XX)
local msgat=$(date +@%s)
local -i n=0
+   local s="s have"
+
+   [ $pkg_count != 1 ] || s=" has"
 
cat > ${msg} <<_EOF
 From cygwin-announce-${PF} $(date '+%a %b %d %H:%M:%S %Y' --date=${msgat})
@@ -181,7 +184,7 @@ Date: $(date -R --date=${msgat})
 Message-Id: <$(date "+%Y%m%d%H%M%S.$$" --date=${msgat})-1-$(echo 
${SMTP_SENDER} | sed 's|.*<\(.*\)>.*|\1|')>
 Subject: ${NAME} ${PVR}
 
-The following packages have been uploaded to the Cygwin distribution:
+The following package${s} been uploaded to the Cygwin distribution:
 
 _EOF
 
-- 
2.43.0

[PATCH cygport] Set TZ=UTC if SOURCE_DATE_EPOCH is used

[Christian Franke via Cygwin-apps]

Further tests of 'repro-check' patch revealed that the "origsrc/*" 
timestamps in patch files contain the local timezone offset.


The attached patch is on top of the "Add more checks of 
SOURCE_DATE_EPOCH" patch. It could also be applied independently but 
then requires conflict resolution due to the then missing 'else' branch.


--
Regards,
Christian

From e9553d8541f9fb2fd4a4e49a488dfc8511fb9c8e Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 20 Feb 2024 10:32:52 +0100
Subject: [PATCH] Set TZ=UTC if SOURCE_DATE_EPOCH is used

---
 bin/cygport.in | 4 
 1 file changed, 4 insertions(+)

diff --git a/bin/cygport.in b/bin/cygport.in
index 3fe8a52e..2261a5bd 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -517,6 +517,10 @@ then
4.6.[6-9]|4.[7-9]*|[5-9]*) ;;
*) error "SOURCE_DATE_EPOCH requires peflags 4.6.6 or later"
esac
+
+   # Ensure that date and time formatting (e.g. in patch files) is 
reproducible
+   [ "${TZ}" = "UTC" ] || inform "Using TZ='UTC' instead of 
TZ='${TZ-(unset)}'"
+   export TZ=UTC
 else
inform "SOURCE_DATE_EPOCH is not set"
 fi
-- 
2.43.0

ddrescue 1.28-2

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* ddrescue-1.28-2

GNU ddrescue is a data recovery tool.  It copies data from
one file or block device (hard disc, cdrom, etc) to another, trying
hard to rescue data in case of read errors.  Ddrescue does not
truncate the output file if not asked to.  So, every time you run
it on the same output file, it tries to fill in the gaps.

https://www.gnu.org/software/ddrescue/ddrescue.html

Changes since previous Cygwin package 1.28-1:
- ddrescue no longer aborts immediately if a read fails with EACCES.

This change is Cygwin-specific. It is a workaround for the following 
interesting behavior of Windows occasionally observed when USB devices 
are read:
If a read error occurs after at least one successful read, Windows 
returns ERROR_CRC(23) which is mapped to the errno EIO by Cygwin. But 
then each additional read fails with ERROR_MEDIA_CHANGED(1110) which is 
mapped to the fallback errno EACCES. This persists until the file handle 
is closed.
If a read error occurs immediately after opening the file handle, even 
the this first read fails with ERROR_MEDIA_CHANGED.


With this workaround, such devices could be processed by ddrescue if the 
option -O (--reopen-on-error) is also specified.


The source package supports reproducible builds.

Regards,
Christian

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
.

[PATCH cygport] Add repro-check command

[Christian Franke via Cygwin-apps]

This could be used to check whether a package is possibly reproducible. 
Then it could make sense to add a reasonable SOURCE_DATE_EPOCH value to 
the cygport file.


Example:

$ export SOURCE_DATE_EPOCH=$(date +%s)

$ cygport project.cygport all repro-check
...
*** Info: Build reproducibility test succeeded

$ TZ=UTC cygport project.cygport repro-check
...
*** Info: Build reproducibility test succeeded

$ unset SOURCE_DATE_EPOCH

$ cygport project.cygport repro-check
...
*** ERROR: Build reproducibility test failed

--
Regards,
Christian

From 97f518478dac722647b8a423068f2a5461c82f19 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Sun, 18 Feb 2024 16:33:07 +0100
Subject: [PATCH] Add repro-check command

This command checks for reproducibility of distribution packages.
The source package from the dist directory is unpacked to the
temp directory.  A nested rebuild of the packages is run there.
If successful, original and rebuild packages are compared and the
result is reported.
---
 README  |  1 +
 bin/cygport.in  |  8 
 lib/help.cygpart|  1 +
 lib/pkg_pkg.cygpart | 42 +-
 4 files changed, 51 insertions(+), 1 deletion(-)

diff --git a/README b/README
index fd16df6b..fec46b13 100644
--- a/README
+++ b/README
@@ -163,6 +163,7 @@ Other COMMANDs are meant primarily for maintainers:
 diff - write a patch file capturing changes to source in the working 
directory
 stage- as upload, but don't request processing of uploaded packages
 announce - compose and send a package announcement
+repro-check - check whether a rebuild produces binary identical packages
 
 The standard arguments --help or --version may also be passed to cygport.
 
diff --git a/bin/cygport.in b/bin/cygport.in
index 5fc89eaf..6acbc85b 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -29,6 +29,10 @@ set -e;
 #
 

 
+# Preserve original environment for repro-check command
+declare -r _cygport_orig_env=$(export)
+declare -r _cygport_orig_pwd=$(pwd)
+
 # for regexes, sort, etc.
 export LC_COLLATE=C
 
@@ -784,6 +788,10 @@ do
test ${PIPESTATUS[0]} -eq 0
_status=$?;
;;
+   repro-check)
+   __pkg_repro_check
+   _status=$?
+   ;;
help)
__show_help;
exit 0;
diff --git a/lib/help.cygpart b/lib/help.cygpart
index a7f30f7a..d851762e 100644
--- a/lib/help.cygpart
+++ b/lib/help.cygpart
@@ -56,6 +56,7 @@ __show_help() {
  finishdelete the working directory
  all   run prep, compile, install and package
  all-test  run prep, compile, install and package-test
+ repro-check   check whether a rebuild produces binary 
identical packages
 
See the included README file for further documentation.
 
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 756a687c..719ffcd1 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -992,6 +992,46 @@ _EOF
fi
 }
 
+__pkg_repro_check() {
+   local rc srcpkg t_cygport t_spkgdir
+
+   srcpkg=${distdir}/${PN}/${PF}-src.tar.${TAR_COMPRESSION_EXT}
+   t_spkgdir=${T}/${spkgdir##*/}
+
+   echo
+   __stage "Checking reproducibility of"
+
+   echo
+   __step "Unpacking ${srcpkg}"
+   [ -f "${srcpkg}" ] || error "Packages not built yet"
+   tar xf ${srcpkg} -C ${T} || error "tar xf ${srcpkg} -C ${T} failed"
+
+   echo
+   __step "Rebuilding in ${t_spkgdir}"
+   t_cygport="cygport ${cygportfile} finish all"
+   echo "${_cygport_orig_env}" > ${T}/.cygport_orig_env
+   __step "=== Start: ${t_cygport} ="
+
+   # Start nested cygport with original environment in temp directory
+   rc=0
+   env --chdir=${_cygport_orig_pwd} --ignore-environment /bin/bash -c \
+   "source ${T}/.cygport_orig_env && cd ${t_spkgdir} && 
${t_cygport}" \
+   || rc=$?
+
+   __step "=== Done: ${t_cygport} (exit $rc) ="
+   echo
+   [ $rc = 0 ] || error "Rebuild in ${t_spkgdir} failed"
+
+   __step "Comparing original and rebuilt packages"
+   if ! diff -qr ${distdir} ${t_spkgdir}/${PF}.${ARCH}/dist
+   then
+   echo
+   error "Build reproducibility test failed"
+   fi
+   echo
+   inform "Build reproducibility test succeeded"
+}
+
 # protect functions
 readonly -f __pkg_binpkg __pkg_diff __gpg_sign __pkg_srcpkg __pkg_dist \
-__squeeze_whitespace __tar
+__pkg_repro_check __squeeze_whitespace __tar
-- 
2.43.0

[ITP] f3 8.0

[Christian Franke via Cygwin-apps]

I would like to contribute f3. Also present in Debian, Fedora, Ubuntu, ...

SUMMARY="Test real flash memory capacity"

DESCRIPTION="f3 is a simple tool that tests flash cards capacity and
performance to see if they live up to claimed specifications.
It fills the device with pseudorandom data and then checks if it
returns the same on reading.
F3 stands for Fight Flash Fraud, or Fight Fake Flash."

List of files:

f3-8.0-1.tar.xz:
usr/bin/f3read.exe
usr/bin/f3write.exe
usr/share/doc/f3/LICENSE
usr/share/doc/f3/README.rst
usr/share/doc/f3/changelog.gz
usr/share/doc/f3/contribute.rst.gz
usr/share/doc/f3/examples/f3write.h2w
usr/share/doc/f3/examples/log-f3wr
usr/share/doc/f3/history.rst.gz
usr/share/doc/f3/usage.rst.gz
usr/share/man/man1/f3read.1.gz
usr/share/man/man1/f3write.1.gz

f3-debuginfo-8.0-1.tar.xz:
usr/lib/debug/usr/bin/f3read.exe.dbg
usr/lib/debug/usr/bin/f3write.exe.dbg
usr/src/debug/f3-8.0-1/f3read.c
usr/src/debug/f3-8.0-1/f3write.c
usr/src/debug/f3-8.0-1/libflow.c
usr/src/debug/f3-8.0-1/libflow.h
usr/src/debug/f3-8.0-1/utils.c
usr/src/debug/f3-8.0-1/utils.h

f3-8.0-1.src.tar.xz:
f3-8.0-1.src/f3-8.0.tar.gz
f3-8.0-1.src/f3.cygport

The source package supports reproducible builds.

--
Regards,
Christian

# cygport script for f3
NAME=f3
VERSION=8.0
RELEASE=1
SOURCE_DATE="2024-02-17 13:00:00 UTC"

SUMMARY="Test real flash memory capacity"

DESCRIPTION="f3 is a simple tool that tests flash cards capacity and
performance to see if they live up to claimed specifications.
It fills the device with pseudorandom data and then checks if it
returns the same on reading.  
F3 stands for Fight Flash Fraud, or Fight Fake Flash."

LICENSE="GPL-3.0-only" # or GPL-3.0-or-later ?
CATEGORY="Utils"
REQUIRES="" # libargp
BUILD_REQUIRES="binutils gcc-core gzip libargp-devel" # make

HOMEPAGE="https://fight-flash-fraud.readthedocs.io/;
SRC_URI="https://codeload.github.com/AltraMayor/f3/tar.gz/refs/tags/v${PV}#/${P}.tar.gz;

export SOURCE_DATE_EPOCH=$(date -d "$SOURCE_DATE" +%s)

src_compile() {
cd ${B}
lndirs
cygmake PREFIX=/usr \
LDFLAGS="${LDFLAGS}${LDFLAGS:+ }-Wl,--stack,400 -largp" 
}

src_install() {
cd ${B}
cyginstall PREFIX=/usr

cd ${S}
dodoc changelog doc/{contribute,history,usage}.rst
gzip -9 -n ${D}/usr/share/doc/${PN}/{changelog,*.rst}

docinto examples
dodoc f3write.h2w log-f3wr
}

[PATCH cygport] Add more checks of SOURCE_DATE_EPOCH

[Christian Franke via Cygwin-apps]

From b04c8f5e9becd6e91095e2add551f72870c9e869 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 16 Feb 2024 13:16:06 +0100
Subject: [PATCH] Add more checks of SOURCE_DATE_EPOCH

Fail if it is out of range.  Warn if it lies in the future.
Inform whether it is set or set but not exported.
---
 bin/cygport.in | 20 +++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/bin/cygport.in b/bin/cygport.in
index 5fc89eaf..3fe8a52e 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -493,14 +493,32 @@ unset restrict
 
 if [ "${SOURCE_DATE_EPOCH+y}" = "y" ]
 then
-   if [ -n "$(echo "${SOURCE_DATE_EPOCH}" | sed -e 's/^$/X/' -e 
's/[0-9]//g')" ]
+   if ! [[ "${SOURCE_DATE_EPOCH}" =~ ^[0-9]+$ ]]
then
error "SOURCE_DATE_EPOCH must be an integer number (seconds 
since the unix epoch)"
fi
+   if ! _d=$(date --iso-8601=seconds -d "@${SOURCE_DATE_EPOCH}" 
2>/dev/null)
+   then
+   error "SOURCE_DATE_EPOCH is out of range"
+   fi
+   if [ "${SOURCE_DATE_EPOCH}" -le "$(printf '%(%s)T')" ]
+   then
+   inform "Reproducible build: 
SOURCE_DATE_EPOCH='${SOURCE_DATE_EPOCH}' [$_d]"
+   else
+   warning "SOURCE_DATE_EPOCH='${SOURCE_DATE_EPOCH}' [$_d] lies in 
the future"
+   fi
+   unset _d
+   if [ -z "$(printenv SOURCE_DATE_EPOCH 2>/dev/null)" ]
+   then
+   inform "SOURCE_DATE_EPOCH is not exported to the environment"
+   fi
+
case $(peflags --version 2>/dev/null | sed -n '1s/^.* //p') in
4.6.[6-9]|4.[7-9]*|[5-9]*) ;;
*) error "SOURCE_DATE_EPOCH requires peflags 4.6.6 or later"
esac
+else
+   inform "SOURCE_DATE_EPOCH is not set"
 fi
 
 
-- 
2.43.0

Re: cygport 0.36.8-1

[Christian Franke via Cygwin]

Jon Turney via Cygwin wrote:

On 13/02/2024 13:02, Christian Franke via Cygwin wrote:

Jon Turney via Cygwin wrote:

On 12/02/2024 16:49, ASSI via Cygwin wrote:

Christian Franke via Cygwin writes:

This requires that always the same build directory is used.


Would that be solvable by using -ffile-prefix-map or is there more to
it?


That should now be used in 0.36.8, so something else leaking the 
local build directory into the package, perhaps


A closer look shows that (only) the pathnames of the assembly (*.S) 
files in cygwin1.dll.dbg now contain the build path instead of the 
mapped path:


$ strings cygwin1.dll.dbg | grep '^/.*bcopy\.S$' | uniq
/tmp/build/cygwin-3.5.0-1.x86_64/src/newlib-cygwin/winsup/cygwin/x86_64/bcopy.S 



The paths in the released package are correct:

$ strings /usr/lib/debug/usr/bin/cygwin1.dll.dbg | grep 
'^/.*bcopy\.S$' | uniq

/usr/src/debug/cygwin-3.5.0-1/winsup/cygwin/x86_64/bcopy.S

The regression was introduced by cygport commit 9e82685 in 
conjunction with the fact that --file-prefix-map has no effect on *.S 
files:


Great.  I guess that means we need to use both options.


If both --file-prefix-map and --debug-prefix-map are used in cygport, 
Cygwin 3.5.0-1 build is apparently reproducible.regardless of build path.



--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: cygport 0.36.8-1

[Christian Franke via Cygwin]

Jon Turney via Cygwin wrote:

On 13/02/2024 13:02, Christian Franke via Cygwin wrote:

Jon Turney via Cygwin wrote:

On 12/02/2024 16:49, ASSI via Cygwin wrote:

Christian Franke via Cygwin writes:

This requires that always the same build directory is used.


Would that be solvable by using -ffile-prefix-map or is there more to
it?


That should now be used in 0.36.8, so something else leaking the 
local build directory into the package, perhaps


A closer look shows that (only) the pathnames of the assembly (*.S) 
files in cygwin1.dll.dbg now contain the build path instead of the 
mapped path:


$ strings cygwin1.dll.dbg | grep '^/.*bcopy\.S$' | uniq
/tmp/build/cygwin-3.5.0-1.x86_64/src/newlib-cygwin/winsup/cygwin/x86_64/bcopy.S 



The paths in the released package are correct:

$ strings /usr/lib/debug/usr/bin/cygwin1.dll.dbg | grep 
'^/.*bcopy\.S$' | uniq

/usr/src/debug/cygwin-3.5.0-1/winsup/cygwin/x86_64/bcopy.S

The regression was introduced by cygport commit 9e82685 in 
conjunction with the fact that --file-prefix-map has no effect on *.S 
files:


Great.  I guess that means we need to use both options.



Also gcc builtin specs show that --file-prefix-map is not handled for 
asm:


$ gcc -dumpspecs | fgrep -A1 '*asm_debug:'
*asm_debug:
%{%:debug-level-gt(0):%{gstabs*:--gstabs;:%{g*:}}} 
%{fdebug-prefix-map=*:--debug-prefix-map %*}




This kind of seems like a bug.



It is fixed in current gcc test release:

$ cygcheck -f /bin/gcc
gcc-core-13.2.1+20240203-0.1

$ gcc -dumpspecs | fgrep -A1 '*asm_debug:'
*asm_debug:
%{g*:%{%:debug-level-gt(0):}} %{ffile-prefix-map=*:--debug-prefix-map 
%*} %{fdebug-prefix-map=*:--debug-prefix-map %*}



--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: cygport 0.36.8-1

[Christian Franke via Cygwin]

Jon Turney via Cygwin wrote:

On 12/02/2024 16:49, ASSI via Cygwin wrote:

Christian Franke via Cygwin writes:

This requires that always the same build directory is used.


Would that be solvable by using -ffile-prefix-map or is there more to
it?


That should now be used in 0.36.8, so something else leaking the local 
build directory into the package, perhaps


A closer look shows that (only) the pathnames of the assembly (*.S) 
files in cygwin1.dll.dbg now contain the build path instead of the 
mapped path:


$ strings cygwin1.dll.dbg | grep '^/.*bcopy\.S$' | uniq
/tmp/build/cygwin-3.5.0-1.x86_64/src/newlib-cygwin/winsup/cygwin/x86_64/bcopy.S

The paths in the released package are correct:

$ strings /usr/lib/debug/usr/bin/cygwin1.dll.dbg | grep '^/.*bcopy\.S$' 
| uniq

/usr/src/debug/cygwin-3.5.0-1/winsup/cygwin/x86_64/bcopy.S

The regression was introduced by cygport commit 9e82685 in conjunction 
with the fact that --file-prefix-map has no effect on *.S files:


Testcase:

$ cygcheck -f /bin/as
binutils-2.42-1

$ cygcheck -f /bin/gcc
gcc-core-11.4.0-1

$ pwd
/tmp/build

$ echo ret > a.S

$ echo > c.c

$ gcc -c -g --file-prefix-map=/tmp/build=/foo/bar a.S c.c

$ strings -f *.o | grep ': /' | uniq
a.o: /tmp/build
c.o: /foo/bar

$ gcc -c -g --debug-prefix-map=/tmp/build=/foo/bar a.S c.c

$ strings -f *.o | grep ': /' | uniq
a.o: /foo/bar
c.o: /foo/bar

Also gcc builtin specs show that --file-prefix-map is not handled for asm:

$ gcc -dumpspecs | fgrep -A1 '*asm_debug:'
*asm_debug:
%{%:debug-level-gt(0):%{gstabs*:--gstabs;:%{g*:}}} 
%{fdebug-prefix-map=*:--debug-prefix-map %*}


--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

busybox 1.36.1-2

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* busybox-1.36.1-2
* busybox-standalone-1.36.1-2
* busybox-doc-1.36.1-2

BusyBox combines tiny versions of many (un)common utilities into a 
single executable. It is still smaller than bash.


https://busybox.net/about.html

This is a bugfix release.

Changes since the last Cygwin package 1.36.1-1:
- Applied an upstream fix for SIGWINCH and SIGCHLD. This (at least) 
prevents that the BusyBox shell rewrites the prompt after each resize of 
the Windows console.

- Fixed various issues if 'sleep' is run from the BusyBox shell.
- Enabled function 'rev'.

The source package supports reproducible builds if used with cygport 
0.36.8-1 or later.


Regards,
Christian

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
.

Re: cygport 0.36.8-1

[Christian Franke via Cygwin]

Jon Turney wrote on cygwin-announce list:


The following packages have been uploaded to the Cygwin distribution:

* cygport-0.36.8-1

cygport is the standard method for building and maintaining packages 
for the Cygwin distribution.



...
  Add initial support for SOURCE_DATE_EPOCH


Testcase with Cygwin 3.5.0-1 src package:
- using current non-test packages including g++ 11.4.0, but
- binutils needs to be downgraded to 2.41-3 (see my related recent post 
on this list)


Then this command apparently produces reproducible packages:

  cd /usr/src/cygwin-3.5.0-1.src &&
  SOURCE_DATE_EPOCH=1704974400 cygport ./cygwin.cygport all

This requires that always the same build directory is used. I attached 
the package checksums in case someone is willing to repeat the test.


It also worked with the following of my packages, regardless of build 
directory:

busybox-1.36.1-1, ddrescue-1.28-1, smartmontools-7.4-1
(busybox and smartmontools have SOURCE_DATE_EPOCH already set in the 
cygport file)


--
Regards,
Christian

b4362a4df2688819e34555173c5f843f 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-3.5.0-1-src.hint
17c671cf6fd3f96cca4b7c411bf5c893 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-3.5.0-1-src.tar.xz
4d11453011e43fd0e97fdba04fbf 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-3.5.0-1.hint
1b97eefb0a3ee5606076dd12abe00ae1 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-3.5.0-1.tar.xz
bd8cfd72f3cffeada421debc9ac06f01 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-debuginfo/cygwin-debuginfo-3.5.0-1.hint
ca80ee415c134c691a5edc2cb5cc6507 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-debuginfo/cygwin-debuginfo-3.5.0-1.tar.xz
1fcf6f79fc6bb1e05eb7c45fcf9c157e 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-devel/cygwin-devel-3.5.0-1.hint
a5255ae6047d9d7e4a97503f1b708696 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-devel/cygwin-devel-3.5.0-1.tar.xz
b761a8949c7e843d43eb97375be53576 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-doc/cygwin-doc-3.5.0-1.hint
06ce0286241c7404a02c1d094cf7e33c 
*cygwin-3.5.0-1.x86_64/dist/cygwin/cygwin-doc/cygwin-doc-3.5.0-1.tar.xz

-- 
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Cygwin build from source is broken since binutils-2.42-1

[Christian Franke via Cygwin]

This is not a binutils regression: A build from cygwin 3.5.0-1 src 
package or current newlib-cygwin git main (bfb68a9) with new 
binutils-2.42-1 fails with:


"
In file included from ../../../../winsup/utils/dumper.cc:23:
/usr/include/bfd.h:2748:1: error: expected initializer before 
‘ATTRIBUTE_WARN_UNUSED_RESULT’

"

Downgrading to binutils 2.41-3 fixes the problem.

A diff of /usr/include/bsd.h shows that ATTRIBUTE_WARN_UNUSED_RESULT is 
a new addition in 2.42-1. This requires /usr/include/ansidecl.h. It is 
included by /usr/include/bsd.h but this has no effect because Cygwin 
source tree provides its own ansidecl.h which lacks 
ATTRIBUTE_WARN_UNUSED_RESULT:


$ g++ MANY_OPTIONS... -E ../../../../winsup/utils/dumper.cc
...
# 1 "../../../../winsup/utils/dumper.cc"
...
# 1 "../../../../winsup/../include/ansidecl.h" 1  <== #define _ANSIDECL_H
...
# 1 "/usr/include/bfd.h" 1 3 4
...
# 1 "/usr/include/ansidecl.h" 1 3 4  <== #ifndef _ANSIDECL_H
...

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

ddrescue 1.28-1

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* ddrescue-1.28-1

GNU ddrescue is a data recovery tool.  It copies data from
one file or block device (hard disc, cdrom, etc) to another, trying
hard to rescue data in case of read errors.  Ddrescue does not
truncate the output file if not asked to.  So, every time you run
it on the same output file, it tries to fill in the gaps.

https://www.gnu.org/software/ddrescue/ddrescue.html

This is an update to the latest upstream release.

Changes in version 1.28:

  * The option '--verify-on-error' has been renamed to '--check-on-error'.

  * The option '--verify-input-size' has been renamed to 
'--check-input-size'.


  * The option synonym '--exit-on-error' has been removed and is no 
longer recognized.


  * In fill and rescue modes, ddrescue now makes a final fsync call on 
outfile to prevent an early exit if the kernel caches all the writes.


  * Option '-t, --show-status' of ddrescuelog now shows the mapfile 
names at verbosity level 0 if more than one mapfile is specified.


Regards,
Christian Franke


--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
<https://cygwin.com/mailman/options/cygwin-announce>, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
<https://sourceware.org/lists.html#unsubscribe>.

Re: [PATCH cygport] Increase _FORTIFY_SOURCE level from 2 to 3 in CFLAGS

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 02/02/2024 16:13, Christian Franke via Cygwin-apps wrote:
_FORTIFY_SOURCE=3 is supported by Cygwin 3.5.0 headers and Cygwin gcc 
13.2.1 test release.


Silently falls back to level 2 if level 3 is unsupported (older 
headers or gcc) or to level 0 if unsupported at all (C++, clang).


Thanks. I applied this.

I'm thinking I want to try to do another cygport release fairly 
soonish, so please feel free to remind me about any other patches by 
you (or others) which I need to look at before then.




Possibly some initial SOURCE_DATE_EPOCH support:
https://sourceware.org/pipermail/cygwin-apps/2023-August/043108.html

Related:
https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=f5e37b9

[PATCH cygport] Increase _FORTIFY_SOURCE level from 2 to 3 in CFLAGS

[Christian Franke via Cygwin-apps]

_FORTIFY_SOURCE=3 is supported by Cygwin 3.5.0 headers and Cygwin gcc 
13.2.1 test release.


Silently falls back to level 2 if level 3 is unsupported (older headers 
or gcc) or to level 0 if unsupported at all (C++, clang).


--
Regards,
Christian

From 1a32375682d0e5ff6b80a47de70d3d9cfe0f0780 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 2 Feb 2024 17:00:18 +0100
Subject: [PATCH] Increase _FORTIFY_SOURCE level from 2 to 3 in CFLAGS

This enables buffer overflow checks if the buffer size is non-const
but known during runtime and GCC 12.0 or later is used.
---
 lib/compilers.cygpart | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/compilers.cygpart b/lib/compilers.cygpart
index 35e6fe28..52df5304 100644
--- a/lib/compilers.cygpart
+++ b/lib/compilers.cygpart
@@ -34,9 +34,9 @@ declare -x CC="gcc";
 #  Flags passed to CC when compiling C code.  Individual packages may append
 #  or override this value if they will not build correctly without it.
 #  DEFAULT VALUE
-#  -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 
-fstack-protector-strong --param=ssp-buffer-size=4
+#  -ggdb -O2 -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=3 
-fstack-protector-strong --param=ssp-buffer-size=4
 #
-declare -x CFLAGS="-ggdb -O2 -pipe -Wall -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=2 -fstack-protector-strong --param=ssp-buffer-size=4";
+declare -x CFLAGS="-ggdb -O2 -pipe -Wall -Werror=format-security 
-Wp,-D_FORTIFY_SOURCE=3 -fstack-protector-strong --param=ssp-buffer-size=4";
 
 #v* Compiling/CPPFLAGS
 #  DESCRIPTION
-- 
2.43.0

Re: Add ability to hide non-standard itoa/utoa() in stdlib.h ?

[Christian Franke via Cygwin]

Corinna Vinschen via Cygwin wrote:

On Jan 22 16:47, Christian Franke via Cygwin wrote:

Busybox does not build OOTB on Cygwin due to the addition of itoa/utoa() to
newlib in 2014:
https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=32c96dd

This is because Busybox use local functions with same name but different
signature. This does not affect build on FreeBSD, Linux (glibc, musl libc),
... because these functions simply do not exits there. For the busybox
Cygwin package, I use an ugly local hack to fix this.

itoa() is mentioned as a non-standard extension here:
https://en.wikibooks.org/wiki/C_Programming/stdlib.h/itoa
https://cplusplus.com/reference/cstdlib/itoa/

MSVC provides itoa() as a "POSIX version" of _itoa():
https://learn.microsoft.com/cpp/c-runtime-library/reference/itoa-itow

But SUS-1997... POSIX-2018 do not mention these functions:
https://pubs.opengroup.org/onlinepubs/7908799/
https://pubs.opengroup.org/onlinepubs/9699919799/

Newlib guards the prototypes with __MISC_VISIBLE in stdlib.h and
sys/features.h says:

  * __MISC_VISIBLE
  *  Extensions found in both BSD and SVr4 (shorthand for
  *  (__BSD_VISIBLE || __SVID_VISIBLE)), or newlib-specific
  *  extensions; enabled by default.

__MISC_VISIBLE is set if and only if _DEFAULT_SOURCE is set, so the comment
below should also include "... or newlib-specific extensions":

  * _DEFAULT_SOURCE (or none of the above)
  *     POSIX-1.2008 with BSD and SVr4 extensions


The above is not suitable to disable only the non-standard functions for
such use cases. Using -D_GNU_SOURCE should IMO not enable functions
unavailable on Linux. This is not the case because _GNU_SOURCE implies
_DEFAULT_SOURCE.

No patch provided for now, as I'm not yet sure how to handle this. Possibly:

- Use __MISC_VISIBLE only for (__BSD_VISIBLE || __SVID_VISIBLE) - under the
assumption that this is applicable for most cases.
- Introduce __NONSTD_VISIBLE for non-standard functions like itoa().
- Introduce _NONSTD_SOURCE to set __NONSTD_VISIBLE independent from other
_*_SOURCE defines.

Counter proposal:

First of all, Cygwin does NOT export itoa/utoa!  Only the prototypes
exist from the Cygwin POV.


Thanks, I missed that.



As Cygwin is usually the only newlib project with backward compat
issues, we have two simple choices:

- Drop itoa/utoa entirely, or
- just guard them with

 #if __MISC_VISIBLE && !defined (__CYGWIN__)

You can make a proposal like that on the newlib list and then we can
discuss it there, ok?


Done.


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

smartmontools 7.4-1

[Christian Franke]

The following packages have been uploaded to the Cygwin distribution:

* smartmontools-7.4-1

smartmontools contains utility programs (smartctl, smartd)
to control/monitor storage systems using the Self-Monitoring, Analysis
and Reporting Technology System (SMART) built into most ATA/SATA,
SCSI/SAS and NVMe hard drives and solid-state drives.  In many cases,
these utilities will provide advanced warning of disk degradation and
failure.

For a full list of changes see NEWS and ChangeLog in
/usr/share/doc/smartmontools and ticket reports at
https://www.smartmontools.org/query?milestone=Release+7.4

The cygport file is prepared for reproducible builds.

Regards,
Christian

--
 *** CYGWIN-ANNOUNCE UNSUBSCRIBE INFO ***

The easiest way to unsubscribe is to visit 
, and click 'Unsubscribe'.

If you need more information on unsubscribing, start reading here: 
.

Re: [PATCH] Cygwin: introduce close_range

[Christian Franke]

Corinna Vinschen wrote:

On Jan 15 13:07, Corinna Vinschen wrote:

Sorry Christian, but..

I was just going to push this patch when I realized that we now have
two lines of debug output per affected file descriptor:

On Jan 15 12:19, Christian Franke wrote:

+  for (unsigned int i = firstfd; i < size; i++)
+{
+  cygheap_fdget cfd ((int) i, false, false);
+  if (cfd < 0)
+   continue;
+
+  if (flags & CLOSE_RANGE_CLOEXEC)
+   {
+ syscall_printf ("set FD_CLOEXEC on fd %u", i);
+ cfd->fcntl (F_SETFD, FD_CLOEXEC);

fhandler::set_close_on_exec() already prints this:

   debug_printf ("set close_on_exec for %s to %d", get_name (), val);


+   }
+  else
+   {
+ syscall_printf ("closing fd %u", i);
+ cfd->close_with_arch ();

fhandler::close() already prints this:

   syscall_printf ("closing '%s' handle %p", get_name (), get_handle ());


I've also seen this duplication, but the drawback of the above messages 
is that the FD itself is not printed. So I decided to keep the 
syscall_printf().





Shan't we drop the syscall calls from close_range()?

  ^^^
syscall_printf


Attached.

From ca97bb30210d3c90ae94f5ff29ddd75a12a8a8b1 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 15 Jan 2024 13:36:30 +0100
Subject: [PATCH] Cygwin: introduce close_range(2)

This function closes or sets the close-on-exec flag for a specified
range of file descriptors.  It is available on FreeBSD and Linux.

Signed-off-by: Christian Franke 
---
 newlib/libc/include/sys/unistd.h   |  6 
 winsup/cygwin/cygwin.din   |  1 +
 winsup/cygwin/include/cygwin/version.h |  3 +-
 winsup/cygwin/release/3.5.0|  2 ++
 winsup/cygwin/syscalls.cc  | 38 ++
 winsup/doc/new-features.xml|  4 +++
 winsup/doc/posix.xml   |  5 
 7 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/newlib/libc/include/sys/unistd.h b/newlib/libc/include/sys/unistd.h
index 25532251c..00901540f 100644
--- a/newlib/libc/include/sys/unistd.h
+++ b/newlib/libc/include/sys/unistd.h
@@ -26,6 +26,12 @@ int chown (const char *__path, uid_t __owner, gid_t 
__group);
 int chroot (const char *__path);
 #endif
 int close (int __fildes);
+#if defined(__CYGWIN__) && (__BSD_VISIBLE || __GNU_VISIBLE)
+/* Available on FreeBSD (__BSD_VISIBLE) and Linux (__GNU_VISIBLE). */
+int close_range (unsigned int __firstfd, unsigned int __lastfd, int 
__flags);
+/*  CLOSE_RANGE_UNSHARE (1 << 1) */ /* Linux-specific, not supported. */
+#define CLOSE_RANGE_CLOEXEC (1 << 2)
+#endif
 #if __POSIX_VISIBLE >= 199209
 size_t confstr (int __name, char *__buf, size_t __len);
 #endif
diff --git a/winsup/cygwin/cygwin.din b/winsup/cygwin/cygwin.din
index 9b76ce67a..9e354acc6 100644
--- a/winsup/cygwin/cygwin.din
+++ b/winsup/cygwin/cygwin.din
@@ -347,6 +347,7 @@ clog10l NOSIGFE
 clogf NOSIGFE
 clogl NOSIGFE
 close SIGFE
+close_range SIGFE
 closedir SIGFE
 closelog SIGFE
 cnd_broadcast SIGFE
diff --git a/winsup/cygwin/include/cygwin/version.h 
b/winsup/cygwin/include/cygwin/version.h
index c8177c2b1..3036878c4 100644
--- a/winsup/cygwin/include/cygwin/version.h
+++ b/winsup/cygwin/include/cygwin/version.h
@@ -484,12 +484,13 @@ details. */
   347: Add c16rtomb, c32rtomb, mbrtoc16, mbrtoc32.
   348: Add c8rtomb, mbrtoc.
   349: Add fallocate.
+  350: Add close_range.
 
   Note that we forgot to bump the api for ualarm, strtoll, strtoull,
   sigaltstack, sethostname. */
 
 #define CYGWIN_VERSION_API_MAJOR 0
-#define CYGWIN_VERSION_API_MINOR 349
+#define CYGWIN_VERSION_API_MINOR 350
 
 /* There is also a compatibity version number associated with the shared memory
regions.  It is incremented when incompatible changes are made to the shared
diff --git a/winsup/cygwin/release/3.5.0 b/winsup/cygwin/release/3.5.0
index d0a6c2fc8..6209064a6 100644
--- a/winsup/cygwin/release/3.5.0
+++ b/winsup/cygwin/release/3.5.0
@@ -43,6 +43,8 @@ What's new:
 
 - New API calls: c8rtomb, c16rtomb, c32rtomb, mbrtoc8, mbrtoc16, mbrtoc32.
 
+- New API call: close_range (available on FreeBSD and Linux).
+
 - New API call: fallocate (Linux-specific).
 
 - Implement OSS-based sound mixer device (/dev/mixer).
diff --git a/winsup/cygwin/syscalls.cc b/winsup/cygwin/syscalls.cc
index 486db1db6..eb04067fd 100644
--- a/winsup/cygwin/syscalls.cc
+++ b/winsup/cygwin/syscalls.cc
@@ -85,6 +85,44 @@ close_all_files (bool norelease)
   cygheap->fdtab.unlock ();
 }
 
+/* Close or set the close-on-exec flag for all open file descriptors
+   from firstfd to lastfd.  CLOSE_RANGE_UNSHARE is not supported.
+   Available on FreeBSD since 13 and Linux since 5.9 */
+extern "C" int
+close_range (unsigned int firstfd, unsigned int lastfd, int flags)
+{
+  pthread_testcancel ();
+
+  if (!(firstfd <= lastfd && !(

Re: [PATCH] Cygwin: introduce close_range

[Christian Franke]

Corinna Vinschen wrote:

Hi Christian,

On Jan 15 09:56, Christian Franke wrote:

Christian Franke wrote:

Jon Turney wrote:

On 14/01/2024 16:07, Christian Franke wrote:

Recently I learned about the existence and usefulness of close_range():
https://github.com/smartmontools/smartmontools/issues/235

https://man.freebsd.org/cgi/man.cgi?query=close_range=2
https://man7.org/linux/man-pages/man2/close_range.2.html

Note that the above Linux man page is not fully correct. The
include file "linux/close_range.h" exists, but provides only the
defines. It is sufficient to include "unistd.h" as on FreeBSD.

The attached patch adds this to Cygwin. It does not implement
the Linux-specific CLOSE_RANGE_UNSHARE as I have no idea how to
do this :-)

This API should also be mentioned in the
"System interfaces compatible with GNU or Linux extensions" section
of doc/posix.xml



Thanks for the info. I used the recent "Cygwin: introduce fallocate(2)"
patch as a blueprint for which other files should be changed (fallocate
is also missing in the posix.xml file).

I will provide a new patch soon which also fixes an unlikely but
possible corner case: Pass a value larger than MAX_INT as lower limit.


Attached. I also decided to simply ignore CLOSE_RANGE_UNSHARE for now.

After reading up on this issue, I think we should not ignore
CLOSE_RANGE_UNSHARE, but quite explicitely not implement it as a valid
flag.

The whole idea behind CLOSE_RANGE_UNSHARE depends on the way the Linux
kernel creates threads and (forked) processes and the fact that it has a
wide range of ways to share parts of the execution context between
parent and child process/thread.

So on Linux, a process/thread can actually decide if they share or not
share the descriptor table with the created process/thread.  That's
what the CLONE_FILES flag to clone(2) and unshare(2) manage.

However, just as in FreeBSD, there's no such thing in Cygwin.  Threads
always share descriptors, processes never share file desriptors.

The bottom line is, I think the decision of the FreeBSD developer not to
expose the CLOSE_RANGE_UNSHARE flag at all, was the right decision.

We should not claim that we even remotely have a way of doing this
the Linux way.

Does that make sense?


Yes - new patch attached.

From 3d538a35732b1e8cd3d7e7921e06dfdffc6d28db Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 15 Jan 2024 12:13:30 +0100
Subject: [PATCH] Cygwin: introduce close_range(2)

This function closes or sets the close-on-exec flag for a specified
range of file descriptors.  It is available on FreeBSD and Linux.

Signed-off-by: Christian Franke 
---
 newlib/libc/include/sys/unistd.h   |  6 
 winsup/cygwin/cygwin.din   |  1 +
 winsup/cygwin/include/cygwin/version.h |  3 +-
 winsup/cygwin/release/3.5.0|  2 ++
 winsup/cygwin/syscalls.cc  | 42 ++
 winsup/doc/new-features.xml|  4 +++
 winsup/doc/posix.xml   |  5 +++
 7 files changed, 62 insertions(+), 1 deletion(-)

diff --git a/newlib/libc/include/sys/unistd.h b/newlib/libc/include/sys/unistd.h
index 25532251c..00901540f 100644
--- a/newlib/libc/include/sys/unistd.h
+++ b/newlib/libc/include/sys/unistd.h
@@ -26,6 +26,12 @@ int chown (const char *__path, uid_t __owner, gid_t 
__group);
 int chroot (const char *__path);
 #endif
 int close (int __fildes);
+#if defined(__CYGWIN__) && (__BSD_VISIBLE || __GNU_VISIBLE)
+/* Available on FreeBSD (__BSD_VISIBLE) and Linux (__GNU_VISIBLE). */
+int close_range (unsigned int __firstfd, unsigned int __lastfd, int 
__flags);
+/*  CLOSE_RANGE_UNSHARE (1 << 1) */ /* Linux-specific, not supported. */
+#define CLOSE_RANGE_CLOEXEC (1 << 2)
+#endif
 #if __POSIX_VISIBLE >= 199209
 size_t confstr (int __name, char *__buf, size_t __len);
 #endif
diff --git a/winsup/cygwin/cygwin.din b/winsup/cygwin/cygwin.din
index 9b76ce67a..9e354acc6 100644
--- a/winsup/cygwin/cygwin.din
+++ b/winsup/cygwin/cygwin.din
@@ -347,6 +347,7 @@ clog10l NOSIGFE
 clogf NOSIGFE
 clogl NOSIGFE
 close SIGFE
+close_range SIGFE
 closedir SIGFE
 closelog SIGFE
 cnd_broadcast SIGFE
diff --git a/winsup/cygwin/include/cygwin/version.h 
b/winsup/cygwin/include/cygwin/version.h
index c8177c2b1..3036878c4 100644
--- a/winsup/cygwin/include/cygwin/version.h
+++ b/winsup/cygwin/include/cygwin/version.h
@@ -484,12 +484,13 @@ details. */
   347: Add c16rtomb, c32rtomb, mbrtoc16, mbrtoc32.
   348: Add c8rtomb, mbrtoc.
   349: Add fallocate.
+  350: Add close_range.
 
   Note that we forgot to bump the api for ualarm, strtoll, strtoull,
   sigaltstack, sethostname. */
 
 #define CYGWIN_VERSION_API_MAJOR 0
-#define CYGWIN_VERSION_API_MINOR 349
+#define CYGWIN_VERSION_API_MINOR 350
 
 /* There is also a compatibity version number associated with the shared memory
regions.  It is incremented when incompatible changes are made to the shared
diff --git a/winsup/cyg

Re: [PATCH] Cygwin: introduce close_range

[Christian Franke]

Christian Franke wrote:

Jon Turney wrote:

On 14/01/2024 16:07, Christian Franke wrote:

Recently I learned about the existence and usefulness of close_range():
https://github.com/smartmontools/smartmontools/issues/235

https://man.freebsd.org/cgi/man.cgi?query=close_range=2
https://man7.org/linux/man-pages/man2/close_range.2.html

Note that the above Linux man page is not fully correct. The include 
file "linux/close_range.h" exists, but provides only the defines. It 
is sufficient to include "unistd.h" as on FreeBSD.


The attached patch adds this to Cygwin. It does not implement the 
Linux-specific CLOSE_RANGE_UNSHARE as I have no idea how to do this :-)


This API should also be mentioned in the
"System interfaces compatible with GNU or Linux extensions" section 
of doc/posix.xml





Thanks for the info. I used the recent "Cygwin: introduce 
fallocate(2)" patch as a blueprint for which other files should be 
changed (fallocate is also missing in the posix.xml file).


I will provide a new patch soon which also fixes an unlikely but 
possible corner case: Pass a value larger than MAX_INT as lower limit.




Attached. I also decided to simply ignore CLOSE_RANGE_UNSHARE for now.

From f7704bf77a926e53e8200528ab6abc1c9fdca511 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 15 Jan 2024 09:52:50 +0100
Subject: [PATCH] Cygwin: introduce close_range(2)

This function closes or sets the close-on-exec flag for a specified
range of file descriptors.  It is available on FreeBSD and Linux.

Signed-off-by: Christian Franke 
---
 newlib/libc/include/sys/unistd.h   |  9 ++
 winsup/cygwin/cygwin.din   |  1 +
 winsup/cygwin/include/cygwin/version.h |  3 +-
 winsup/cygwin/release/3.5.0|  2 ++
 winsup/cygwin/syscalls.cc  | 43 ++
 winsup/doc/new-features.xml|  4 +++
 winsup/doc/posix.xml   |  5 +++
 7 files changed, 66 insertions(+), 1 deletion(-)

diff --git a/newlib/libc/include/sys/unistd.h b/newlib/libc/include/sys/unistd.h
index 25532251c..0a31544ed 100644
--- a/newlib/libc/include/sys/unistd.h
+++ b/newlib/libc/include/sys/unistd.h
@@ -26,6 +26,15 @@ int chown (const char *__path, uid_t __owner, gid_t 
__group);
 int chroot (const char *__path);
 #endif
 int close (int __fildes);
+#if defined(__CYGWIN__) && (__BSD_VISIBLE || __GNU_VISIBLE)
+/* Available on FreeBSD (__BSD_VISIBLE) and Linux (__GNU_VISIBLE). */
+int close_range (unsigned int __firstfd, unsigned int __lastfd, int 
__flags);
+#if __GNU_VISIBLE
+/* Linux-specific, ignored by Cygwin. */
+#define CLOSE_RANGE_UNSHARE (1 << 1)
+#endif
+#define CLOSE_RANGE_CLOEXEC (1 << 2)
+#endif
 #if __POSIX_VISIBLE >= 199209
 size_t confstr (int __name, char *__buf, size_t __len);
 #endif
diff --git a/winsup/cygwin/cygwin.din b/winsup/cygwin/cygwin.din
index 9b76ce67a..9e354acc6 100644
--- a/winsup/cygwin/cygwin.din
+++ b/winsup/cygwin/cygwin.din
@@ -347,6 +347,7 @@ clog10l NOSIGFE
 clogf NOSIGFE
 clogl NOSIGFE
 close SIGFE
+close_range SIGFE
 closedir SIGFE
 closelog SIGFE
 cnd_broadcast SIGFE
diff --git a/winsup/cygwin/include/cygwin/version.h 
b/winsup/cygwin/include/cygwin/version.h
index c8177c2b1..3036878c4 100644
--- a/winsup/cygwin/include/cygwin/version.h
+++ b/winsup/cygwin/include/cygwin/version.h
@@ -484,12 +484,13 @@ details. */
   347: Add c16rtomb, c32rtomb, mbrtoc16, mbrtoc32.
   348: Add c8rtomb, mbrtoc.
   349: Add fallocate.
+  350: Add close_range.
 
   Note that we forgot to bump the api for ualarm, strtoll, strtoull,
   sigaltstack, sethostname. */
 
 #define CYGWIN_VERSION_API_MAJOR 0
-#define CYGWIN_VERSION_API_MINOR 349
+#define CYGWIN_VERSION_API_MINOR 350
 
 /* There is also a compatibity version number associated with the shared memory
regions.  It is incremented when incompatible changes are made to the shared
diff --git a/winsup/cygwin/release/3.5.0 b/winsup/cygwin/release/3.5.0
index d0a6c2fc8..6209064a6 100644
--- a/winsup/cygwin/release/3.5.0
+++ b/winsup/cygwin/release/3.5.0
@@ -43,6 +43,8 @@ What's new:
 
 - New API calls: c8rtomb, c16rtomb, c32rtomb, mbrtoc8, mbrtoc16, mbrtoc32.
 
+- New API call: close_range (available on FreeBSD and Linux).
+
 - New API call: fallocate (Linux-specific).
 
 - Implement OSS-based sound mixer device (/dev/mixer).
diff --git a/winsup/cygwin/syscalls.cc b/winsup/cygwin/syscalls.cc
index 486db1db6..2e1b56b7f 100644
--- a/winsup/cygwin/syscalls.cc
+++ b/winsup/cygwin/syscalls.cc
@@ -85,6 +85,49 @@ close_all_files (bool norelease)
   cygheap->fdtab.unlock ();
 }
 
+/* Close or set the close-on-exec flag for all open file descriptors
+   from firstfd to lastfd.  CLOSE_RANGE_UNSHARE is ignored.
+   Available on FreeBSD since 13 and Linux since 5.9 */
+extern "C" int
+close_range (unsigned int firstfd, unsigned int lastfd, int flags)
+{
+  pthread_testcancel ();
+
+  if (!(firstfd <= lastfd &&
+

Re: [PATCH] Cygwin: introduce close_range

[Christian Franke]

Jon Turney wrote:

On 14/01/2024 16:07, Christian Franke wrote:

Recently I learned about the existence and usefulness of close_range():
https://github.com/smartmontools/smartmontools/issues/235

https://man.freebsd.org/cgi/man.cgi?query=close_range=2
https://man7.org/linux/man-pages/man2/close_range.2.html

Note that the above Linux man page is not fully correct. The include 
file "linux/close_range.h" exists, but provides only the defines. It 
is sufficient to include "unistd.h" as on FreeBSD.


The attached patch adds this to Cygwin. It does not implement the 
Linux-specific CLOSE_RANGE_UNSHARE as I have no idea how to do this :-)


This API should also be mentioned in the
"System interfaces compatible with GNU or Linux extensions" section of 
doc/posix.xml





Thanks for the info. I used the recent "Cygwin: introduce fallocate(2)" 
patch as a blueprint for which other files should be changed (fallocate 
is also missing in the posix.xml file).


I will provide a new patch soon which also fixes an unlikely but 
possible corner case: Pass a value larger than MAX_INT as lower limit.

[PATCH] Cygwin: introduce close_range

[Christian Franke]

Recently I learned about the existence and usefulness of close_range():
https://github.com/smartmontools/smartmontools/issues/235

https://man.freebsd.org/cgi/man.cgi?query=close_range=2
https://man7.org/linux/man-pages/man2/close_range.2.html

Note that the above Linux man page is not fully correct. The include 
file "linux/close_range.h" exists, but provides only the defines. It is 
sufficient to include "unistd.h" as on FreeBSD.


The attached patch adds this to Cygwin. It does not implement the 
Linux-specific CLOSE_RANGE_UNSHARE as I have no idea how to do this :-)


--
Regards,
Christian

From 2393e82a62e19e29e61ef3253e227c19ae7222eb Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Sun, 14 Jan 2024 16:54:17 +0100
Subject: [PATCH] Cygwin: introduce close_range

This function closes or sets the close-on-exec flag for a specified
range of file descriptors.  It is available on FreeBSD and Linux.

Signed-off-by: Christian Franke 
---
 newlib/libc/include/sys/unistd.h   |  6 
 winsup/cygwin/cygwin.din   |  1 +
 winsup/cygwin/include/cygwin/version.h |  3 +-
 winsup/cygwin/release/3.5.0|  2 ++
 winsup/cygwin/syscalls.cc  | 42 ++
 winsup/doc/new-features.xml|  4 +++
 6 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/newlib/libc/include/sys/unistd.h b/newlib/libc/include/sys/unistd.h
index 25532251c..f5f1963c8 100644
--- a/newlib/libc/include/sys/unistd.h
+++ b/newlib/libc/include/sys/unistd.h
@@ -26,6 +26,12 @@ int chown (const char *__path, uid_t __owner, gid_t 
__group);
 int chroot (const char *__path);
 #endif
 int close (int __fildes);
+#if defined(__CYGWIN__) && (__BSD_VISIBLE || __GNU_VISIBLE)
+/* Available on FreeBSD (__BSD_VISIBLE) and Linux (__GNU_VISIBLE). */
+int close_range (unsigned int __firstfd, unsigned int __lastfd, int 
__flags);
+/*  CLOSE_RANGE_UNSHARE 0x01 */ /* Linux only, not yet supported. */
+#define CLOSE_RANGE_CLOEXEC 0x02
+#endif
 #if __POSIX_VISIBLE >= 199209
 size_t confstr (int __name, char *__buf, size_t __len);
 #endif
diff --git a/winsup/cygwin/cygwin.din b/winsup/cygwin/cygwin.din
index 9b76ce67a..9e354acc6 100644
--- a/winsup/cygwin/cygwin.din
+++ b/winsup/cygwin/cygwin.din
@@ -347,6 +347,7 @@ clog10l NOSIGFE
 clogf NOSIGFE
 clogl NOSIGFE
 close SIGFE
+close_range SIGFE
 closedir SIGFE
 closelog SIGFE
 cnd_broadcast SIGFE
diff --git a/winsup/cygwin/include/cygwin/version.h 
b/winsup/cygwin/include/cygwin/version.h
index c8177c2b1..3036878c4 100644
--- a/winsup/cygwin/include/cygwin/version.h
+++ b/winsup/cygwin/include/cygwin/version.h
@@ -484,12 +484,13 @@ details. */
   347: Add c16rtomb, c32rtomb, mbrtoc16, mbrtoc32.
   348: Add c8rtomb, mbrtoc.
   349: Add fallocate.
+  350: Add close_range.
 
   Note that we forgot to bump the api for ualarm, strtoll, strtoull,
   sigaltstack, sethostname. */
 
 #define CYGWIN_VERSION_API_MAJOR 0
-#define CYGWIN_VERSION_API_MINOR 349
+#define CYGWIN_VERSION_API_MINOR 350
 
 /* There is also a compatibity version number associated with the shared memory
regions.  It is incremented when incompatible changes are made to the shared
diff --git a/winsup/cygwin/release/3.5.0 b/winsup/cygwin/release/3.5.0
index d0a6c2fc8..6209064a6 100644
--- a/winsup/cygwin/release/3.5.0
+++ b/winsup/cygwin/release/3.5.0
@@ -43,6 +43,8 @@ What's new:
 
 - New API calls: c8rtomb, c16rtomb, c32rtomb, mbrtoc8, mbrtoc16, mbrtoc32.
 
+- New API call: close_range (available on FreeBSD and Linux).
+
 - New API call: fallocate (Linux-specific).
 
 - Implement OSS-based sound mixer device (/dev/mixer).
diff --git a/winsup/cygwin/syscalls.cc b/winsup/cygwin/syscalls.cc
index 486db1db6..aba864475 100644
--- a/winsup/cygwin/syscalls.cc
+++ b/winsup/cygwin/syscalls.cc
@@ -85,6 +85,48 @@ close_all_files (bool norelease)
   cygheap->fdtab.unlock ();
 }
 
+/* Close or set the close-on-exec flag for all open file descriptors
+   from firstfd to lastfd.
+   Available on FreeBSD since 13 and Linux since 5.9 */
+extern "C" int
+close_range (unsigned int firstfd, unsigned int lastfd, int flags)
+{
+  pthread_testcancel ();
+
+  if (!(firstfd <= lastfd && (!flags || flags == CLOSE_RANGE_CLOEXEC)))
+{
+  set_errno (EINVAL);
+  return -1;
+}
+
+  cygheap->fdtab.lock ();
+
+  int size = (int) (lastfd < cygheap->fdtab.size ? lastfd + 1 :
+   cygheap->fdtab.size);
+
+  for (int i = (int) firstfd; i < size; i++)
+{
+  cygheap_fdget cfd (i, false, false);
+  if (cfd < 0)
+   continue;
+
+  if (!flags)
+   {
+ syscall_printf ("closing fd %d", i);
+ cfd->close_with_arch ();
+ cfd.release ();
+   }
+  else /* CLOSE_RANGE_CLOEXEC */
+   {
+ syscall_printf ("set FD_CLOEXEC on fd %d", i);
+ cfd->fcntl (F_SETFD, FD_CLOEXEC);
+   }
+}
+
+  cygheap->fdtab.un

Re: where is parted?

[Christian Franke via Cygwin]

matth...@gmx.li wrote:

fdisk reports the same partition type as sfdisk. It report "Microsoft basic 
data" for NTFS as well
as for FAT32 partitions.


That is as expected and differs from MBR disks. The same GPT partition 
GUID is used for NTFS and the various FAT filesystem types.

https://en.wikipedia.org/wiki/Microsoft_basic_data_partition


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: where is parted?

[Christian Franke via Cygwin]

Matthias--- via Cygwin wrote:

I'm using sfdisk for analysing partitions on msdos partition tables. 
Unfortunately it don't support
GPT tables.
Is there another tool, like parted, what can be used?


/sbin/fdisk from package util-linux-2.33.1-2 supports GPT.

--
Regards,
Christian


--
Problem reports:  https://cygwin.com/problems.html
FAQ:  https://cygwin.com/faq/
Documentation:https://cygwin.com/docs.html
Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Hi Corinna,

Corinna Vinschen wrote:

Hi Christian,


On second thought...

I had a bad night tonight and was thinking a long time about this and
that.  It suddenly occured to me that there might be another problem
with this approach, attaching ordinals to the label name.

Assuming you have a single filesystem labled "VOLUME" which is on a
fixed disk.  So you get something like this:

   $ ls -l /dev/disk/by-label
   total 0
   lrwxrwxrwx 1 corinna vinschen 0 Nov 22 10:09  VOLUME -> ../../sdb1
   lrwxrwxrwx 1 corinna vinschen 0 Nov 22 10:10  root -> ../../sda3

Now you insert an USB Stick with a FAT32 filesystem, also labeled
"VOLUME".  Now you get something like this:

   $ ls -l /dev/disk/by-label
   total 0
   lrwxrwxrwx 1 corinna vinschen 0 Nov 22 10:12 'VOLUME#0' -> ../../sdb1
   lrwxrwxrwx 1 corinna vinschen 0 Nov 22 10:12 'VOLUME#1' -> ../../sdc1
   lrwxrwxrwx 1 corinna vinschen 0 Nov 22 10:10  root -> ../../sda3

So the label name changes, depending on inserting or removing another
partition.


This is intentional. If the first duplicate appears, it is IMO better to 
also replace the original name to show that a duplicate exists.





Not saying I have a good solution myself, so I wonder if we should just
let it slip, but I thought we should at least talk about it...


Users should be aware that unspecific label names like VOLUME could not 
be used as a persistent link if drives are changed.


Same may apply to by-partuuid names as preformatted SD-cards and USB 
flash drives may have a null MBR serial number.


Regards,
Christian

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

Hi Christian,

Looks good, but I just realized that I was already wondering about the
sanitization and forgot to talk about it:

On Nov 21 12:24, Christian Franke wrote:

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index c5d72816f..d12ac52fa 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -64,10 +64,12 @@ sanitize_label_string (WCHAR *s)
/* Linux does not skip leading spaces. */
return sanitize_string (s, L'\0', L' ', L'_', [] (WCHAR c) -> bool
  {
-  /* Labels may contain characters not allowed in filenames.
-Linux replaces spaces with \x20 which is not an option here. */
+  /* Labels may contain characters not allowed in filenames.  Also

Apart from slash and backslash, we don't have this problem in Cygwin,
usually.  Even control characters are no problem.  All chars not allowed
in filenames are just transposed into the Unicode private use area, as
per strfuncs.cc, line 20ff on the way to storage, and back when reading
the names from storage.  This, and especially in a virtual filesystem
like /proc, there's no reason to avoid these characters.


Thanks for clarification.





+ replace '#' to avoid that duplicate markers introduce new
+duplicates.  Linux replaces spaces with \x20 which is not an
+option here. */
return !((0 <= c && c <= L' ') || c == L':' || c == L'/' || c == L'\\'
- || c == L'"');
+ || c == L'#' || c == L'"');

If you really want to avoid chars not allowed in DOS filenames, the
list seems incomplete, missing '<', '>', '?', '*', '|'.

But as I said, there's really no reason for that.  I simply reduced the
above expression to

   return !(c == L'/' || c == L'\\' || c == L'#');

and created a disk label

   test"foo*bar?baz:"

It works nicely, including stuff like

   $ ls *\"*
   $ ls *\**

So, I can push it as is, or we just allow everything and the kitchen sink
as per the reduced filter expression.  What do you prefer?


The latter - patch attached.

Christian

From ecc54356adbe7768bd5fd5561c78c67cd5725183 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 21 Nov 2023 19:28:02 +0100
Subject: [PATCH] Cygwin: /dev/disk: Append '#N' if the same name appears more
 than once

No longer drop ranges of identical link names.  Append '#0, #1, ...'
to each name instead.  Enhance charset allowed in label names.
No longer ignore null volume serial numbers.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc | 54 ++
 1 file changed, 33 insertions(+), 21 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index c5d72816f..29af9de95 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -64,10 +64,11 @@ sanitize_label_string (WCHAR *s)
   /* Linux does not skip leading spaces. */
   return sanitize_string (s, L'\0', L' ', L'_', [] (WCHAR c) -> bool
 {
-  /* Labels may contain characters not allowed in filenames.
-Linux replaces spaces with \x20 which is not an option here. */
-  return !((0 <= c && c <= L' ') || c == L':' || c == L'/' || c == L'\\'
- || c == L'"');
+  /* Labels may contain characters not allowed in filenames.  Also
+ replace '#' to avoid that duplicate markers introduce new
+duplicates.  Linux replaces spaces with \x20 which is not an
+option here. */
+  return !(c == L'/' || c == L'\\' || c == L'#');
 }
   );
 }
@@ -304,8 +305,7 @@ partition_to_label_or_uuid(bool uuid, const UNICODE_STRING 
*drive_uname,
   const NTFS_VOLUME_DATA_BUFFER *nvdb =
 reinterpret_cast(ioctl_buf);
   if (uuid && DeviceIoControl (volhdl, FSCTL_GET_NTFS_VOLUME_DATA, nullptr, 0,
-  ioctl_buf, NT_MAX_PATH, _read, nullptr)
-  && nvdb->VolumeSerialNumber.QuadPart)
+  ioctl_buf, NT_MAX_PATH, _read, nullptr))
 {
   /* Print without any separator as on Linux. */
   __small_sprintf (name, "%016X", nvdb->VolumeSerialNumber.QuadPart);
@@ -327,13 +327,9 @@ partition_to_label_or_uuid(bool uuid, const UNICODE_STRING 
*drive_uname,
   FILE_FS_VOLUME_INFORMATION *ffvi =
 reinterpret_cast(ioctl_buf);
   if (uuid)
-{
-  if (!ffvi->VolumeSerialNumber)
-   return false;
-  /* Print with separator as on Linux. */
-  __small_sprintf (name, "%04x-%04x", ffvi->VolumeSerialNumber >> 16,
-  ffvi->VolumeSerialNumber & 0x);
-}
+/* Print with separator as on Linux. */
+__small_sprintf (name, "%04x-%04x", ffvi->VolumeSerialNumber >> 16,
+ffvi->VolumeSerialNumber & 0x);
   else
 {
   /* La

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Christian Franke wrote:

Corinna Vinschen wrote:

On Nov 20 10:40, Corinna Vinschen wrote:

Hi Christian,

This puzzles me:

On Nov 17 21:25, Christian Franke wrote:
@@ -610,7 +607,7 @@ get_by_id_table (by_id_entry * , 
fhandler_dev_disk::dev_disk_location loc)

    if (!table)
  return (errno_set ? -1 : 0);
  -  /* Sort by name and remove duplicates. */
+  /* Sort by name and mark duplicates. */
    qsort (table, table_size, sizeof (*table), by_id_compare_name);
    for (unsigned i = 0; i < table_size; i++)

by_id_compare_name only compars the actual names...


  {
@@ -619,12 +616,13 @@ get_by_id_table (by_id_entry * , 
fhandler_dev_disk::dev_disk_location loc)

  j++;
    if (j == i + 1)
  continue;
-  /* Duplicate(s) found, remove all entries with this name. */
-  debug_printf ("removing duplicates %d-%d: '%s'", i, j - 1, 
table[i].name);

-  if (j < table_size)
-    memmove (table + i, table + j, (table_size - j) * sizeof 
(*table));

-  table_size -= j - i;
-  i--;
+  /* Duplicate(s) found, append "#N" to all entries. This never

...but the names are identical.  So the *order* within the identically
named entries depends on qsort's reshuffling of table
entries.  Which in turn depends on outside factors like number of table
entries and the ultimate position of the identical entries within the
ordered table.

Having said that, I don't see how adding ordinals to the names can be
unambiguous.  AFAICS, the numbers may change by just adding another
disk (USB Stick) to the system...

Oops, that's not exactly what I was trying to say, sorry.

The problem is not adding ordinals to the name, AFAICS, the problem is
that the sorting function by_id_compare_name is not up to the task to
make sure the order is unambiguous within the entries of identical name.


That's correct, thanks for catching. qsort is not a stable sort. 
Changing drives outside the duplicate range may also change the order 
within the range. Could be fixed by a lexicographic compare of {name, 
drive, part}.




Attached.

From 7dafb85210ef77ea8798f22160f7782c394ef5c3 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 21 Nov 2023 12:17:14 +0100
Subject: [PATCH] Cygwin: /dev/disk: Append '#N' if the same name appears more
 than once

No longer drop ranges of identical link names.  Append '#0, #1, ...'
to each name instead.  No longer ignore null volume serial numbers.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc | 53 +++---
 1 file changed, 33 insertions(+), 20 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index c5d72816f..d12ac52fa 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -64,10 +64,12 @@ sanitize_label_string (WCHAR *s)
   /* Linux does not skip leading spaces. */
   return sanitize_string (s, L'\0', L' ', L'_', [] (WCHAR c) -> bool
 {
-  /* Labels may contain characters not allowed in filenames.
-Linux replaces spaces with \x20 which is not an option here. */
+  /* Labels may contain characters not allowed in filenames.  Also
+ replace '#' to avoid that duplicate markers introduce new
+duplicates.  Linux replaces spaces with \x20 which is not an
+option here. */
   return !((0 <= c && c <= L' ') || c == L':' || c == L'/' || c == L'\\'
- || c == L'"');
+ || c == L'#' || c == L'"');
 }
   );
 }
@@ -304,8 +306,7 @@ partition_to_label_or_uuid(bool uuid, const UNICODE_STRING 
*drive_uname,
   const NTFS_VOLUME_DATA_BUFFER *nvdb =
 reinterpret_cast(ioctl_buf);
   if (uuid && DeviceIoControl (volhdl, FSCTL_GET_NTFS_VOLUME_DATA, nullptr, 0,
-  ioctl_buf, NT_MAX_PATH, _read, nullptr)
-  && nvdb->VolumeSerialNumber.QuadPart)
+  ioctl_buf, NT_MAX_PATH, _read, nullptr))
 {
   /* Print without any separator as on Linux. */
   __small_sprintf (name, "%016X", nvdb->VolumeSerialNumber.QuadPart);
@@ -327,13 +328,9 @@ partition_to_label_or_uuid(bool uuid, const UNICODE_STRING 
*drive_uname,
   FILE_FS_VOLUME_INFORMATION *ffvi =
 reinterpret_cast(ioctl_buf);
   if (uuid)
-{
-  if (!ffvi->VolumeSerialNumber)
-   return false;
-  /* Print with separator as on Linux. */
-  __small_sprintf (name, "%04x-%04x", ffvi->VolumeSerialNumber >> 16,
-  ffvi->VolumeSerialNumber & 0x);
-}
+/* Print with separator as on Linux. */
+__small_sprintf (name, "%04x-%04x", ffvi->VolumeSerialNumber >> 16,
+ffvi->VolumeSerialNumber & 0x);
   else
 {
   /* Label is not null terminated. */
@@ -361,6 +358,20 @@ by_id_compare_name (const void *a, const void *b)
   return strcmp (ap->name, bp->

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov 20 10:40, Corinna Vinschen wrote:

Hi Christian,

This puzzles me:

On Nov 17 21:25, Christian Franke wrote:

@@ -610,7 +607,7 @@ get_by_id_table (by_id_entry * , 
fhandler_dev_disk::dev_disk_location loc)
if (!table)
  return (errno_set ? -1 : 0);
  
-  /* Sort by name and remove duplicates. */

+  /* Sort by name and mark duplicates. */
qsort (table, table_size, sizeof (*table), by_id_compare_name);
for (unsigned i = 0; i < table_size; i++)

by_id_compare_name only compars the actual names...


  {
@@ -619,12 +616,13 @@ get_by_id_table (by_id_entry * , 
fhandler_dev_disk::dev_disk_location loc)
j++;
if (j == i + 1)
continue;
-  /* Duplicate(s) found, remove all entries with this name. */
-  debug_printf ("removing duplicates %d-%d: '%s'", i, j - 1, 
table[i].name);
-  if (j < table_size)
-   memmove (table + i, table + j, (table_size - j) * sizeof (*table));
-  table_size -= j - i;
-  i--;
+  /* Duplicate(s) found, append "#N" to all entries.  This never

...but the names are identical.  So the *order* within the identically
named entries depends on qsort's reshuffling of table
entries.  Which in turn depends on outside factors like number of table
entries and the ultimate position of the identical entries within the
ordered table.

Having said that, I don't see how adding ordinals to the names can be
unambiguous.  AFAICS, the numbers may change by just adding another
disk (USB Stick) to the system...

Oops, that's not exactly what I was trying to say, sorry.

The problem is not adding ordinals to the name, AFAICS, the problem is
that the sorting function by_id_compare_name is not up to the task to
make sure the order is unambiguous within the entries of identical name.


That's correct, thanks for catching. qsort is not a stable sort. 
Changing drives outside the duplicate range may also change the order 
within the range. Could be fixed by a lexicographic compare of {name, 
drive, part}.


I'll provide a new patch soon. For now, I attached the unrelated but 
important part of the dropped patch.


Christian

From 64c8a0eb3ead91de36ee65516483e40cf7f49446 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Mon, 20 Nov 2023 15:40:42 +0100
Subject: [PATCH] Cygwin: /dev/disk/by-uuid: Fix NTFS serial number print
 format

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index 016b4c7bc..c5d72816f 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -308,7 +308,7 @@ partition_to_label_or_uuid(bool uuid, const UNICODE_STRING 
*drive_uname,
   && nvdb->VolumeSerialNumber.QuadPart)
 {
   /* Print without any separator as on Linux. */
-  __small_sprintf (name, "%16X", nvdb->VolumeSerialNumber.QuadPart);
+  __small_sprintf (name, "%016X", nvdb->VolumeSerialNumber.QuadPart);
   NtClose(volhdl);
   return true;
 }
-- 
2.42.1

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov 17 18:53, Christian Franke wrote:

Corinna Vinschen wrote:

...
I see.  Admittedly, I don't know how Linux handles this either.

A quick test on Debian 12 with by-label suggests that the last duplicate
wins. Also not very sophisticated :-)

Given this is all controlled by rather simple udev rules, see
/usr/lib/udev/rules.d/60-persistent-storage.rules, that's not
really surprising.


IIRC in the past I've seen in another of these directories (by-id?) that
'#N' was appended if duplicates occur.

I don't see anything like that in 60-persistent-storage.rules, though.
It has been removed at one point, it seems.


A quick followup to mark duplicates and (more important) avoid leading 
spaces in NTFS serial number.


From 1e212814cfb17579105ead4835410bd74dfb188a Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 17 Nov 2023 21:20:26 +0100
Subject: [PATCH] Cygwin: /dev/disk: Mark duplicates, fix serial number format

Keep ranges of duplicate names and append '#N' to each name.
Add missing leading zeros to output format of NTFS serial number.
No longer ignore null volume serial numbers.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc | 38 ++
 1 file changed, 18 insertions(+), 20 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index 016b4c7bc..131ffd935 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -64,10 +64,12 @@ sanitize_label_string (WCHAR *s)
   /* Linux does not skip leading spaces. */
   return sanitize_string (s, L'\0', L' ', L'_', [] (WCHAR c) -> bool
 {
-  /* Labels may contain characters not allowed in filenames.
-Linux replaces spaces with \x20 which is not an option here. */
+  /* Labels may contain characters not allowed in filenames.  Also
+ replace '#' to avoid that duplicate markers introduce new
+duplicates.  Linux replaces spaces with \x20 which is not an
+option here. */
   return !((0 <= c && c <= L' ') || c == L':' || c == L'/' || c == L'\\'
- || c == L'"');
+ || c == L'#' || c == L'"');
 }
   );
 }
@@ -304,11 +306,10 @@ partition_to_label_or_uuid(bool uuid, const 
UNICODE_STRING *drive_uname,
   const NTFS_VOLUME_DATA_BUFFER *nvdb =
 reinterpret_cast(ioctl_buf);
   if (uuid && DeviceIoControl (volhdl, FSCTL_GET_NTFS_VOLUME_DATA, nullptr, 0,
-  ioctl_buf, NT_MAX_PATH, _read, nullptr)
-  && nvdb->VolumeSerialNumber.QuadPart)
+  ioctl_buf, NT_MAX_PATH, _read, nullptr))
 {
   /* Print without any separator as on Linux. */
-  __small_sprintf (name, "%16X", nvdb->VolumeSerialNumber.QuadPart);
+  __small_sprintf (name, "%016X", nvdb->VolumeSerialNumber.QuadPart);
   NtClose(volhdl);
   return true;
 }
@@ -327,13 +328,9 @@ partition_to_label_or_uuid(bool uuid, const UNICODE_STRING 
*drive_uname,
   FILE_FS_VOLUME_INFORMATION *ffvi =
 reinterpret_cast(ioctl_buf);
   if (uuid)
-{
-  if (!ffvi->VolumeSerialNumber)
-   return false;
-  /* Print with separator as on Linux. */
-  __small_sprintf (name, "%04x-%04x", ffvi->VolumeSerialNumber >> 16,
-  ffvi->VolumeSerialNumber & 0x);
-}
+/* Print with separator as on Linux. */
+__small_sprintf (name, "%04x-%04x", ffvi->VolumeSerialNumber >> 16,
+ffvi->VolumeSerialNumber & 0x);
   else
 {
   /* Label is not null terminated. */
@@ -610,7 +607,7 @@ get_by_id_table (by_id_entry * , 
fhandler_dev_disk::dev_disk_location loc)
   if (!table)
 return (errno_set ? -1 : 0);
 
-  /* Sort by name and remove duplicates. */
+  /* Sort by name and mark duplicates. */
   qsort (table, table_size, sizeof (*table), by_id_compare_name);
   for (unsigned i = 0; i < table_size; i++)
 {
@@ -619,12 +616,13 @@ get_by_id_table (by_id_entry * , 
fhandler_dev_disk::dev_disk_location loc)
j++;
   if (j == i + 1)
continue;
-  /* Duplicate(s) found, remove all entries with this name. */
-  debug_printf ("removing duplicates %d-%d: '%s'", i, j - 1, 
table[i].name);
-  if (j < table_size)
-   memmove (table + i, table + j, (table_size - j) * sizeof (*table));
-  table_size -= j - i;
-  i--;
+  /* Duplicate(s) found, append "#N" to all entries.  This never
+introduces new duplicates because '#' never occurs in the
+original names. */
+  debug_printf ("mark duplicates %u-%u of '%s'", i, j - 1, table[i].name);
+  size_t len = strlen (table[i].name);
+  for (unsigned k = i; k < j; k++)
+   __small_sprintf (table[k].name + len, "#%u", k - i);
 }
 
   debug_printf ("table_size: %d", table_size);
-- 
2.42.1

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov 17 17:45, Christian Franke wrote:

Corinna Vinschen wrote:

On Nov 17 15:39, Christian Franke wrote:

The last two /dev/disk subdirectories :-)

Note a minor difference: On Linux, empty /dev/disk subdirectories apparently
never appear. A subdirectory is not listed in /dev/disk if it would be
empty. Not worth the effort to emulate.

Agreed.  This is really great.  I just pushed your patch.

However, there's something strange in terms of by-label:

I have two partitions with labels:

$ ls -l /dev/disk/by-label
total 0
lrwxrwxrwx 1 corinna vinschen 0 Nov 17 17:18 blub -> ../../sda3
lrwxrwxrwx 1 corinna vinschen 0 Nov 17 17:18 blub2 -> ../../sdb2
$

Now I change the label of sdb2 to the same "blub" string as on sda3:

$ ls -l /dev/disk/by-label
total 0
$

I'd expected to see only one, due to the name collision, but en empty
dir is a bit surprising...  And it may occur more often than not, given
that the default label "New_Volume" probably won't get changed very
often.


This is intentional and inherited from the very first patch, see the loop
behind qsort(). If a range of identical names appear, all these entries are
removed. If some "random" entry would be kept, it might no longer be the
persistent link the user expects. We could possibly add some hash like done
for by-id or append a number in such cases later. Need some more time to
thing about it

I see.  Admittedly, I don't know how Linux handles this either.


A quick test on Debian 12 with by-label suggests that the last duplicate 
wins. Also not very sophisticated :-)
IIRC in the past I've seen in another of these directories (by-id?) that 
'#N' was appended if duplicates occur.




I will sent a patch for the new-features doc soon.


Attached.

Christian

From 5d1c82f08e4b37d65acc8bf0c76f75641d762667 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 17 Nov 2023 18:41:08 +0100
Subject: [PATCH] Cygwin: Document /dev/disk/by-* subdirectories

Signed-off-by: Christian Franke 
---
 winsup/cygwin/release/3.5.0 | 16 +++-
 winsup/doc/new-features.xml | 20 +++-
 2 files changed, 22 insertions(+), 14 deletions(-)

diff --git a/winsup/cygwin/release/3.5.0 b/winsup/cygwin/release/3.5.0
index 2d59818b5..aee21c972 100644
--- a/winsup/cygwin/release/3.5.0
+++ b/winsup/cygwin/release/3.5.0
@@ -17,11 +17,17 @@ What's new:
   class expressions, and collating symbols in the search pattern, i.e.,
   [:alnum:], [=a=], [.aa.].
 
-- Introduce /dev/disk directory with subdirectories by-id and by-partuuid.
-  The by-id directory provides symlinks for each disk and its partitions:
-  BUSTYPE-[VENDOR_]PRODUCT_[SERIAL|HASH][-partN] -> ../../sdX[N].
-  The by-partuuid directory provides symlinks for each MBR and GPT disk
-  partition: MBR_SERIAL-OFFSET -> ../../sdXN, GPT_GUID -> ../../sdXN.
+- Introduce /dev/disk directory with various by-* subdirectories which
+  provide symlinks to disk and partition raw devices:
+  by-drive/DRIVE_LETTER ->  ../../sdXN
+  by-label/VOLUME_LABEL ->  ../../sdXN
+  by-id/BUSTYPE-[VENDOR_]PRODUCT_[SERIAL|0xHASH][-partN] -> ../../sdX[N]
+  by-partuuid/MBR_SERIAL-OFFSET -> ../../sdXN
+  by-partuuid/GPT_GUID -> ../../sdXN
+  by-uuid/VOLUME_SERIAL -> ../../sdXN
+  by-voluuid/MBR_SERIAL-OFFSET -> ../../sdXN
+  by-voluuid/VOLUME_GUID -> ../../sdXN
+  The subdirectories by-drive and by-voluuid are Cygwin specific.
 
 - Introduce /proc/codesets and /proc/locales with information on
   supported codesets and locales for all interested parties.  Locale(1)
diff --git a/winsup/doc/new-features.xml b/winsup/doc/new-features.xml
index a8e8a7991..2c31a4acc 100644
--- a/winsup/doc/new-features.xml
+++ b/winsup/doc/new-features.xml
@@ -35,17 +35,19 @@ class expressions, and collating symbols in the search 
pattern, i.e.,
 
 
 
-Introduce /dev/disk directory with subdirectories by-id and by-partuuid.
-The by-id directory provides symlinks for each disk and its partitions:
+Introduce /dev/disk directory with various by-* subdirectories which
+provide symlinks to disk and partition raw devices:
   
-  BUSTYPE-[VENDOR_]PRODUCT_[SERIAL|0xHASH][-partN] -> ../../sdX[N]
-  
-The by-partuuid directory provides symlinks for each MBR and GPT disk
-partition:
-  
-  MBR_SERIAL-OFFSET -> ../../sdXN
-  GPT_GUID -> ../../sdXN
+  by-drive/DRIVE_LETTER ->  ../../sdXN
+  by-label/VOLUME_LABEL ->  ../../sdXN
+  by-id/BUSTYPE-[VENDOR_]PRODUCT_[SERIAL|0xHASH][-partN] -> ../../sdX[N]
+  by-partuuid/MBR_SERIAL-OFFSET -> ../../sdXN
+  by-partuuid/GPT_GUID -> ../../sdXN
+  by-uuid/VOLUME_SERIAL -> ../../sdXN
+  by-voluuid/MBR_SERIAL-OFFSET -> ../../sdXN
+  by-voluuid/VOLUME_GUID -> ../../sdXN
   
+The subdirectories by-drive and by-voluuid are Cygwin specific.
 
 
 
-- 
2.42.1

Re: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov 17 15:39, Christian Franke wrote:

The last two /dev/disk subdirectories :-)

Note a minor difference: On Linux, empty /dev/disk subdirectories apparently
never appear. A subdirectory is not listed in /dev/disk if it would be
empty. Not worth the effort to emulate.

Agreed.  This is really great.  I just pushed your patch.

However, there's something strange in terms of by-label:

I have two partitions with labels:

   $ ls -l /dev/disk/by-label
   total 0
   lrwxrwxrwx 1 corinna vinschen 0 Nov 17 17:18 blub -> ../../sda3
   lrwxrwxrwx 1 corinna vinschen 0 Nov 17 17:18 blub2 -> ../../sdb2
   $

Now I change the label of sdb2 to the same "blub" string as on sda3:

   $ ls -l /dev/disk/by-label
   total 0
   $

I'd expected to see only one, due to the name collision, but en empty
dir is a bit surprising...  And it may occur more often than not, given
that the default label "New_Volume" probably won't get changed very
often.



This is intentional and inherited from the very first patch, see the 
loop behind qsort(). If a range of identical names appear, all these 
entries are removed. If some "random" entry would be kept, it might no 
longer be the persistent link the user expects. We could possibly add 
some hash like done for by-id or append a number in such cases later. 
Need some more time to thing about it


I will sent a patch for the new-features doc soon.

Christian

[PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

[Christian Franke]

The last two /dev/disk subdirectories :-)

Note a minor difference: On Linux, empty /dev/disk subdirectories 
apparently never appear. A subdirectory is not listed in /dev/disk if it 
would be empty. Not worth the effort to emulate.


--
Regards,
Christian

From 27f4a10431aacbdc9725571f4ad34732081fb63a Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Fri, 17 Nov 2023 15:22:52 +0100
Subject: [PATCH] Cygwin: Add /dev/disk/by-label and /dev/disk/by-uuid symlinks

The new directories '/dev/disk/by-label' and '/dev/disk/by-uuid'
provide symlinks for each disk related volume label and serial
number:
'VOLUME_LABEL' -> '../../sdXN'
'VOLUME_SERIAL' -> '../../sdXN'

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc  | 159 +---
 winsup/cygwin/local_includes/fhandler.h |   3 +-
 2 files changed, 141 insertions(+), 21 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index 5f79ab5e9..016b4c7bc 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -14,33 +14,62 @@ details. */
 #include 
 #include 
 
-/* Replace spaces, non-printing and unexpected characters.  Remove
-   leading and trailing spaces.  Return remaining string length. */
+/* Replace invalid characters.  Optionally remove leading and trailing
+   characters.  Return remaining string length. */
+template 
 static int
-sanitize_id_string (char *s)
+sanitize_string (char_type *s, char_type leading, char_type trailing,
+char_type replace, func_type valid)
 {
   int first = 0;
-  while (s[first] == ' ')
-first++;
-  int last = -1, i;
+  if (leading)
+while (s[first] == leading)
+  first++;
+  int len = -1, i;
   for (i = 0; s[first + i]; i++)
 {
-  char c = s[first + i];
-  if (c != ' ')
-   last = -1;
-  else if (last < 0)
-   last = i;
-  if (!(('0' <= c && c <= '9') || c == '.' || c == '-'
- || ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z')))
-   c = '_';
+  char_type c = s[first + i];
+  if (c != trailing)
+   len = -1;
+  else if (len < 0)
+   len = i;
+  if (!valid (c))
+   c = replace;
   else if (!first)
continue;
   s[i] = c;
 }
-  if (last < 0)
-last = i;
-  s[last] = '\0';
-  return last;
+  if (len < 0)
+len = i;
+  s[len] = (char_type) 0;
+  return len;
+}
+
+/* Variant for device identify strings. */
+static int
+sanitize_id_string (char *s)
+{
+  return sanitize_string (s, ' ', ' ', '_', [] (char c) -> bool
+{
+  return (('0' <= c && c <= '9') || c == '.' || c == '-'
+ || ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z'));
+}
+  );
+}
+
+/* Variant for volume labels. */
+static int
+sanitize_label_string (WCHAR *s)
+{
+  /* Linux does not skip leading spaces. */
+  return sanitize_string (s, L'\0', L' ', L'_', [] (WCHAR c) -> bool
+{
+  /* Labels may contain characters not allowed in filenames.
+Linux replaces spaces with \x20 which is not an option here. */
+  return !((0 <= c && c <= L' ') || c == L':' || c == L'/' || c == L'\\'
+ || c == L'"');
+}
+  );
 }
 
 /* Fetch storage properties and create the ID string.
@@ -244,6 +273,79 @@ partition_to_voluuid(const UNICODE_STRING *drive_uname, 
DWORD part_num,
   return true;
 }
 
+/* ("HarddiskN", PART_NUM) -> "VOLUME_LABEL" or "VOLUME_SERIAL" */
+static bool
+partition_to_label_or_uuid(bool uuid, const UNICODE_STRING *drive_uname,
+  DWORD part_num, char *ioctl_buf, char *name)
+{
+  WCHAR wpath[MAX_PATH];
+  /* Trailing backslash is required. */
+  size_t len = __small_swprintf (wpath, L"\\Device\\%S\\Partition%u\\",
+drive_uname, part_num);
+  len *= sizeof (WCHAR);
+  UNICODE_STRING upath = {(USHORT) len, (USHORT) (len + 1), wpath};
+  OBJECT_ATTRIBUTES attr;
+  InitializeObjectAttributes (, , OBJ_CASE_INSENSITIVE, nullptr,
+ nullptr);
+  IO_STATUS_BLOCK io;
+  HANDLE volhdl;
+  NTSTATUS status = NtOpenFile (, READ_CONTROL, , ,
+   FILE_SHARE_VALID_FLAGS, 0);
+  if (!NT_SUCCESS (status))
+{
+  /* Fails with STATUS_UNRECOGNIZED_VOLUME (0xC14F) if the
+ partition/filesystem type is unsupported. */
+  debug_printf ("NtOpenFile(%S), status %y", upath, status);
+  return false;
+}
+
+  /* Check for possible 64-bit NTFS serial number first. */
+  DWORD bytes_read;
+  const NTFS_VOLUME_DATA_BUFFER *nvdb =
+reinterpret_cast(ioctl_buf);
+  if (uuid && DeviceIoControl (volhdl, FSCTL_GET_NTFS_VOLUME_DATA, nullptr, 0,
+  ioctl_buf, NT_MAX_PATH, _read, nullptr)
+  && nvdb->VolumeSerialNumber.QuadPart)
+{
+   

Re: [PATCH] Cygwin: Add /dev/disk/by-drive and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov 16 12:50, Christian Franke wrote:
...

I also tried an NTFS partition and the output looks like this:

0FD4F62866CFBF09 -> ../../sdc1

This is the 64 bit volume serial number as returned by
DeviceIoControl(FSCTL_GET_NTFS_VOLUME_DATA)(*).

Wouldn't that be what we want to see, too?

Hmm.. yes. Should both information be provided in by-uuid or only the
serial numbers? In the latter case, should we add e.g. by-voluuid for the
volume GUIDs ?

Good question... by-voluuid sounds like a nice idea.  It's a Windows-only
concept anyway, so it might make sense to present it in its own directory.


Then by-voluuid is easy, changed patch is attached. I try to provide a 
patch for a new by-uuid with filesystem serial numbers soon.


From 39cdbadeacfb226ad082f4f635285298aa5ad1bb Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Thu, 16 Nov 2023 17:51:08 +0100
Subject: [PATCH] Cygwin: Add /dev/disk/by-drive and /dev/disk/by-voluuid
 symlinks

The new directory '/dev/disk/by-drive' provides symlinks for each
disk related drive letter:
'x' -> '../../sdXN'
The new directory '/dev/disk/by-voluuid' provides symlinks for each
disk related storage volume:
'MBR_SERIAL-OFFSET' -> '../../sdXN'
'VOLUME_GUID' -> '../../sdXN'
Both directories provide Windows specific information and do not
exist on Linux.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc  | 143 
 winsup/cygwin/local_includes/fhandler.h |   3 +-
 2 files changed, 126 insertions(+), 20 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index 11b24042f..5f79ab5e9 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -158,6 +158,92 @@ storprop_to_id_name (HANDLE devhdl, const UNICODE_STRING 
*upath,
   return 1;
 }
 
+/* ("HarddiskN", PART_NUM) -> "?\\Volume{GUID}\\" */
+static bool
+partition_to_volpath (const UNICODE_STRING *drive_uname, DWORD part_num,
+ WCHAR (& volpath)[MAX_PATH])
+{
+  WCHAR gpath[MAX_PATH];
+  __small_swprintf (gpath, L"?\\GLOBALROOT\\Device\\%S\\Partition%u\\",
+   drive_uname, part_num);
+  if (!GetVolumeNameForVolumeMountPointW (gpath, volpath, sizeof(volpath)))
+{
+  debug_printf ("GetVolumeNameForVolumeMountPointW(%W): %E", gpath);
+  return false;
+}
+  debug_printf ("%W -> %W", gpath, volpath);
+  return true;
+}
+
+/* ("HarddiskN", PART_NUM) -> "x" */
+static bool
+partition_to_drive(const UNICODE_STRING *drive_uname, DWORD part_num,
+  WCHAR *w_buf, char *name)
+{
+  WCHAR volpath[MAX_PATH];
+  if (!partition_to_volpath (drive_uname, part_num, volpath))
+return false;
+
+  DWORD len;
+  if (!GetVolumePathNamesForVolumeNameW (volpath, w_buf, NT_MAX_PATH, ))
+{
+  debug_printf ("GetVolumePathNamesForVolumeNameW(%W): %E", volpath);
+  return false;
+}
+  debug_printf ("%W -> '%W'%s", volpath, w_buf,
+   (w_buf[0] && wcschr (w_buf, L'\0')[1] ? ", ..." : ""));
+
+  /* Find first "X:\\", skip if not found.
+ FIXME: Support multiple drive letters. */
+  WCHAR *p;
+  for (p = w_buf; ; p = wcschr (p, L'\0') + 1)
+{
+   if (!*p)
+ return false;
+   if (L'A' <= p[0] && p[0] <= L'Z' && p[1] == L':' && p[2] == L'\\' && 
!p[3])
+ break;
+}
+  name[0] = (p[0] - L'A') + 'a';
+  name[1] = '\0';
+  return true;
+}
+
+/* ("HarddiskN", PART_NUM) -> "VOLUME_GUID" */
+static bool
+partition_to_voluuid(const UNICODE_STRING *drive_uname, DWORD part_num,
+char *name)
+{
+  WCHAR volpath[MAX_PATH];
+  if (!partition_to_volpath (drive_uname, part_num, volpath))
+return false;
+
+  /* Skip if not "?\\Volume{GUID}...". */
+  static const WCHAR prefix[] = L"?\\Volume{";
+  const size_t prefix_len = sizeof (prefix) / sizeof(WCHAR) - 1, uuid_len = 36;
+  if (!(!wcsncmp (volpath, prefix, prefix_len)
+  && volpath[prefix_len + uuid_len] == L'}'))
+return false;
+
+  /* Extract GUID. */
+  volpath[prefix_len + uuid_len] = 0;
+  __small_sprintf (name, "%W", volpath + prefix_len);
+
+  if (!strncmp (name + 9, "--00", 12) && !strcmp (name + 32, ""))
+{
+  /* MBR "GUID": Use same SERIAL-OFFSET format as in by-partuuid directory.
+ SERIAL---009a-78563412 -> SERIAL-123456789a00 */
+  for (int i = 9, j = 30; i < 19; i += 2, j -= 2)
+   {
+ if (j == 22) // name[j + 1] == '-'
+   j--;
+ name[i] = name[j];
+ name[i + 1] = name[j + 1];
+   }
+  name[21] = '\0';
+}
+  return true;
+}
+
 struct by_id_entry
 {
   char name[NAME_MAX + 1

Re: [PATCH] Cygwin: Add /dev/disk/by-drive and /dev/disk/by-uuid symlinks

[Christian Franke]

Corinna Vinschen wrote:

Hi Christian,

On Nov 15 18:23, Christian Franke wrote:

This is the next (and possibly last for now) extension to the /dev/disk
directory. Limited to disk related entries which allowed a straightforward
extension of the existing code.

My original idea was to add also other drive letters and volume GUIDs. Too
complex for now.

Interestingly the volume GUID (by-uuid) for partitions on MBR disks is
sometimes identical to the partition "GUID" (by-partuuid), sometimes (always
for C:?) not. With GPT disks, both GUIDs are possibly always identical.

That looks great, but in terms of by-uuid, I'm not sure it's the
right thing to do.  On Linux I have a vfat partition (/boot/efi).
The uuid in /dev/disk/by-uuid is the volume serial number, just
with an extra dash, i.e.

   057A-B3A7 -> ../../sda1

That's what you get for FAT/FAT32/exFAT.


What is the best way to retrieve a FAT* serial? There is 
GetVolumeInformation{ByHandleW}(), but this may not work with the NT 
Layer pathnames / handles used here. In Cygwin tree, 
GetVolumeInformation only appears in cygcheck.cc and very old ChangeLogs.




I also tried an NTFS partition and the output looks like this:

   0FD4F62866CFBF09 -> ../../sdc1

This is the 64 bit volume serial number as returned by
DeviceIoControl(FSCTL_GET_NTFS_VOLUME_DATA)(*).

Wouldn't that be what we want to see, too?


Hmm.. yes. Should both information be provided in by-uuid or only 
the serial numbers? In the latter case, should we add e.g. by-voluuid 
for the volume GUIDs ?


Christian

[PATCH] Cygwin: Add /dev/disk/by-drive and /dev/disk/by-uuid symlinks

[Christian Franke]

This is the next (and possibly last for now) extension to the /dev/disk 
directory. Limited to disk related entries which allowed a 
straightforward extension of the existing code.


My original idea was to add also other drive letters and volume GUIDs. 
Too complex for now.


Interestingly the volume GUID (by-uuid) for partitions on MBR disks is 
sometimes identical to the partition "GUID" (by-partuuid), sometimes 
(always for C:?) not. With GPT disks, both GUIDs are possibly always 
identical.


I will provide a related new-features doc patch later.

--
Regards,
Christian

From 1dc3d3a8378ab2aeff766ea2d211750079fd27f8 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Wed, 15 Nov 2023 17:54:18 +0100
Subject: [PATCH] Cygwin: Add /dev/disk/by-drive and /dev/disk/by-uuid symlinks

The new directory '/dev/disk/by-drive' provides symlinks for each
disk related drive letter:
'x' -> '../../sdXN'
The new directory '/dev/disk/by-uuid' provides symlinks for each
disk related storage volume:
'MBR_SERIAL-OFFSET' -> '../../sdXN'
'VOLUME_GUID' -> '../../sdXN'

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc  | 143 
 winsup/cygwin/local_includes/fhandler.h |   3 +-
 2 files changed, 126 insertions(+), 20 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index 11b24042f..977e8a64f 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -158,6 +158,92 @@ storprop_to_id_name (HANDLE devhdl, const UNICODE_STRING 
*upath,
   return 1;
 }
 
+/* ("HarddiskN", PART_NUM) -> "?\\Volume{GUID}\\" */
+static bool
+partition_to_volpath (const UNICODE_STRING *drive_uname, DWORD part_num,
+ WCHAR (& volpath)[MAX_PATH])
+{
+  WCHAR gpath[MAX_PATH];
+  __small_swprintf (gpath, L"?\\GLOBALROOT\\Device\\%S\\Partition%u\\",
+   drive_uname, part_num);
+  if (!GetVolumeNameForVolumeMountPointW (gpath, volpath, sizeof(volpath)))
+{
+  debug_printf ("GetVolumeNameForVolumeMountPointW(%W): %E", gpath);
+  return false;
+}
+  debug_printf ("%W -> %W", gpath, volpath);
+  return true;
+}
+
+/* ("HarddiskN", PART_NUM) -> "x" */
+static bool
+partition_to_drive(const UNICODE_STRING *drive_uname, DWORD part_num,
+  WCHAR *w_buf, char *name)
+{
+  WCHAR volpath[MAX_PATH];
+  if (!partition_to_volpath (drive_uname, part_num, volpath))
+return false;
+
+  DWORD len;
+  if (!GetVolumePathNamesForVolumeNameW (volpath, w_buf, NT_MAX_PATH, ))
+{
+  debug_printf ("GetVolumePathNamesForVolumeNameW(%W): %E", volpath);
+  return false;
+}
+  debug_printf ("%W -> '%W'%s", volpath, w_buf,
+   (w_buf[0] && wcschr (w_buf, L'\0')[1] ? ", ..." : ""));
+
+  /* Find first "X:\\", skip if not found.
+ FIXME: Support multiple drive letters. */
+  WCHAR *p;
+  for (p = w_buf; ; p = wcschr (p, L'\0') + 1)
+{
+   if (!*p)
+ return false;
+   if (L'A' <= p[0] && p[0] <= L'Z' && p[1] == L':' && p[2] == L'\\' && 
!p[3])
+ break;
+}
+  name[0] = (p[0] - L'A') + 'a';
+  name[1] = '\0';
+  return true;
+}
+
+/* ("HarddiskN", PART_NUM) -> "VOLUME_GUID" */
+static bool
+partition_to_voluuid(const UNICODE_STRING *drive_uname, DWORD part_num,
+char *name)
+{
+  WCHAR volpath[MAX_PATH];
+  if (!partition_to_volpath (drive_uname, part_num, volpath))
+return false;
+
+  /* Skip if not "?\\Volume{GUID}...". */
+  static const WCHAR prefix[] = L"?\\Volume{";
+  const size_t prefix_len = sizeof (prefix) / sizeof(WCHAR) - 1, uuid_len = 36;
+  if (!(!wcsncmp (volpath, prefix, prefix_len)
+  && volpath[prefix_len + uuid_len] == L'}'))
+return false;
+
+  /* Extract GUID. */
+  volpath[prefix_len + uuid_len] = 0;
+  __small_sprintf (name, "%W", volpath + prefix_len);
+
+  if (!strncmp (name + 9, "--00", 12) && !strcmp (name + 32, ""))
+{
+  /* MBR "GUID": Use same SERIAL-OFFSET format as in by-partuuid directory.
+ SERIAL---009a-78563412 -> SERIAL-123456789a00 */
+  for (int i = 9, j = 30; i < 19; i += 2, j -= 2)
+   {
+ if (j == 22) // name[j + 1] == '-'
+   j--;
+ name[i] = name[j];
+ name[i + 1] = name[j + 1];
+   }
+  name[21] = '\0';
+}
+  return true;
+}
+
 struct by_id_entry
 {
   char name[NAME_MAX + 1];
@@ -208,6 +294,7 @@ format_partuuid (char *name, const PARTITION_INFORMATION_EX 
*pix)
guid->Data1, guid->Data2, guid->Data3,
guid->Data4[0], guid->Data4[1], guid->Data4[2], 
guid->Data4[3],
  

[PATCH] Cygwin: /dev/disk/by-id: Remove leading spaces from identify fields

[Christian Franke]

Minor improvement, avoids "/dev/disk/by-id/sata-VENDOR_MODEL___SERIAL".

--
Regards,
Christian

From 9927cd46435da97d234b0c7e97b5fc2a9b7632d1 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Wed, 8 Nov 2023 17:03:57 +0100
Subject: [PATCH] Cygwin: /dev/disk/by-id: Remove leading spaces from identify
 fields

Various drives align the serial number to the right of the fixed
length field.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc | 34 ++
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index fcd0de651..11b24042f 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -14,27 +14,33 @@ details. */
 #include 
 #include 
 
-/* Replace non-printing and unexpected characters, remove trailing spaces,
-   return remaining string length. */
+/* Replace spaces, non-printing and unexpected characters.  Remove
+   leading and trailing spaces.  Return remaining string length. */
 static int
 sanitize_id_string (char *s)
 {
-  int lastspace = -1, i;
-  for (i = 0; s[i]; i++)
+  int first = 0;
+  while (s[first] == ' ')
+first++;
+  int last = -1, i;
+  for (i = 0; s[first + i]; i++)
 {
-  char c = s[i];
+  char c = s[first + i];
   if (c != ' ')
-   lastspace = -1;
-  else if (lastspace < 0)
-   lastspace = i;
-  if (('0' <= c && c <= '9') || c == '.' || c == '-'
- || ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z'))
+   last = -1;
+  else if (last < 0)
+   last = i;
+  if (!(('0' <= c && c <= '9') || c == '.' || c == '-'
+ || ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z')))
+   c = '_';
+  else if (!first)
continue;
-  s[i] = '_';
+  s[i] = c;
 }
-  if (lastspace >= 0)
-s[(i = lastspace)] = '\0';
-  return i;
+  if (last < 0)
+last = i;
+  s[last] = '\0';
+  return last;
 }
 
 /* Fetch storage properties and create the ID string.
-- 
2.42.1

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

..
Looking forward to it. We'll just need an entry for the release text
in winsup/cygwin/release/3.5.0 and doc/new-features.xml in the end :)


Attached for now as implementing the remaining subdirs is not yet 
scheduled. Docbook formatting not tested.


Christian

From b07de21461207a2b57465d3dd8f7db2b36d886c0 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 7 Nov 2023 15:25:54 +0100
Subject: [PATCH] Cygwin: Document /dev/disk/by-id and /dev/disk/by-partuuid

Signed-off-by: Christian Franke 
---
 winsup/cygwin/release/3.5.0 |  6 ++
 winsup/doc/new-features.xml | 14 ++
 2 files changed, 20 insertions(+)

diff --git a/winsup/cygwin/release/3.5.0 b/winsup/cygwin/release/3.5.0
index dbbf8009d..2d59818b5 100644
--- a/winsup/cygwin/release/3.5.0
+++ b/winsup/cygwin/release/3.5.0
@@ -17,6 +17,12 @@ What's new:
   class expressions, and collating symbols in the search pattern, i.e.,
   [:alnum:], [=a=], [.aa.].
 
+- Introduce /dev/disk directory with subdirectories by-id and by-partuuid.
+  The by-id directory provides symlinks for each disk and its partitions:
+  BUSTYPE-[VENDOR_]PRODUCT_[SERIAL|HASH][-partN] -> ../../sdX[N].
+  The by-partuuid directory provides symlinks for each MBR and GPT disk
+  partition: MBR_SERIAL-OFFSET -> ../../sdXN, GPT_GUID -> ../../sdXN.
+
 - Introduce /proc/codesets and /proc/locales with information on
   supported codesets and locales for all interested parties.  Locale(1)
   opens these files and uses the info for printing locale info like any
diff --git a/winsup/doc/new-features.xml b/winsup/doc/new-features.xml
index 78b2dbafd..a8e8a7991 100644
--- a/winsup/doc/new-features.xml
+++ b/winsup/doc/new-features.xml
@@ -34,6 +34,20 @@ class expressions, and collating symbols in the search 
pattern, i.e.,
 [:alnum:], [=a=], [.aa.].
 
 
+
+Introduce /dev/disk directory with subdirectories by-id and by-partuuid.
+The by-id directory provides symlinks for each disk and its partitions:
+  
+  BUSTYPE-[VENDOR_]PRODUCT_[SERIAL|0xHASH][-partN] -> ../../sdX[N]
+  
+The by-partuuid directory provides symlinks for each MBR and GPT disk
+partition:
+  
+  MBR_SERIAL-OFFSET -> ../../sdXN
+  GPT_GUID -> ../../sdXN
+  
+
+
 
 Introduce /proc/codesets and /proc/locales with information on supported
 codesets and locales for all interested parties.  Locale(1) opens these
-- 
2.42.1

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

Hi Christian,

On Nov  5 16:45, Christian Franke wrote:

...
Old IOCTL dropped and code simplified.

Great.  I pushed your patch.


Thanks.



...

Last, but not least, do you see a chance to add any other /dev/disk
subdir?  by-partuuid, perhaps?

Possibly, but not very soon. I'm not yet sure which API functions could be
used.
Some early draft ideas:

/dev/disk/by-partuid (Partition UUID -> device)
   GPT_PART_UUID -> ../../sdXN (GPT partition UUID)
   MBR_SERIAL-partN -> ../../sdYM (Fake UUID for MBR)

That should only require IOCTL_DISK_GET_PARTITION_INFO_EX, I think.


Easier than expected: DRIVE_LAYOUT_INFORMATION_EX already contains 
PARTITION_INFORMATION_EX so existing scanning function could be 
enhanced. Patch attached.






/dev/disk/by-uuid (Windows Volume UUID -> device)
   Vol_UUID1 -> ../../sdXN  (disk volume)
   Vol_UUID2 -> ../../scd0  (CD/DVD drive volume)
   Vol_UUID3 -> /proc/sys/GLOBAL??/Volume{UUID}  (others, e.g. VeraCrypt
volume)

Yeah, tricky. These are not the partition GUIDs but the filesystem
GUIDs or serial numbers.  AFAICS, Windows filesystems (FAT*, NTFS)
don't maintain a filesystem GUID, as, e. g., ext4 or xfs, but only
serial numbers you can fetch via NtQueryVolumeInformationFile.
A Linux example of that is the serial number from a FAT32 filesytem
as the EFI boot partition in by-uuid:

   lrwxrwxrwx 1 root root 10 Oct 30 10:20 DC38-0407 -> ../../sda1

On second thought, maybe that's sufficient for our by-uuid emulation.


/dev/disk/by-drive (Cygwin specific: drive letter -> volume)
   c -> ../by-uuid/UUID (if UUID available)
   x -> /proc/sys/DosDevices/X: (others, e.g. Network, "mounted" Volume
Shadow Copy)

Ah, good idea. That's what my extension in /proc/partition already
provides, but a /dev/disk/by-drive sounds like a great idea.


Left for later :-)

Christian

From e9c9d2a1a1df9ddecd815300c62321a480f0de9b Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Tue, 7 Nov 2023 10:57:15 +0100
Subject: [PATCH] Cygwin: Add /dev/disk/by-partuuid symlinks

The new directory '/dev/disk/by-partuuid' provides symlinks for each
MBR or GPT disk partition:
'MBR_SERIAL-OFFSET' -> '../../sdXN'
'GPT_GUID' -> '../../sdXN'

Signed-off-by: Christian Franke 
---
 winsup/cygwin/fhandler/dev_disk.cc  | 173 
 winsup/cygwin/local_includes/fhandler.h |   7 +-
 2 files changed, 125 insertions(+), 55 deletions(-)

diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index 9a1cae5eb..fcd0de651 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -176,9 +176,38 @@ by_id_realloc (by_id_entry *p, size_t n)
   return reinterpret_cast(p2);
 }
 
+static bool
+format_partuuid (char *name, const PARTITION_INFORMATION_EX *pix)
+{
+  const GUID *guid;
+  switch (pix->PartitionStyle)
+{
+  case PARTITION_STYLE_MBR: guid = >Mbr.PartitionId; break;
+  case PARTITION_STYLE_GPT: guid = >Gpt.PartitionId; break;
+  default: return false;
+}
+
+  if (pix->PartitionStyle == PARTITION_STYLE_MBR && !guid->Data2
+  && !guid->Data3 && !guid->Data4[6] && !guid->Data4[7])
+  /* MBR "GUID": SERIAL---00NN-NNN0
+Byte offset in LE order --^^^
+Print as: SERIAL-OFFSET */
+__small_sprintf(name,
+   "%08_x-%02_x%02_x%02_x%02_x%02_x%02_x",
+   guid->Data1, guid->Data4[5], guid->Data4[4], guid->Data4[3],
+   guid->Data4[2], guid->Data4[1], guid->Data4[0]);
+  else
+__small_sprintf(name,
+   
"%08_x-%04_x-%04_x-%02_x%02_x-%02_x%02_x%02_x%02_x%02_x%02_x",
+   guid->Data1, guid->Data2, guid->Data3,
+   guid->Data4[0], guid->Data4[1], guid->Data4[2], 
guid->Data4[3],
+   guid->Data4[4], guid->Data4[5], guid->Data4[6], 
guid->Data4[7]);
+   return true;
+}
+
 /* Create sorted name -> drive mapping table. Must be freed by caller. */
 static int
-get_by_id_table (by_id_entry * )
+get_by_id_table (by_id_entry * , fhandler_dev_disk::dev_disk_location 
loc)
 {
   table = nullptr;
 
@@ -282,25 +311,31 @@ get_by_id_table (by_id_entry * )
}
}
 
- /* Fetch storage properties and create the ID string. */
- int rc = storprop_to_id_name (devhdl, , ioctl_buf,
-   table[table_size].name);
- if (rc <= 0)
+ const char *drive_name = "";
+ if (loc == fhandler_dev_disk::disk_by_id)
{
- if (rc < 0)
-   errno_set = true;
- continue;
+ /* Fetch storage properties and create the ID string. */
+ int rc = storprop_to_id_name (devhdl, , ioctl_buf,

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Hi Corinna,

Corinna Vinschen wrote:

Hi Christian,

patch looks pretty good to me.  Just two minor points:


+  /* Traverse \Device directory ... */
+  bool errno_set = false;
+  HANDLE devhdl = INVALID_HANDLE_VALUE;

INVALID_HANDLE_VALUE is a Win32 API concept and is only returned by
CreateFile and friends.  The native NT API doesn't know
INVALID_HANDLE_VALUE and, fortunately, only uses NULL.  I think it's
puzzeling to use INVALID_HANDLE_VALUE in a native NT context. So, would
you mind to just use NULL (or nullptr) instead?  As in...


+  if (devhdl != INVALID_HANDLE_VALUE)

  if (devhdl)

Sounds easier to me :)


Yes...done.



+ /* Fetch drive layout info to get size of all partitions on disk. */
+ DWORD part_cnt = 0;
+ PARTITION_INFORMATION_EX *pix = nullptr;
+ PARTITION_INFORMATION *pi = nullptr;
+ DWORD bytes_read;
+ if (DeviceIoControl (devhdl, IOCTL_DISK_GET_DRIVE_LAYOUT_EX, nullptr, 
0,
+  ioctl_buf, NT_MAX_PATH, _read, nullptr))
+   {
+ DRIVE_LAYOUT_INFORMATION_EX *pdlix =
+ reinterpret_cast(ioctl_buf);
+ part_cnt = pdlix->PartitionCount;
+ pix = pdlix->PartitionEntry;
+   }
+ else if (DeviceIoControl (devhdl, IOCTL_DISK_GET_DRIVE_LAYOUT, 
nullptr,

Do we really still need IOCTL_DISK_GET_DRIVE_LAYOUT w/o _EX?


Possibly not. I borrowed this code from code behind /proc/partitions.



fhandler_proc still uses it, too, but the EX variation is available
since XP. I can't imagine that a fallback to the non-EX-version is still
necessary.  If you like you could drop it immediately, or we can drop
both usages as a followup patch.


Old IOCTL dropped and code simplified.



Last, but not least, do you see a chance to add any other /dev/disk
subdir?  by-partuuid, perhaps?


Possibly, but not very soon. I'm not yet sure which API functions could 
be used.

Some early draft ideas:

/dev/disk/by-partuid (Partition UUID -> device)
  GPT_PART_UUID -> ../../sdXN (GPT partition UUID)
  MBR_SERIAL-partN -> ../../sdYM (Fake UUID for MBR)

/dev/disk/by-uuid (Windows Volume UUID -> device)
  Vol_UUID1 -> ../../sdXN  (disk volume)
  Vol_UUID2 -> ../../scd0  (CD/DVD drive volume)
  Vol_UUID3 -> /proc/sys/GLOBAL??/Volume{UUID}  (others, e.g. VeraCrypt 
volume)


/dev/disk/by-drive (Cygwin specific: drive letter -> volume)
  c -> ../by-uuid/UUID (if UUID available)
  x -> /proc/sys/DosDevices/X: (others, e.g. Network, "mounted" Volume 
Shadow Copy)


Christian

From aa8c35e041ffe5a4b06104c122d9ba1fdc492683 Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Sun, 5 Nov 2023 15:54:23 +0100
Subject: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

The new directory '/dev/disk/by-id' provides symlinks for each
disk and its partitions:
'BUSTYPE-[VENDOR_]PRODUCT_SERIAL[-partN]' -> '../../sdX[N]'.
This is based on strings provided by STORAGE_DEVICE_DESCRIPTOR.
If this information is too short, a 128-bit hash of the
STORAGE_DEVICE_UNIQUE_IDENTIFIER raw data is added.
Administrator privileges are not required.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/Makefile.am   |   1 +
 winsup/cygwin/devices.in|   4 +
 winsup/cygwin/dtable.cc |   3 +
 winsup/cygwin/fhandler/dev_disk.cc  | 621 
 winsup/cygwin/local_includes/devices.h  |   6 +-
 winsup/cygwin/local_includes/fhandler.h |  47 ++
 winsup/cygwin/mount.cc  |  10 +
 7 files changed, 691 insertions(+), 1 deletion(-)
 create mode 100644 winsup/cygwin/fhandler/dev_disk.cc

diff --git a/winsup/cygwin/Makefile.am b/winsup/cygwin/Makefile.am
index 64b252a22..376c79fc3 100644
--- a/winsup/cygwin/Makefile.am
+++ b/winsup/cygwin/Makefile.am
@@ -84,6 +84,7 @@ FHANDLER_FILES= \
fhandler/console.cc \
fhandler/cygdrive.cc \
fhandler/dev.cc \
+   fhandler/dev_disk.cc \
fhandler/dev_fd.cc \
fhandler/disk_file.cc \
fhandler/dsp.cc \
diff --git a/winsup/cygwin/devices.in b/winsup/cygwin/devices.in
index 2545dd85e..48d3843fe 100644
--- a/winsup/cygwin/devices.in
+++ b/winsup/cygwin/devices.in
@@ -115,6 +115,9 @@ const _device dev_cygdrive_storage =
 const _device dev_fs_storage =
   {"", {FH_FS}, "", exists};
 
+const _device dev_dev_disk_storage =
+  {"", {FH_DEV_DISK}, "", exists};
+
 const _device dev_proc_storage =
   {"", {FH_PROC}, "", exists};
 
@@ -173,6 +176,7 @@ const _device dev_error_storage =
the POSIX namespace.  */
 %%
 "/dev", BRACK(FH_DEV), "", exists, S_IFDIR
+"/dev/disk", BRACK(FH_DEV_DISK), "", exists, S_IFDIR
 "/dev/tty", BRACK(FH_TTY), "/dev/tty", exists, S_IFCHR
 "/dev/pty%(0-127)d", BRACK(FHDEV(DEV_PTYS_MAJOR, {$1})), "/dev/pty{$1}", 
exists_pty, S_IFCHR, =ptys_dev
 ":p

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov  4 10:34, Corinna Vinschen wrote:

On Nov  3 18:54, Christian Franke wrote:

...
Conclusion: The behavior of the current patch is compatible with Linux :-)

Ok, but with the DUID we have a workaround which makes it  work even
better than on Linux, so it would begreat if we used it, unless we find
out where the UUID in "\GLOBAL??\Disk{} comes from...

Given the size of the STORAGE_DEVICE_UNIQUE_IDENTIFIER struct, we could
even contemplate a 128 bit hash, just to be on the safe side.

Kind of like this

-  strcat (name, ioctl_buf + desc->SerialNumberOffset);
+  /* Use SerialNumber in the first place, if available */
+  if (desc->SerialNumberOffset && desc_buf[desc->SerialNumberOffset])
+strcat (name, desc_buf + desc->SerialNumberOffset);
+  else /* Utilize the DUID as defined by MSDN to generate a hash */
+{
+  union {
+   unsigned __int128 all;
+   struct {
+ unsigned long high;
+ unsigned long low;
+   };
+  } hash = { 0 };
+
+  for (ULONG i = 0; i < id->Size; ++i)
+   hash.all = ioctl_buf[i] + (hash.all << 6) + (hash.all << 16) - hash.all;
+  __small_sprintf (name + strlen (name), "%X%X", hash.high, hash.low);
+}



New patch attached. Only fhandler/dev_disk.cc changed, devices.cc again 
not included.


Christian

From 5db1cd009e8dc6ff01fabdeb8cddf65d609bfe2f Mon Sep 17 00:00:00 2001
From: Christian Franke 
Date: Sat, 4 Nov 2023 16:49:28 +0100
Subject: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

The new directory '/dev/disk/by-id' provides symlinks for each
disk and its partitions:
'BUSTYPE-[VENDOR_]PRODUCT_SERIAL[-partN]' -> '../../sdX[N]'.
This is based on strings provided by STORAGE_DEVICE_DESCRIPTOR.
If this information is too short, a 128-bit hash of the
STORAGE_DEVICE_UNIQUE_IDENTIFIER raw data is added.
Administrator privileges are not required.

Signed-off-by: Christian Franke 
---
 winsup/cygwin/Makefile.am   |   1 +
 winsup/cygwin/devices.in|   4 +
 winsup/cygwin/dtable.cc |   3 +
 winsup/cygwin/fhandler/dev_disk.cc  | 646 
 winsup/cygwin/local_includes/devices.h  |   6 +-
 winsup/cygwin/local_includes/fhandler.h |  47 ++
 winsup/cygwin/mount.cc  |  10 +
 7 files changed, 716 insertions(+), 1 deletion(-)
 create mode 100644 winsup/cygwin/fhandler/dev_disk.cc

diff --git a/winsup/cygwin/Makefile.am b/winsup/cygwin/Makefile.am
index 64b252a22..376c79fc3 100644
--- a/winsup/cygwin/Makefile.am
+++ b/winsup/cygwin/Makefile.am
@@ -84,6 +84,7 @@ FHANDLER_FILES= \
fhandler/console.cc \
fhandler/cygdrive.cc \
fhandler/dev.cc \
+   fhandler/dev_disk.cc \
fhandler/dev_fd.cc \
fhandler/disk_file.cc \
fhandler/dsp.cc \
diff --git a/winsup/cygwin/devices.in b/winsup/cygwin/devices.in
index 2545dd85e..48d3843fe 100644
--- a/winsup/cygwin/devices.in
+++ b/winsup/cygwin/devices.in
@@ -115,6 +115,9 @@ const _device dev_cygdrive_storage =
 const _device dev_fs_storage =
   {"", {FH_FS}, "", exists};
 
+const _device dev_dev_disk_storage =
+  {"", {FH_DEV_DISK}, "", exists};
+
 const _device dev_proc_storage =
   {"", {FH_PROC}, "", exists};
 
@@ -173,6 +176,7 @@ const _device dev_error_storage =
the POSIX namespace.  */
 %%
 "/dev", BRACK(FH_DEV), "", exists, S_IFDIR
+"/dev/disk", BRACK(FH_DEV_DISK), "", exists, S_IFDIR
 "/dev/tty", BRACK(FH_TTY), "/dev/tty", exists, S_IFCHR
 "/dev/pty%(0-127)d", BRACK(FHDEV(DEV_PTYS_MAJOR, {$1})), "/dev/pty{$1}", 
exists_pty, S_IFCHR, =ptys_dev
 ":ptym%(0-127)d", BRACK(FHDEV(DEV_PTYM_MAJOR, {$1})), "/dev/ptym{$1}", 
exists_internal, S_IFCHR, =ptym_dev
diff --git a/winsup/cygwin/dtable.cc b/winsup/cygwin/dtable.cc
index 21d525389..9508f3e0b 100644
--- a/winsup/cygwin/dtable.cc
+++ b/winsup/cygwin/dtable.cc
@@ -585,6 +585,9 @@ fh_alloc (path_conv& pc)
case FH_DEV:
  fh = cnew (fhandler_dev);
  break;
+   case FH_DEV_DISK:
+ fh = cnew (fhandler_dev_disk);
+ break;
case FH_DEV_FD:
  fh = cnew (fhandler_dev_fd);
  break;
diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
new file mode 100644
index 0..2cbe5434e
--- /dev/null
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -0,0 +1,646 @@
+/* fhandler/dev_disk.cc: fhandler for the /dev/disk/by-id/... symlinks.
+
+This file is part of Cygwin.
+
+This software is a copyrighted work licensed under the terms of the
+Cygwin license.  Please consult the file "CYGWIN_LICENSE" for
+details. */
+
+#include "winsup.h"
+#include "path.h"
+#include "fhandler.h"
+#include "tls_pbuf.h"
+#include 
+#include 
+#include 
+
+/* Replace no

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov  4 10:34, Corinna Vinschen wrote:

On Nov  3 18:54, Christian Franke wrote:

...

MSDN claims:

If the storage device is SCSI-compliant, the port driver attempts to
extract the serial number from the optional Unit Serial Number page
(page 0x80) of the VPD.

Now I'm puzzled.

A quick test with a Debian 12 VM in VirtualBox with many virtual
controllers+drives shows the same problem:
Entries in /dev/disk/by-id appear only for virtual disks behind emulated
SATA and NVMe controllers, but not for SCSI and SAS controllers.
A test with "smartctl -i ..." with SCSI/SAS devices doesn't print a serial
number. In debug mode it prints "Vital Product Data (VPD) INQUIRY failed..."
and other messages that suggest limited/buggy support of optional SCSI
commands.

If a Win11 PE (from install ISO) is run in same VM, the
STORAGE_DEVICE_DESCRIPTOR only provides the serial number for SATA (NVMe
drives not detected), but not for SCSI.

Conclusion: The behavior of the current patch is compatible with Linux :-)

Ok, but with the DUID we have a workaround which makes it  work even
better than on Linux, so it would begreat if we used it, unless we find
out where the UUID in "\GLOBAL??\Disk{} comes from...

Given the size of the STORAGE_DEVICE_UNIQUE_IDENTIFIER struct, we could
even contemplate a 128 bit hash, just to be on the safe side.

Kind of like this

-  strcat (name, ioctl_buf + desc->SerialNumberOffset);
+  /* Use SerialNumber in the first place, if available */
+  if (desc->SerialNumberOffset && desc_buf[desc->SerialNumberOffset])
+strcat (name, desc_buf + desc->SerialNumberOffset);
+  else /* Utilize the DUID as defined by MSDN to generate a hash */
+{
+  union {
+   unsigned __int128 all;
+   struct {
+ unsigned long high;
+ unsigned long low;
+   };
+  } hash = { 0 };
+
+  for (ULONG i = 0; i < id->Size; ++i)
+   hash.all = ioctl_buf[i] + (hash.all << 6) + (hash.all << 16) - hash.all;
+  __small_sprintf (name + strlen (name), "%X%X", hash.high, hash.low);
+}



I agree and will provide a new patch soon.

Christian

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov  3 17:27, Corinna Vinschen wrote:

On Nov  3 17:09, Christian Franke wrote:

Unlike (S)ATA and NVMe, the serial number
is not available for free in the device identify data block but requires an
extra command (SCSI INQUIRY of VPD page 0x80). This might not be supported
by the emulated controller or Windows does not use this command.

AFAICS, only the data from STORAGE_DEVICE_ID_DESCRIPTOR is available
which is equivalent to the data from VPD page 0x83.  As you can see,
it's part of the STORAGE_DEVICE_UNIQUE_IDENTIFIER data.  The data
returned for the VirtIo device is the identifier string "\x01\x00",
which is a bit underwhelming.

Would be great if we would learn how to access page 0x80...

Uhm...

MSDN claims:

   If the storage device is SCSI-compliant, the port driver attempts to
   extract the serial number from the optional Unit Serial Number page
   (page 0x80) of the VPD.

Now I'm puzzled.


A quick test with a Debian 12 VM in VirtualBox with many virtual 
controllers+drives shows the same problem:
Entries in /dev/disk/by-id appear only for virtual disks behind emulated 
SATA and NVMe controllers, but not for SCSI and SAS controllers.
A test with "smartctl -i ..." with SCSI/SAS devices doesn't print a 
serial number. In debug mode it prints "Vital Product Data (VPD) INQUIRY 
failed..." and other messages that suggest limited/buggy support of 
optional SCSI commands.


If a Win11 PE (from install ISO) is run in same VM, the 
STORAGE_DEVICE_DESCRIPTOR only provides the serial number for SATA (NVMe 
drives not detected), but not for SCSI.


Conclusion: The behavior of the current patch is compatible with Linux :-)

Christian

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov  3 12:10, Corinna Vinschen wrote:

On Nov  3 11:09, Corinna Vinschen wrote:

On Nov  3 10:55, Corinna Vinschen wrote:

On Oct  3 14:39, Christian Franke wrote:

According to NtQueryObject(., ObjectBasicInformation, ...), using
NtOpenFile(., MAXIMUM_ALLOWED, ...) without admin rights sets GrantedAccess
to 0x001200a0 (FILE_EXECUTE|FILE_READ_ATTRIBUTES|READ_CONTROL|SYNCHRONIZE).
For some unknown reason, NVMe drives behind stornvme.sys additionally
require SYNCHRONIZE to use IOCTL_STORAGE_QUERY_PROPERTY. Possibly a harmless
bug in the access check somewhere in the NVMe stack.

The disk scanning from the first patch has been reworked based on code
borrowed from proc.cc:format_proc_partitions(). For the longer term, it may
make sense to provide one flexible scanning function for /dev/sdXN,
/proc/partitions and /proc/disk/by-id.

I applied your patch locally (patch looks pretty well, btw) but found
that /dev/disk/by-id is empty, even when running with admin rights.

I ran this on Windows 11 and Windows 2K19 in a QEMU/KVM VM.  A
\Device\Harddisk0\Partition0 symlink pointing to \Device\Harddisk0\DR0
exists in both cases.  I straced it, and found the following debug
output:

   1015 1155432 [main] ls 361 stordesc_to_id_name: Harddisk0\Partition0: 
'Red_Hat' 'VirtIO' '' (ignored)

Is that really desired?

We could fake a serial number dependent on the path.  See
https://sourceware.org/git/?p=newlib-cygwin.git;a=blob;f=winsup/cygwin/mount.cc;hb=main#l253

Alternatively, there's also a symlink in GLOBAL?? pointing
to the same target as \Device\Harddisk0\Partition0, i. e.:

   \Device\Harddisk0\Partition0 -> \Device\Harddisk0\DR0

and

   \GLOBAL??\Disk{4c622943-27e4-e81d-3fc7-c43fc9c7e126} -> \Device\Harddisk0\DR0

We could use that UUID from that path, but that's quite a hassle
to grab, because it requires to enumerate GLOBAL??.

But then again, where does Windows get the UUID from?  Something to
find out...

I haven't found out where the UUID is coming from, yet, but based on the
description from
https://learn.microsoft.com/en-us/windows-hardware/drivers/storage/device-unique-identifiers--duids--for-storage-devices
I came up with this Q solution:

=== SNIP 
diff --git a/winsup/cygwin/fhandler/dev_disk.cc 
b/winsup/cygwin/fhandler/dev_disk.cc
index caca57d63216..74abfb8a3288 100644
--- a/winsup/cygwin/fhandler/dev_disk.cc
+++ b/winsup/cygwin/fhandler/dev_disk.cc
@@ -36,29 +36,51 @@ sanitize_id_string (char *s)
return i;
  }
  
+typedef struct _STORAGE_DEVICE_UNIQUE_IDENTIFIER {

+  ULONG Version;
+  ULONG Size;
+  ULONG StorageDeviceIdOffset;
+  ULONG StorageDeviceOffset;
+  ULONG DriveLayoutSignatureOffset;
+} STORAGE_DEVICE_UNIQUE_IDENTIFIER, *PSTORAGE_DEVICE_UNIQUE_IDENTIFIER;
+
+typedef struct _STORAGE_DEVICE_LAYOUT_SIGNATURE {
+  ULONG   Version;
+  ULONG   Size;
+  BOOLEAN Mbr;
+  union {
+ULONG MbrSignature;
+GUID  GptDiskId;
+  } DeviceSpecific;
+} STORAGE_DEVICE_LAYOUT_SIGNATURE, *PSTORAGE_DEVICE_LAYOUT_SIGNATURE;
+


These are available in storduid.h



  /* Get ID string from STORAGE_DEVICE_DESCRIPTIOR. */
  static bool
  stordesc_to_id_name (const UNICODE_STRING *upath, char *ioctl_buf,
char (& name)[NAME_MAX + 1])
  {
+  const STORAGE_DEVICE_UNIQUE_IDENTIFIER *id =
+reinterpret_cast(ioctl_buf);
+  char *desc_buf = ioctl_buf + id->StorageDeviceOffset;
const STORAGE_DEVICE_DESCRIPTOR *desc =
[...]
strcat (name, "_");
-  strcat (name, ioctl_buf + desc->SerialNumberOffset);
+  if (1) /* Utilize the DUID as defined by MSDN */
+{
+  unsigned long hash = 0;
+
+  for (ULONG i = 0; i < id->Size; ++i)
+   hash = ioctl_buf[i] + (hash << 6) + (hash << 16) - hash;
+  __small_sprintf (name + strlen (name), "%X", hash);
+}
+  else
+strcat (name, desc_buf + desc->SerialNumberOffset);
return true;
  }
  
@@ -212,7 +243,7 @@ get_by_id_table (by_id_entry * )

  /* Fetch vendor, product and serial number. */
  DWORD bytes_read;
  STORAGE_PROPERTY_QUERY query =
-   { StorageDeviceProperty, PropertyStandardQuery, { 0 } };
+   { StorageDeviceUniqueIdProperty, PropertyStandardQuery, { 0 } };
  if (!DeviceIoControl (devhdl, IOCTL_STORAGE_QUERY_PROPERTY,
, sizeof (query),
ioctl_buf, NT_MAX_PATH,
=== SNAP 


Thanks. Using this makes plenty of sense as a fallback if the serial 
number is unavailable. But if available, the serial number should be in 
the generated name as on Linux. This would provide a persistent name 
which reflects the actual device without a number invented by MS.


The serial number is usually available with (S)ATA and NVMe (namespace 
uuid in the latter case). I'm not familiar with QEMU/KVM details. The 
fact that both 'vendor' and 'product' are returned on your system 
sugge

Re: [PATCH] Cygwin: Add /dev/disk/by-id symlinks

[Christian Franke]

Corinna Vinschen wrote:

On Nov  3 10:55, Corinna Vinschen wrote:

Hi Christian,

On Oct  3 14:39, Christian Franke wrote:

Christian Franke wrote:

This is a first attempt to partly emulate the Linux directory
/dev/disk/by-id. Useful to make sure the correct device is accessed in
conjunction with dd, ddrescue, fdisk, 

Attached is the second attempt.



The additional '*-partN' links to partitions are not yet included.

These are now included.



This only works properly if Win32 path '\\.\PhysicalDriveN' is always
trivially mapped to NT path '\Device\HarddiskN\Partition0'.
IOCTL_STORAGE_QUERY_PROPERTY with a handle from NtOpenFile(.,
READ_CONTROL,...) instead of CreateFile(., 0, ...) did not work with all
drivers. With stornvme.sys, it fails with permission denied. Perhaps
other permission bits are required for NtOpenFile(). Thanks for any info
regarding this.

According to NtQueryObject(., ObjectBasicInformation, ...), using
NtOpenFile(., MAXIMUM_ALLOWED, ...) without admin rights sets GrantedAccess
to 0x001200a0 (FILE_EXECUTE|FILE_READ_ATTRIBUTES|READ_CONTROL|SYNCHRONIZE).
For some unknown reason, NVMe drives behind stornvme.sys additionally
require SYNCHRONIZE to use IOCTL_STORAGE_QUERY_PROPERTY. Possibly a harmless
bug in the access check somewhere in the NVMe stack.

The disk scanning from the first patch has been reworked based on code
borrowed from proc.cc:format_proc_partitions(). For the longer term, it may
make sense to provide one flexible scanning function for /dev/sdXN,
/proc/partitions and /proc/disk/by-id.

I applied your patch locally (patch looks pretty well, btw) but found


Thanks!
Meantime I found 4 missing NtClose() in the main loop of 
get_by_id_table(). Will be fixed in the next version of the patch.




that /dev/disk/by-id is empty, even when running with admin rights.


Admin rights should not be necessary for IOCTL_STORAGE_QUERY_PROPERTY.




I ran this on Windows 11 and Windows 2K19 in a QEMU/KVM VM.  A
\Device\Harddisk0\Partition0 symlink pointing to \Device\Harddisk0\DR0
exists in both cases.  I straced it, and found the following debug
output:

   1015 1155432 [main] ls 361 stordesc_to_id_name: Harddisk0\Partition0: 
'Red_Hat' 'VirtIO' '' (ignored)

Is that really desired?


Yes - if IOCTL_STORAGE_QUERY_PROPERTY{... PropertyStandardQuery} does 
not return a serial number (''), the device is intentionally ignored.




Thread 1 "ls" hit Breakpoint 2, stordesc_to_id_name (upath=0x7c500,
 ioctl_buf=0x10e0720 "(", name=...)
 at /home/corinna/src/cygwin/vanilla/winsup/cygwin/fhandler/dev_disk.cc:44
44const STORAGE_DEVICE_DESCRIPTOR *desc =
(gdb) n
47int vendor_len = 0, product_len = 0, serial_len = 0;
(gdb)
48if (desc->VendorIdOffset)
(gdb)
49  vendor_len = sanitize_id_string (ioctl_buf + desc->VendorIdOffset);
(gdb)
50if (desc->ProductIdOffset)
(gdb)
51  product_len = sanitize_id_string (ioctl_buf + 
desc->ProductIdOffset);
(gdb)
52if (desc->SerialNumberOffset)
(gdb)
53  serial_len = sanitize_id_string (ioctl_buf + 
desc->SerialNumberOffset);


If possibly, please check whether (desc->SerialNumberOffset) is 0 or 
(ioctl_buf + desc->SerialNumberOffset) points to '\0' or a string of 
spaces. If no, there is possibly something wrong in sanitize_id_string().




(gdb)
55bool valid = (4 <= vendor_len + product_len && 4 <= serial_len
(gdb) p vendor_len
$1 = 7
(gdb) p product_len
$2 = 6
(gdb) p serial_len
$3 = 0
(gdb) n
[New Thread 3944.0x1958]
56  && (20 + vendor_len + 1 + product_len + 1 + serial_len 
+ 10)
(gdb) n
55bool valid = (4 <= vendor_len + product_len && 4 <= serial_len
(gdb)
55bool valid = (4 <= vendor_len + product_len && 4 <= serial_len
(gdb)
58debug_printf ("%S: '%s' '%s' '%s'%s", upath,
(gdb) p valid
$4 = false


--
Regards,
Christian

Re: [PATCH cygport] Add initial support for SOURCE_DATE_EPOCH

[Christian Franke via Cygwin-apps]

Jon Turney wrote:

On 28/08/2023 16:12, Christian Franke via Cygwin-apps wrote:

Christian Franke wrote:

A small step towards reproducible packaging...



Thanks very much for this. Sorry for taking so long to look at it.



No problem.



A few questions and suggestions interspersed

[...]


--
Regards,
Christian


0001-Add-initial-support-for-SOURCE_DATE_EPOCH.patch

 From 73dde4d2dabb74b7b9ee40655204f84e1d4086d6 Mon Sep 17 00:00:00 2001
From: Christian Franke
Date: Mon, 28 Aug 2023 16:24:36 +0200
Subject: [PATCH] Add initial support for SOURCE_DATE_EPOCH

If specified, set the header timestamps of executables and
patch files to SOURCE_DATE_EPOCH.
Suppress the header timestamp of .gz files.
Instruct tar to avoid more recent modification times and
to sort all entries by name.
---
  bin/cygport.in   | 17 +++--
  lib/pkg_pkg.cygpart  | 20 ++--
  lib/src_postinst.cygpart | 22 +++---
  3 files changed, 52 insertions(+), 7 deletions(-)

diff --git a/bin/cygport.in b/bin/cygport.in
index 3f89ac67..e2fe1785 100755
--- a/bin/cygport.in
+++ b/bin/cygport.in
@@ -231,8 +231,9 @@ source ${_privlibdir}/check_funcs.cygpart
  ###
    # check now for all mandatory programs
-for _myprog in bzip2 cat chmod cp diff diffstat dos2unix file find 
gawk grep gzip \
-   install ln mkdir mv patch rm rsync sed sort tar xargs 
which xz
+for _myprog in bzip2 cat chmod cp date diff diffstat dos2unix file 
find gawk grep \
+   gzip install ln mkdir mv patch rm rsync sed sort tar 
touch which \

+   xargs xz
  do
  if ! check_prog ${_myprog}
  then
@@ -490,6 +491,18 @@ do
  done
  unset restrict
  +if [ "${SOURCE_DATE_EPOCH+y}" = "y" ]
+then
+    if [ -n "$(echo "${SOURCE_DATE_EPOCH}" | sed -e 's/^$/X/' -e 
's/[0-9]//g')" ]

+    then
+    error "Malformed SOURCE_DATE_EPOCH: '${SOURCE_DATE_EPOCH}'"


This error message should perhaps say what a well-formed 
SOURCE_DATE_EPOCH looks like: an integer number of seconds since the 
unix epoch?




error "SOURCE_DATE_EPOCH must be an integer number (seconds since the 
unix epoch)" ?




+    fi
+    case $(peflags --version 2>/dev/null | sed -n '1s/^.* //p') in
+    4.6.[6-9]|4.[7-9]*|[5-9]*) ;;
+    *) error "SOURCE_DATE_EPOCH requires peflags 4.6.6 or later"
+    esac
+fi
+

  #
diff --git a/lib/pkg_pkg.cygpart b/lib/pkg_pkg.cygpart
index 2a2bb663..4e6a7cd2 100644
--- a/lib/pkg_pkg.cygpart
+++ b/lib/pkg_pkg.cygpart
@@ -42,7 +42,7 @@ TAR_COMPRESSION_EXT="${TAR_COMPRESSION_EXT:-xz}"
  #
    __tar() {
-    local TAR_COMPRESSION_OPT;
+    local TAR_COMPRESSION_OPT TAR_SOURCE_DATE_OPTS;
    # We could use --auto-compress, but this also constrains the 
extension

  # to the currently valid set. We could probe if tar supports the
@@ -65,7 +65,14 @@ __tar() {
  error "tar option for 
TAR_COMPRESSION_EXT='${TAR_COMPRESSION_EXT}' unknown"

  ;;
  esac
-    tar ${TAR_COMPRESSION_OPT} --owner=Guest:501 --group=None:513 
-cvf "$@"

+
+    if [ -n "${SOURCE_DATE_EPOCH}" ]
+    then
+    # Ensure reproducible sort order and last modification times 
<= SOURCE_DATE_EPOCH
+    TAR_SOURCE_DATE_OPTS="--sort=name 
--mtime=@${SOURCE_DATE_EPOCH} --clamp-mtime"


I'm slightly concerned that maybe this is masking problems elsewhere, 
at least when making the source archive: In 2eb7c0eb I started making 
an effort so that if the "source inputs" have fixed, upstream 
timestamps, we'll preserve those in the output the source package.


(Obviously that's not always going to be the case, e.g. where cygport 
patches are from a local git checkout, so maybe that's not a real 
problem...)


If the following condition holds, the timestamps of "source inputs" are 
not affected:

newest_source_timestamp < SOURCE_DATE_EPOCH < build_time





+    fi
+
+    tar ${TAR_COMPRESSION_OPT} ${TAR_SOURCE_DATE_OPTS} 
--owner=Guest:501 --group=None:513 -cvf "$@"

  }
    __pkg_binpkg() {
@@ -319,6 +326,7 @@ __pkg_diff() {
  local difflevel;
  local exclude;
  local optional_patchfiles;
+    local source_date;
    default_excludes="CYGWIN-PATCHES aclocal.m4~ aclocal.m4t 
autom4te.cache

  config.cache config.guess config.log config.status config.sub
@@ -498,6 +506,14 @@ __pkg_diff() {
    sed -b -e '/^diff -u/d' -i ${optional_patchfiles} 
${patchdir}/${src_patchfile};

  +    if [ -n "${SOURCE_DATE_EPOCH}" ]
+    then
+    # Ensure that the timestamp comment of the new file is 
reproducible


"timestamp comments in the generated patch files" or suchlike?


OK.





+    source_date=$(date -d @"${SOURCE_DATE_EPOCH}" -u +'%Y-%m-%d 
%H:%M:%S.0 +')

+    se