Re: WSL2

[Ben Gamari]

Viktor Dukhovni  writes:

> On Thu, Mar 11, 2021 at 07:53:20PM +, Simon Peyton Jones via ghc-devs 
> wrote:
>
>> Voila
>
> Thanks! 
>
>> /etc/nsswitch.conf group entry
>> group:  files systemd
>
> The main "suspicious" thing here (decoded traces below my signature) is
> that the nsswitch.conf file is configured to try "systemd" as a source
> of group data, but attempts to contact "systemd" or read the underlying
> systemd store directly are failing.  This is different from "not found",
> where systemd might have furnished a negative reply (as is the case on
> my Fedora 31 system, see below).
>
This rings a bell. See #15230.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: What type of performance regression testing does GHC go through?

[Ben Gamari]

Tom Ellis  writes:

> A user posted the following to the ghc-proposals repository.  Both JB
> and RAE suggested ghc-devs as a more appropriate forum.  Since I have
> no idea whether the user has even ever used a mailing list before I
> thought I would lower the activation energy by posting their message
> for them.
>
> https://github.com/ghc-proposals/ghc-proposals/issues/410
>
>> Hi,
>> 
>> Does the GHC release or development process include regression
>> testing for performance?
>> 
>> Is this the place to discuss ideas for implementing such a thing and
>> to eventually craft a proposal?
>> 
>> I believe the performance impact of changes to GHC needs to be
>> verified/validated before release. I also believe this would be
>> feasible if we tracked metrics on building a wide variety of
>> real-world packages. Using real-world packages is one of the best
>> ways to see the actual impact users will experience. It's also a
>> great way to broaden the scope of tests, particularly with the
>> combination of language pragmas and enabled features within the
>> compiler.

We already do this, but help is definitely wanted! In short, every
commit to GHC goes through a variety of performance testing including:

 * the performance testsuite in `base` (which I'm sure all GHC
   developers are all-too-familiar with at this point)

 * a run of the nofib benchmark suite

 * compile-time benchmarking using the head.hackage patchset (when it is
   buildable)

In addition to being preserved as CI artifacts, all of this information
also gets thrown into a PostgreSQL database (see [1]) which is exposed via
Postgrest. The problem is that we currently don't *do* anything with it.
I have occassionally found it useful to do quick queries against it, but
it would be great if someone would step up to help improve this
infrastructure.

Cheers,

- Ben


[1] https://github.com/bgamari/ghc-perf-import


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC Exactprint merge process

[Richard Eisenberg]

I've started a review, but sent along what I had when dinner was ready. 
Hopefully more later, but don't wait up for me!

Incidentally: this is a monstrous patch, and so there is a strong incentive 
just to get on with it without resolving all these quibbles. I won't stand in 
your way on that front -- it might be better to improve this after it lands. 
However, I also see quite a few TODO:AZ notes. Are you intending to fix these 
before landing? Or do you think it's OK to merge first and then return?

High level piece: I'm in support of this direction of movement -- I just want 
to make sure that the new code is understandable and maintainable.

Thanks,
Richard

> On Mar 6, 2021, at 12:39 PM, Alan & Kim Zimmerman  wrote:
> 
> I have been running a branch in !2418[1] for just over a year to migrate the 
> ghc-exactprint functionality directly into the GHC AST[2], and I am now 
> satisfied that it is able to provide all the same functionality as the 
> original.
> 
> This is one of the features intended for the impending 9.2.1 release, and it 
> needs to be reviewed to be able to land.  But the change is huge, as it 
> mechanically affects most files that interact with the GHC AST.
> 
> So I have split out a precursor !5158 [3] with just the new types that are 
> used to represent the annotations, so it can be a focal point for discussion.
> 
> It is ready for review, please comment if you have time and interest.
> 
> Regards
>   Alan
> 
> [1] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/2418 
> 
> [2] https://gitlab.haskell.org/ghc/ghc/-/issues/17638 
> 
> [3] https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5158 
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: WSL2

[Simon Peyton Jones via ghc-devs]

OK thanks. Let's pursue this further on this ticket:
https://gitlab.haskell.org/ghc/ghc/-/issues/19525

Simon

|  -Original Message-
|  From: ghc-devs  On Behalf Of Viktor
|  Dukhovni
|  Sent: 11 March 2021 20:36
|  To: ghc-devs@haskell.org
|  Subject: Re: WSL2
|  
|  On Thu, Mar 11, 2021 at 07:53:20PM +, Simon Peyton Jones via ghc-
|  devs wrote:
|  
|  > Voila
|  
|  Thanks!
|  
|  > /etc/nsswitch.conf group entry
|  > group:  files systemd
|  
|  The main "suspicious" thing here (decoded traces below my signature)
|  is that the nsswitch.conf file is configured to try "systemd" as a
|  source of group data, but attempts to contact "systemd" or read the
|  underlying systemd store directly are failing.  This is different from
|  "not found", where systemd might have furnished a negative reply (as
|  is the case on my Fedora 31 system, see below).
|  
|  So a failure return code is not surprising, because the answer is not
|  authoritative, systemd might have answered differently if it had been
|  possible to query it.  It appears the WSL2 systems have a systemically
|  misconfigured "nsswitch.conf" that wants to query "group" (and likely
|  other) data from an unavailable source.
|  
|  [ Bottom line, the "unix" test case in question may need to be
|  prepared
|to encounter such misconfiguration of the test platform and accept
|either type of error.  Perhaps catch the IO expected IO exception,
|  and
|output a fixed "not found" message regardless of the exception
|  details,
|or by specifically checking for either of the two expected forms. ]
|  
|  By way of contrast, on my Fedora system, systemd can actually be
|  reached and appears to respond to the "nss" library's satisfaction:
|  
|  execve("/usr/bin/getent", ["getent", "group", "xyzzy0"],
|  0x7fff3afbcca0 /* 31 vars */) = 0
|  ...
|  openat(AT_FDCWD, "/lib64/libnss_files.so.2", O_RDONLY|O_CLOEXEC) =
|  3
|  openat(AT_FDCWD, "/etc/group", O_RDONLY|O_CLOEXEC) = 3
|  read(3, "root:x:0:\nbin:x:1:\ndaemon:x:2:\ns"..., 4096) = 1161
|  read(3, "", 4096)   = 0
|  ...
|  openat(AT_FDCWD, "/lib64/libnss_systemd.so.2", O_RDONLY|O_CLOEXEC)
|  = 3
|  access("/etc/systemd/dont-synthesize-nobody", F_OK) = -1 ENOENT
|  (No such file or directory)
|  socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
|  connect(3, {sa_family=AF_UNIX,
|  sun_path="/run/dbus/system_bus_socket"}, 30) = 0
|  getsockopt(3, SOL_SOCKET, SO_PEERCRED, {pid=1, uid=0, gid=0},
|  [12]) = 0
|  getsockopt(3, SOL_SOCKET, SO_PEERSEC, 0x5568c64660e0, [64]) = -1
|  ENOPROTOOPT (Protocol not available)
|  getsockopt(3, SOL_SOCKET, SO_PEERGROUPS, 0x5568c6466130, [256->0])
|  = 0
|  sendmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="\0AUTH EXTERNAL\r\nDATA\r\n", iov_len=22},
|  {iov_base="NEGOTIATE_UNIX_FD\r\n", iov_len=19}, {iov_base="BEGIN\r\n",
|  iov_len=7}], msg_iovlen=3, msg_controllen=0, msg_flags=0},
|  MSG_DONTWAIT|MSG_NOSIGNAL) = 48
|  recvmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="DATA\r\nOK 7bc788e33c85b875f6b74a6"...,
|  iov_len=256}], msg_iovlen=1, msg_controllen=0,
|  msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 58
|  sendmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="l\1\0\1\0\0\0\0\1\0\0\0m\0\0\0\1\1o\0\25\0\0\0/org
|  /fre"..., iov_len=128}], msg_iovlen=1, msg_controllen=0, msg_flags=0},
|  MSG_DONTWAIT|MSG_NOSIGNAL) = 128
|  recvmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="l\2\1\1\16\0\0\0\377\377\377\377G\0\0\0\5\1u\0\1\0
|  \0\0", iov_len=24}], msg_iovlen=1, msg_controllen=0,
|  msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 24
|  recvmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="\7\1s\0\24\0\0\0org.freedesktop.DBus\0\0\0\0"...,
|  iov_len=78}], msg_iovlen=1, msg_controllen=0,
|  msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 78
|  sendmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="l\1\0\1\v\0\0\0\2\0\0\0\247\0\0\0\1\1o\0\31\0\0\0/
|  org/fre"..., iov_len=184}, {iov_base="\6\0\0\0xyzzy0\0", iov_len=11}],
|  msg_iovlen=2, msg_controllen=0, msg_flags=0},
|  MSG_DONTWAIT|MSG_NOSIGNAL) = 195
|  recvmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="l\4\1\1\16\0\0\0\377\377\377\377\227\0\0\0\7\1s\0\
|  24\0\0\0", iov_len=24}], msg_iovlen=1, msg_controllen=0,
|  msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 24
|  recvmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="org.freedesktop.DBus\0\0\0\0\6\1s\0\t\0\0\0"...,
|  iov_len=158}], msg_iovlen=1, msg_controllen=0,
|  msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 158
|  recvmsg(3, {msg_name=NULL, msg_namelen=0,
|  msg_iov=[{iov_base="l\3\1\1(\0\0\0\257\30\r\0m\0\0\0\5\1u\0\2\0\0\0",
|  iov_len=24}], msg_iovlen=1, msg_controllen=0,
|  msg_flags=MSG_CMSG_CLOEXEC}, 

Re: WSL2

[Viktor Dukhovni]

On Thu, Mar 11, 2021 at 07:53:20PM +, Simon Peyton Jones via ghc-devs wrote:

> Voila

Thanks! 

> /etc/nsswitch.conf group entry
> group:  files systemd

The main "suspicious" thing here (decoded traces below my signature) is
that the nsswitch.conf file is configured to try "systemd" as a source
of group data, but attempts to contact "systemd" or read the underlying
systemd store directly are failing.  This is different from "not found",
where systemd might have furnished a negative reply (as is the case on
my Fedora 31 system, see below).

So a failure return code is not surprising, because the answer is not
authoritative, systemd might have answered differently if it had been
possible to query it.  It appears the WSL2 systems have a systemically
misconfigured "nsswitch.conf" that wants to query "group" (and likely
other) data from an unavailable source.

[ Bottom line, the "unix" test case in question may need to be prepared
  to encounter such misconfiguration of the test platform and accept
  either type of error.  Perhaps catch the IO expected IO exception, and
  output a fixed "not found" message regardless of the exception details,
  or by specifically checking for either of the two expected forms. ]

By way of contrast, on my Fedora system, systemd can actually be reached
and appears to respond to the "nss" library's satisfaction:

execve("/usr/bin/getent", ["getent", "group", "xyzzy0"], 0x7fff3afbcca0 /* 
31 vars */) = 0
...
openat(AT_FDCWD, "/lib64/libnss_files.so.2", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/etc/group", O_RDONLY|O_CLOEXEC) = 3
read(3, "root:x:0:\nbin:x:1:\ndaemon:x:2:\ns"..., 4096) = 1161
read(3, "", 4096)   = 0
...
openat(AT_FDCWD, "/lib64/libnss_systemd.so.2", O_RDONLY|O_CLOEXEC) = 3
access("/etc/systemd/dont-synthesize-nobody", F_OK) = -1 ENOENT (No such 
file or directory)
socket(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_UNIX, sun_path="/run/dbus/system_bus_socket"}, 30) 
= 0
getsockopt(3, SOL_SOCKET, SO_PEERCRED, {pid=1, uid=0, gid=0}, [12]) = 0
getsockopt(3, SOL_SOCKET, SO_PEERSEC, 0x5568c64660e0, [64]) = -1 
ENOPROTOOPT (Protocol not available)
getsockopt(3, SOL_SOCKET, SO_PEERGROUPS, 0x5568c6466130, [256->0]) = 0
sendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="\0AUTH 
EXTERNAL\r\nDATA\r\n", iov_len=22}, {iov_base="NEGOTIATE_UNIX_FD\r\n", 
iov_len=19}, {iov_base="BEGIN\r\n", iov_len=7}], msg_iovlen=3, 
msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 48
recvmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="DATA\r\nOK 
7bc788e33c85b875f6b74a6"..., iov_len=256}], msg_iovlen=1, msg_controllen=0, 
msg_flags=MSG_CMSG_CLOEXEC}, MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 58
sendmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="l\1\0\1\0\0\0\0\1\0\0\0m\0\0\0\1\1o\0\25\0\0\0/org/fre"..., 
iov_len=128}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 
MSG_DONTWAIT|MSG_NOSIGNAL) = 128
recvmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="l\2\1\1\16\0\0\0\377\377\377\377G\0\0\0\5\1u\0\1\0\0\0", 
iov_len=24}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 24
recvmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="\7\1s\0\24\0\0\0org.freedesktop.DBus\0\0\0\0"..., 
iov_len=78}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 78
sendmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="l\1\0\1\v\0\0\0\2\0\0\0\247\0\0\0\1\1o\0\31\0\0\0/org/fre"...,
 iov_len=184}, {iov_base="\6\0\0\0xyzzy0\0", iov_len=11}], msg_iovlen=2, 
msg_controllen=0, msg_flags=0}, MSG_DONTWAIT|MSG_NOSIGNAL) = 195
recvmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="l\4\1\1\16\0\0\0\377\377\377\377\227\0\0\0\7\1s\0\24\0\0\0",
 iov_len=24}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 24
recvmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="org.freedesktop.DBus\0\0\0\0\6\1s\0\t\0\0\0"..., 
iov_len=158}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 158
recvmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="l\3\1\1(\0\0\0\257\30\r\0m\0\0\0\5\1u\0\2\0\0\0", 
iov_len=24}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 24
recvmsg(3, {msg_name=NULL, msg_namelen=0, 
msg_iov=[{iov_base="\6\1s\0\t\0\0\0:1.303526\0\0\0\0\0\0\0\4\1s\0*\0\0\0"..., 
iov_len=144}], msg_iovlen=1, msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, 
MSG_DONTWAIT|MSG_CMSG_CLOEXEC) = 144
close(3)= 0

-- 
Viktor.

So group lookups are configured to try /etc/group first, and then some
systemd-based machinery (possibly creating groups on the fly, ...).

> == Tracing getent group xyzzy0

execve("/usr/bin/getent", 

RE: WSL2

[Simon Peyton Jones via ghc-devs]

Voila

simonpj@MSRC-3645512:~$ sh /mnt/c/tmp/getgrnam.sh 
/etc/nsswitch.conf group entry
group:  files systemd
== Tracing getent group xyzzy0
1f8b080818754a6374726163652e6f757400ed5a6d73daba12fe9e5fa1e1cc9d212d09b6e4
373ac3e9e182493c0593b14ddb4ce9788c2d12a6bc5ddbe4909ef4bfdf950d040c26b831bdf743
9336962d69779fd56a575a892ea8fb408b85f23cf0cbfde1a47c47433a090b25f4a5b02e16eefc
e97cc60a8bc7efdf1fb9c2d712e216f26040fb6265203b8443e537084be8c1f103f4a67c8eaa88
3bebfbdf8a7ab7d53a47877fa0ed4214dd3e2f7b3c073f678eefdedb33df0d47456e41388e67d4
6b46fdda7efffe3d905f32773941ea2b32cf3176173c5235fd63ad858adae4c1190d3de4f877f3
3100383f735c97060180a4a15b1e7997c1f472e6d3d1d4f1009361773e9caf246164f48eaa5ba8
a84f513077efd16038a268ea236fe853379cfa8fe767d3199d3861b166d9cd46fd5303f4b241d9
75dc7b0a743bb6d1e8e8addba78e5d6f75d4cf6a9dc949ce0641087d4909fd1384f678ead1aa69
6b4d43bd7ae224412821f81a0cbfd32a11154529a1cbcbcb1fb13ec763671629b48496753746c7
b20db50612b46b37f68da17dac592a548386a23ea026ecf62587ef33bdbaa369408be4a5f18898
ed81381af6cb0b45b225e162349ccc1717779339fbe832d4d201c43e753c06b8d0e365596d357b
b8c7c32fe971895ff8f2273cf8d5abc4fd07daaeeb0ba08b1252086664e1713663942521a62d2d
9b09f1f3af6dda875f63cab202ea9704461d8adbd497547b78d5478c9f577a977d8de5e5d6dc93
e0403b0483ec8212a904a7104ff48e89873d4c480f63fcaf1e96851e96a01ec420a041f61f4b3d
9ec90535b2f4f94fa827311c092c4451224390944366278be2b3d9610e573016520c4fe12b78c3
ee9ea2d227436356b761824fac5cd33bfa6dbbd3354b30ad762cb2c22cf2f7001e3f801ba38039
2255449ce60022ed3754fd7639304977a06099297f3cf3a721b8b4e2fabbe0c2f712e21541c615
69491e4651ddb0839dc6a2c071ebc6cf46c1a6ffae544dedb3dad827df028b406e4bca88c136d3
8a2b444c0947784e38883f9d135ff112ac62b2dbac1c6e10b1c28228efa24b31f9034ca99c601a
334830e5c558a944c44a76a687679c9835066cc4e128fa9a2a4403b3b441d21163f3dc634c2bfd
f1186f85aa9de671e87758e487e6c2b62da51117f903ade7936d9df6639b6131f33c01f1a7d728
ace3fa13e1a3b17da163d46a5f60654b2f165c4753d719d1e5e382697ff890cb4a822362e42ef6
af2556b5c7ae264481cf6135c14007f78e4fb7615fc2d2cd0972008d2bcc349e11af17217fa056
c4094d9c3145113be439a183fa4e402f7b933f62e7cb4c8bf564749e3b175615e95873d0cb8631
d42fbb56f34229b7eab6063ec5d29a5abd66691d3d0f15a5da04ce620f8ec8e5640f7b579877ee
74f210ffbd0030f3110d76d6d81971832b4e451ed725b09bd735436dec5b587b3942df1df2b65a
33bb86da8681cf63bc491a6892652fe19e14b2a5b6d49b6b5873e40058905300b38a5fbb794a07
5c6b340cd5347380cb133e056f5473bc7fcf733eef02d66bed3c06574a9bc2529658460627c57a
53bb518d3c4278da868ca4ae82f781a527f656a659bb52b39b72433312bbd07861f78cf8e43297
cd5bd37e0d863dde272d7b236448dd88e4c411063cad55336ef3082f32971e54b3203e6d80a977
5a2d609c87bf157985e7d340af6b8f054edcca6923ab968bdf25444a831c571d8f57396d9ce9b6
5543abe700594c032c66822b9fd6aeaddb9b3cc617733c965397c7cbcaa3415732671d82a9fb8d
02e8a6ddd5b5cf256476ea1f6cd30266eda7a8bc4415bfe81dfddf2d282cf99233d8274c58aa20
02ead803673c1c3d56d7d482f9c49e39e17db5507e70fcb23f9f942781eb9563ae851f306f9f0f
145e3e09f80d6cbf99b2e3904910fc3d0cddfb4b907c90c72cdc76b51b1bfa5d76b08d877f485d
38e3d9886eefe881ceafdcd0ff5f9f0c11258f609b763204436233db0a187a9cfd8428793eb47b
42c471e2ee31c0c609d1f1b6a590349f2757a47497f752ae9dc8ca4e9697c87146122bdb847f2e
714e1279e5987a82a382238e89e39b2c6c9c249b8864928db08fcd4fa6ccfb3b1c77b3f4449162
55629277c23c269d696aece4a9570a39264f1d4fc5943c758a5f599dcba7cdab5140e93766fe40
d7545508315d631f12c6e2d8a9a26065af1bf6a7d3f0dde21df7ae37f11c3a9e4ee0858797fe90
953094826d3f0c843208c85afff6dabfca6b078f4148c7dea9fc36b88e692e7e1b938ab29d29d8
5cac0a024fa49ff6dc64f7fc91087879e8294ac2aebfc9eabaa5a48713f6f85449895d48857fe1
c4f7c0d9aa9b6424ed8b4ab1afe299d77abdf726c9f3ce98fea98c7616de33636406cb9dc26031
335afea0cd66b8114004f1cae91149ee6149b4e2db00b200cb19fee3f9ede02bb0233d42e48d6b
0058c8788f8317e5c4358e5f23df660644e0042ed5adbd38ffe22b22db368ae35362a92291d7af
9c762c34a29ee048e29366cc094aea66ffa5b37f25c9283e0a4e3092725c3bf13b333e229f64a9
2c6f1b087bb4f9bac513c99cdbd95d3c2d3572dc9580b5fbda7bdf20a0a11d0e3ddbf13c9f5d05
dcb8c2a0881b47f755e82f0b246aef4ffb73984da361106eb597a25b21e76bc17d36e3ee1c371c
4e274553bb32acb6a6c79bf47b67e28da85fddb0b0fe00ba43d5d809be55bf7c8dca8391731754
cd1a086c5a1d43359ea00c9434bdd9891a80ccb02ddf2244213cc1a67e792deb3c45129b4f15c4
ad1c29082bd70ceb5532c168b98c1193caeeae321b5f22552126e6d744b7993f1a8e8721b82b90
d268696dcdb2410cd62b6ef70f6b60bb73bf5a9115e50dcf3167177d1b3b8b2aeb21093608abe9
9a75fb23fb8a7b8fdc2ba9afbb37a86b1a3cfb83d18d76a3a25acb68a3fa75ab812cd3825ae30a
7db4a28f60884df449d3ebd748eb004aa6e925c63b1a824d168f981d4ba3dc97406a5c1989fc51
e68cd15f8579407daf7fc14b7d85488e57c1dcc0237deaf032e5078ec0d1ca000f88cbb248025e
2591ea1d5d37d466d7541ba8588f7981e1219f0e809ef7ea1c124b612dd7a1e558c0f276645fa5
c89e833c941a9aa1d6c1766f8fcf75fd2f79ef313353b5da35f3436c2a5b93822e86a11dedf98a
f89056abe8fdd9dbb76f116b4f3df4f730bc4718c197b3ff02e18adc56ef2d
== Tracing getgrnam xyzzy0
(null)((nil)) No such process(3)
1f8b080818754a6374726163652e6f757400ed597d6fda4813ff9f4fb1a27a24686958effa

Re: WSL2

[Viktor Dukhovni]

On Thu, Mar 11, 2021 at 12:21:15PM +, Simon Peyton Jones via ghc-devs wrote:

> Like Tom, I'm not following the details, but if you want me to run
> some commands and send you the output I can do that.  Just send the
> script!

See attached.  If any of the prerequisite shell utilities are not
installed, the script will exit asking that they be installed.

Please email me the output, or post to the list.  (Should be just a
couple of hundred lines of mostly hex output).

-- 
Viktor.


getgrnam.sh
Description: Bourne shell script
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: Inlining of `any @[]` vs `elem @[]`

[Simon Peyton Jones via ghc-devs]

Gergo

With HEAD, and -O, I get the exact same (good code) for these two functions:

f x = any (x ==) [1, 5, 7::Int]

g x = elem x [2, 6, 9 :: Int]

namely

f = \ (x_aga :: Int) ->
  case x_aga of { GHC.Types.I# x1_a13b ->
  case x1_a13b of {
__DEFAULT -> GHC.Types.False;
1# -> GHC.Types.True;
5# -> GHC.Types.True;
7# -> GHC.Types.True
  }
  }

g = \ (x_aQu :: Int) ->
  case x_aQu of { GHC.Types.I# x1_a13b ->
  case x1_a13b of {
__DEFAULT -> GHC.Types.False;
2# -> GHC.Types.True;
6# -> GHC.Types.True;
9# -> GHC.Types.True
  }
  }

Maybe this is fixed?  If you think not, maybe open a ticket?

Simon

|  -Original Message-
|  From: ghc-devs  On Behalf Of ÉRDI Gergo
|  Sent: 07 March 2021 02:59
|  To: GHC Devs 
|  Subject: Inlining of `any @[]` vs `elem @[]`
|  
|  Hi,
|  
|  The inlining behaviour of `any @[]` and `elem @[]` differs in a way
|  that I am not sure is intentional, and it is affecting Clash (see
|  https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
|  ub.com%2Fclash-lang%2Fclash-
|  compiler%2Fissues%2F1691data=04%7C01%7Csimonpj%40microsoft.com%7C
|  e37a9761e8814eada5f208d8e115026d%7C72f988bf86f141af91ab2d7cd011db47%7C
|  1%7C0%7C637506827802688772%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDA
|  iLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=Kik8v
|  KuwNobr9kiQOcIHuKTn%2BbEmQ7oY8tqP9tFjs6M%3Dreserved=0). I would
|  think that if it is a good idea to inline `any` then inlining `elem`
|  would be just as good an idea, or vice versa.
|  
|  However, `any` is defined polymorphically over `Foldable`, via
|  `foldMap` using `foldr`, with all steps between (and `foldr @[]`!)
|  marked as `INLINE`. The result is that if you use `any (x ==) [1, 5,
|  7]` you get the following beautiful Core:
|  
|  ```
|  topEntity
| = \ (x_agAF :: Int) ->
| case x_agAF of { GHC.Types.I# y_ahao ->
| case y_ahao of {
|   __DEFAULT -> GHC.Types.False;
|   1# -> GHC.Types.True;
|   5# -> GHC.Types.True;
|   7# -> GHC.Types.True
| }
| }
|  ```
|  
|  As the kids these days would say: *chef's kiss*.
|  
|  
|  `elem`, on the other hand, is a typeclass method of `Foldable`, with a
|  default implementation in terms of `any`, but overridden for lists
|  with the following implementation:
|  
|  ```
|  GHC.List.elem :: (Eq a) => a -> [a] -> Bool
|  GHC.List.elem _ []   = False
|  GHC.List.elem x (y:ys)   = x==y || GHC.List.elem x ys
|  {-# NOINLINE [1] elem #-}
|  {-# RULES
|  "elem/build"forall x (g :: forall b . Eq a => (a -> b -> b) -> b -
|  > b)
|  . elem x (build g) = g (\ y r -> (x == y) || r) False
|#-}
|  ```
|  
|  This is marked as non-inlineable until phase 1 (so that `elem/build`
|  has a chance of firing), but it seems that when build fusion doesn't
|  apply (since `[1, 5, 7]` is, of course, not built via `build`), no
|  inlining happens AT ALL, even in later phases, so we end up with this:
|  
|  ```
|  topEntity
| = \ (x_agAF :: Int) ->
| GHC.List.elem
|   @ Int
|   GHC.Classes.$fEqInt
|   x_agAF
|   (GHC.Types.:
|  @ Int
|  (GHC.Types.I# 1#)
|  (GHC.Types.:
| @ Int
| (GHC.Types.I# 5#)
| (GHC.Types.: @ Int (GHC.Types.I# 7#) (GHC.Types.[] @
|  Int ```
|  
|  So not only does it trip up Clash, it would also result in less
|  efficient code in software when using "normal" GHC.
|  
|  Is this all intentional? Wouldn't it make more sense to mark
|  `GHC.List.elem` as `INLINE [1]` instead of `NOINLINE [1]`, so that any
|  calls remaining after build fusion would be inlined?
|  
|  Thanks,
|   Gergo
|  ___
|  ghc-devs mailing list
|  ghc-devs@haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.
|  haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devsdata=04%7C01%7Csimonpj%40microsoft.com%7Ce37a9761e8814eada5f2
|  08d8e115026d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637506827802
|  688772%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
|  BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=yMXu0XJQU2GmlDTH9ZaHXhl33
|  ZRBjHMe41rr8lKVxkk%3Dreserved=0
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: WSL2

[Simon Peyton Jones via ghc-devs]

PS: since this is not, apparently, just my stupidity, it would be good to open 
a ticket and transfer this thread to it.  Would someone like to do that?

|  -Original Message-
|  From: ghc-devs  On Behalf Of Tom Ellis
|  Sent: 11 March 2021 11:41
|  To: ghc-devs@haskell.org
|  Subject: Re: WSL2
|  
|  On Thu, Mar 11, 2021 at 06:19:46AM -0500, Viktor Dukhovni wrote:
|  > On Thu, Mar 11, 2021 at 06:05:04AM -0500, Viktor Dukhovni wrote:
|  > > So the question is why the lookup is failing.  To that end
|  compiling
|  > > a tracing with "strace" the below C program should tell the story:
|  [...]
|  > To experiment with other group names and make sure that at least
|  group
|  > "root" or similar works, a slightly extended version is:
|  [...]
|  
|  I'm not really following the details, but is this useful to you?
|  
|  % cat g.c && cc g.c -o g && ./g
|  #include 
|  #include 
|  #include 
|  #include 
|  
|  int main(int argc, char **argv)
|  {
|  char buf[1024];
|  struct group g, *p;
|  int rc;
|  
|  errno = 0;
|  rc = getgrnam_r(argc > 1 ? argv[1] : "nosuchgrouphere",
|  , buf, sizeof(buf), );
|  printf("%s(%p) %m(%d)\n", p ? g.gr_name : NULL, p, errno);
|  return (rc == 0 && p == NULL);
|  }
|  (null)((nil)) No such process(3)
|  ___
|  ghc-devs mailing list
|  ghc-devs@haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.
|  haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devsdata=04%7C01%7Csimonpj%40microsoft.com%7C48a10ad0766c4dd6caf4
|  08d8e4829c7d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637510597246
|  441070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
|  BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=nQdF9H7BpTqQL%2Bm0URWQmXh
|  1KEQAV1KgfPvG75mOR%2B0%3Dreserved=0
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: WSL2

[Simon Peyton Jones via ghc-devs]

Like Tom, I'm not following the details, but if you want me to run some 
commands and send you the output I can do that.  Just send the script!

|  -Original Message-
|  From: ghc-devs  On Behalf Of Tom Ellis
|  Sent: 11 March 2021 11:41
|  To: ghc-devs@haskell.org
|  Subject: Re: WSL2
|  
|  On Thu, Mar 11, 2021 at 06:19:46AM -0500, Viktor Dukhovni wrote:
|  > On Thu, Mar 11, 2021 at 06:05:04AM -0500, Viktor Dukhovni wrote:
|  > > So the question is why the lookup is failing.  To that end
|  compiling
|  > > a tracing with "strace" the below C program should tell the story:
|  [...]
|  > To experiment with other group names and make sure that at least
|  group
|  > "root" or similar works, a slightly extended version is:
|  [...]
|  
|  I'm not really following the details, but is this useful to you?
|  
|  % cat g.c && cc g.c -o g && ./g
|  #include 
|  #include 
|  #include 
|  #include 
|  
|  int main(int argc, char **argv)
|  {
|  char buf[1024];
|  struct group g, *p;
|  int rc;
|  
|  errno = 0;
|  rc = getgrnam_r(argc > 1 ? argv[1] : "nosuchgrouphere",
|  , buf, sizeof(buf), );
|  printf("%s(%p) %m(%d)\n", p ? g.gr_name : NULL, p, errno);
|  return (rc == 0 && p == NULL);
|  }
|  (null)((nil)) No such process(3)
|  ___
|  ghc-devs mailing list
|  ghc-devs@haskell.org
|  https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.
|  haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-
|  devsdata=04%7C01%7Csimonpj%40microsoft.com%7C48a10ad0766c4dd6caf4
|  08d8e4829c7d%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637510597246
|  441070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJ
|  BTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000sdata=nQdF9H7BpTqQL%2Bm0URWQmXh
|  1KEQAV1KgfPvG75mOR%2B0%3Dreserved=0
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: WSL2

[Viktor Dukhovni]

> On Mar 11, 2021, at 9:41 AM, Tom Ellis 
>  wrote:
> 
> I'm not really following the details, but is this useful to you?
> 
> % cat g.c && cc g.c -o g && ./g
> #include 
> #include 
> #include 
> #include 
> 
> int main(int argc, char **argv)
> {
>char buf[1024];
>struct group g, *p;
>int rc;
> 
>errno = 0;
>rc = getgrnam_r(argc > 1 ? argv[1] : "nosuchgrouphere",
>, buf, sizeof(buf), );
>printf("%s(%p) %m(%d)\n", p ? g.gr_name : NULL, p, errno);
>return (rc == 0 && p == NULL);
> }
> (null)((nil)) No such process(3)

Yes, it means that the reported error is not an artefact of
the Haskell "unix" package, but rather originates directly
from normal use of the getpwnam_r(3) glibc API on these
systems.

It would now be useful to also post:

  - The output of "./g root" or some other group known to exist.
  - The output of "./g xyzzy" or some other short group name known to
 not exist
  - The output of "grep group /etc/nsswitch.conf"
  - Attach an strace output file (g.trace.txt) from:

strace -o g.trace.txt ./g

-- 
Viktor.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: WSL2

[Tom Ellis]

On Thu, Mar 11, 2021 at 06:19:46AM -0500, Viktor Dukhovni wrote:
> On Thu, Mar 11, 2021 at 06:05:04AM -0500, Viktor Dukhovni wrote:
> > So the question is why the lookup is failing.  To that end compiling a
> > tracing with "strace" the below C program should tell the story:
[...]
> To experiment with other group names and make sure that at least
> group "root" or similar works, a slightly extended version is:
[...]

I'm not really following the details, but is this useful to you?

% cat g.c && cc g.c -o g && ./g
#include 
#include 
#include 
#include 

int main(int argc, char **argv)
{
char buf[1024];
struct group g, *p;
int rc;

errno = 0;
rc = getgrnam_r(argc > 1 ? argv[1] : "nosuchgrouphere",
, buf, sizeof(buf), );
printf("%s(%p) %m(%d)\n", p ? g.gr_name : NULL, p, errno);
return (rc == 0 && p == NULL);
}
(null)((nil)) No such process(3)
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: WSL2

[Viktor Dukhovni]

On Thu, Mar 11, 2021 at 06:05:04AM -0500, Viktor Dukhovni wrote:

> So the question is why the lookup is failing.  To that end compiling a
> tracing with "strace" the below C program should tell the story:
> 
> #include 
> #include 
> #include 
> #include 
> 
> int main(int argc, char **argv)
> {
> struct group g, *p;
> char buf[1024];
> int rc;
> 
> errno = 0;
> rc = getgrnam_r("nosuchgrouphere", , buf, sizeof(buf), );
> printf("%p: %m(%d)\n", p, errno);
> return (rc == 0 && p == NULL);
> }

To experiment with other group names and make sure that at least
group "root" or similar works, a slightly extended version is:

#include 
#include 
#include 
#include 

int main(int argc, char **argv)
{
char buf[1024];
struct group g, *p;
int rc;

errno = 0;
rc = getgrnam_r(argc > 1 ? argv[1] : "nosuchgrouphere",
, buf, sizeof(buf), );
printf("%s(%p) %m(%d)\n", p ? g.gr_name : NULL, p, errno);
return (rc == 0 && p == NULL);
}

This gives (again Fedora 31) the expected results:

$ make g
cc g.c   -o g
$ ./g
(null)((nil)) Success(0)
$ ./g root
root(0x7ffe6a6225d0) Success(0)

-- 
Viktor.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: WSL2

[Viktor Dukhovni]

On Thu, Mar 11, 2021 at 10:19:52AM +, Tom Ellis wrote:

> SPJ Wrote:
> > I've just installed WSL2 and built GHC. I get this (single)
> > validation failure in libraries/unix/tests/getGroupEntryForName.  It
> > seems to be just an error message wibble, but I can't push a change
> > to master because that'll affect everyone else.
> 
> Interesting, I've only ever built GHC on WSL and WSL2. I've seen this
> error message on WSL2 during every test run, I think.  I didn't
> realise that it never occurred on other platforms, let alone that it
> was WSL2 specific!

I am curious what specific version/branch of GHC (and associated
submodule commit of "unix") is being tested.

I've recently cleaned a bunch of the upstream "unix" handling of the
group/passwd database handling, but I don't believe that GHC has yet
switched to the newer code.

A subtle facet of the delta points in the right direction:

-getGroupEntryForName: getGroupEntryForName: does not exist (no such group)
+getGroupEntryForName: getGroupEntryForName: does not exist (No such 
process)

not only is it complaining about "process" rather than "group", but
crucially the case of the word "No" is different.  The variance is due
to the fact that there are two possible error paths with group lookup
in the group lookup code:

doubleAllocWhileERANGE loc enttype initlen unpack action =
  alloca $ go initlen
 where
  go len res = do
r <- allocaBytes len $ \buf -> do
   rc <- action buf (fromIntegral len) res
   if rc /= 0
--hard-error->   then return (Left rc)
 else do p <- peek res
--not-found-->   when (p == nullPtr) $ notFoundErr
 fmap Right (unpack p)
case r of
  Right x -> return x
  Left rc | Errno rc == eRANGE ->
-- ERANGE means this is not an error
-- we just have to try again with a larger buffer
go (2 * len) res
  Left rc ->
--1-->  ioError (errnoToIOError loc (Errno rc) Nothing Nothing)
  notFoundErr =
--2-->  ioError $ flip ioeSetErrorString ("no such " ++ enttype)
$ mkIOError doesNotExistErrorType loc Nothing Nothing

The expected error path is "not-found" -> (2), where the group lookup
works, but no result is found (rc == 0).  This reports the lower-case
"no such group".

The unexpected error path is a non-zero return from "getgrnam_r"
(action) -> (1), which uses `errno` to build the error string, which
ends up being "No such process".

On Linux systems that's: ESRCH 3 /* No such process */

So the call to "getgrnam_r" failed by returning ESRCH, rather than 0.
The Linux manpage does not suggest to me that one might expect a
non-zero return from getgrnam_r(3) just from a missing entry in the
group file:

RETURN VALUE
   The getgrnam() and getgrgid() functions return a pointer to a
   group structure, or NULL if the matching entry is not found
   or an error occurs.  If an error occurs, errno is set
   appropriately.  If one wants to check errno after the call,
   it should be set to zero before the call.

   The return value may point to a static area, and may be
   overwritten by subsequent calls to getgrent(3), getgrgid(),
   or getgrnam().  (Do not pass the  returned  pointer  to
   free(3).)

   On  success, getgrnam_r() and getgrgid_r() return zero, and
--->   set *result to grp.  If no matching group record was found,
--->   these functions return 0 and store NULL in *result.  In case
--->   of error, an error number is returned, and NULL is stored in
--->   *result.

ERRORS
   0 or ENOENT or ESRCH or EBADF or EPERM or ...
  The given name or gid was not found.

   EINTR  A signal was caught; see signal(7).

   EIOI/O error.

   EMFILE The per-process limit on the number of open file descriptors 
has been reached.

   ENFILE The system-wide limit on the total number of open files has 
been reached.

   ENOMEM Insufficient memory to allocate group structure.

   ERANGE Insufficient buffer space supplied.

The "0 or ENOENT or ESRCH ..." text then plausibly applies to
getgrnam(3), and its legacy behaviour.

So the question is why the lookup is failing.  To that end compiling a
tracing with "strace" the below C program should tell the story:

#include 
#include 
#include 
#include 

int main(int argc, char **argv)
{
struct group g, *p;
char buf[1024];
int rc;

errno = 0;
rc = getgrnam_r("nosuchgrouphere", , buf, sizeof(buf), );
printf("%p: %m(%d)\n", p, errno);
return (rc == 0 && p == NULL);
}

On a Fedora 31 system I get:

$ make g
cc g.c   -o g
$ ./g
(nil): Success(0)

If something 

What type of performance regression testing does GHC go through?

[Tom Ellis]

A user posted the following to the ghc-proposals repository.  Both JB
and RAE suggested ghc-devs as a more appropriate forum.  Since I have
no idea whether the user has even ever used a mailing list before I
thought I would lower the activation energy by posting their message
for them.

https://github.com/ghc-proposals/ghc-proposals/issues/410

> Hi,
> 
> Does the GHC release or development process include regression
> testing for performance?
> 
> Is this the place to discuss ideas for implementing such a thing and
> to eventually craft a proposal?
> 
> I believe the performance impact of changes to GHC needs to be
> verified/validated before release. I also believe this would be
> feasible if we tracked metrics on building a wide variety of
> real-world packages. Using real-world packages is one of the best
> ways to see the actual impact users will experience. It's also a
> great way to broaden the scope of tests, particularly with the
> combination of language pragmas and enabled features within the
> compiler.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: WSL2

[Tom Ellis]

SPJ Wrote:
> I've just installed WSL2 and built GHC. I get this (single)
> validation failure in libraries/unix/tests/getGroupEntryForName.  It
> seems to be just an error message wibble, but I can't push a change
> to master because that'll affect everyone else.

Interesting, I've only ever built GHC on WSL and WSL2. I've seen this
error message on WSL2 during every test run, I think.  I didn't
realise that it never occurred on other platforms, let alone that it
was WSL2 specific!

Tom
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs