Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

2022-01-31 Thread Reuben Thomas
On Mon, 31 Jan 2022 at 18:43, Reuben Thomas  wrote:
>
> Thanks! I'll make a new release.

I've released 2.12.

-- 
https://rrt.sc3d.org



Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

2022-01-31 Thread Reuben Thomas
On Mon, 31 Jan 2022 at 18:37, Sergei Trofimovich  wrote:

> Your varian of atext-1 works on musl for me. Thank you!

Thanks! I'll make a new release.

-- 
https://rrt.sc3d.org



Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

2022-01-31 Thread Sergei Trofimovich
On Mon, 31 Jan 2022 11:39:47 +
Reuben Thomas  wrote:

> Thanks very much for the analysis and suggested patch.
> 
> I don't think however that a code change is needed. Arguably, the
> correct fix would be to check the return value of wprintf, but in fact
> close_stdout() already checks if there was a previous error on the
> stream. I think that arbitrarily setting a large buffer size is a bit
> of a hack, to be honest.
> 
> So I think it's sufficient, at the cost of a slightly less nice error
> message, to allow the code to proceed as at present, and instead to
> change the test so that it does not specifically check for "No space
> left on device", but instead just "write error".
> 
> I attach an updated version of tests/atexit-1; please could you check
> that with it the test now passes with musl? (Also, of course let me
> know if you disagree with my analysis!)

Your varian of atext-1 works on musl for me. Thank you!

-- 

  Sergei



Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

2022-01-31 Thread Reuben Thomas
Thanks very much for the analysis and suggested patch.

I don't think however that a code change is needed. Arguably, the
correct fix would be to check the return value of wprintf, but in fact
close_stdout() already checks if there was a previous error on the
stream. I think that arbitrarily setting a large buffer size is a bit
of a hack, to be honest.

So I think it's sufficient, at the cost of a slightly less nice error
message, to allow the code to proceed as at present, and instead to
change the test so that it does not specifically check for "No space
left on device", but instead just "write error".

I attach an updated version of tests/atexit-1; please could you check
that with it the test now passes with musl? (Also, of course let me
know if you disagree with my analysis!)

-- 
https://rrt.sc3d.org


atexit-1
Description: Binary data


Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

2022-01-29 Thread Sergei Trofimovich
On Thu, 27 Jan 2022 22:08:57 +
Reuben Thomas  wrote:

> On Wed, 26 Jan 2022 at 12:59, Sergei Trofimovich  wrote:
> >  
> 
> Thanks for the report!
> 
> > From what I understand at
> > 
> > https://wiki.musl-libc.org/functional-differences-from-glibc.html#Character-sets-and-locale
> > musl deliberately supports any char -> wchar_t (and back) conversion for
> > any char in LC_CTYPE=C. And thus the multibyte-1 test will always fail
> > there.  
> 
> I also found this problem on macOS. I tried on my GNU/Linux system
> using an ISO-8859 locale for the test, but the multibyte conversion
> does not fail! (e.g. in en_GB.iso88591). I also tried with
> ru_RU.KOI8-R, same result.
> 
> It seems then that it's not going to be possible in general to find a
> locale that will fail. These results suggest that on some libc's,
> conversion will always succeed.
> 
> Hence, I'm removing the test.
> 
> > Not sure why atexit-1 fails. It seems to rely on
> > close_stdout_set_file_name() call in atexit(), but I don't see where in
> > code it's registered to call.  
> 
> Line 155 registers close_stdout. I can't see anything wrong with the
> way this is done; I'd be very grateful if you could investigate
> further.

Aha, I misinterpreted the atexit handler and did not notice
close_stdout_set_file_name() is part of gnulib.

Attached possible patch. Passes atexit-1 test on musl and glibc.

Some details:

The real reason to failure on musl is stream status and errno value
status in close_stream(). If I add a bit of fprintf() debugging it
becomes clearer:

  void
  close_stdout (void)
  {
fprintf(stderr, "%s: errno=%d\n", __func__, errno);
if (close_stream (stdout) != 0
&& !(ignore_EPIPE && errno == EPIPE))
  {
fprintf(stderr, "%s: after close_stream() errno=%d\n", __func__, errno);
  ...


Execution on glibc:

  close_stdout: errno=0
  close_stdout: after close_stream() errno=28
  ./hello: write error: No space left on device


Execution on glibc:

  close_stdout: errno=28
  close_stdout: after close_stream() errno=0
  hello: write error

On glibc stdout flush did not happen yet by the time we get to atexit().
ltrace also gives the same hint:

On glibc:

$ ltrace ./hello
...
strlen("Hello, world!") = 13
malloc(56)  = 0x10c78c0
mbsrtowcs(0x10c78c0, 0x7ffeb2b630a0, 14, 0x7ffeb2b630a8)= 13
wprintf(0x408298, 0x10c78c0, 177, 0)= 14
free(0x10c78c0) = 
exit(0 
__errno_location()  = 0x7fa894ceb548
fprintf(0x7fa894cdf440, "%s: errno=%d\n", "close_stdout", 0close_stdout: errno=0
) = 22
__fpending(0x7fa894cdf520, 0x7ffeb2b609d0, 0, 0x7fa894c11343) = 14
ferror(0x7fa894cdf520)  = 0
fclose(0x7fa894cdf520)  = -1
fprintf(0x7fa894cdf440, "%s: after close_stream() errno=%"..., "close_stdout", 
28close_stdout: after close_stream() errno=28
) = 44
dcgettext(0, 0x40836e, 5, 0x7fa894c11343)   = 0x40836e
error(0, 28, 0x4080c8, 0x40836e./hello: write error: No space left on device
)= 0
_exit(1 
+++ exited (status 1) +++

Note that wprintf() queued 14 symbols successfully.

On musl queueing did not happen and thus stream close did not return error:

$ ltrace ./hello
...
strlen("Hello, world!") = 13
malloc(56)  = 0x5570a96ddb10
mbsrtowcs(0x5570a96ddb10, 0x7ffdcb755398, 14, 0x7ffdcb7553a0) = 13
wprintf(0x5570a96d9298, 0x5570a96ddb10, 0xff3e, 1)  = 0x
free(0x5570a96ddb10)= 
exit(0 
__errno_location()  = 0x7fad63ed0b1c
fprintf(0x7fad63ece0c0, "%s: errno=%d\n", "close_stdout", 28close_stdout: 
errno=28
) = 23
__fpending(0x7fad63ece2c0, 0x7ffdcb755040, 0x, 0)   = 0
ferror(0x7fad63ece2c0)  = 1
fclose(0x7fad63ece2c0)  = 0
__errno_location()  = 0x7fad63ed0b1c
fprintf(0x7fad63ece0c0, "%s: after close_stream() errno=%"..., "close_stdout", 
0close_stdout: after close_stream() errno=0
) = 43
gettext(0x5570a96d92ba, 0x7ffdcb755040, 0x, 0)  = 0x5570a96d92ba
fileno(0x7fad63ece2c0)  = 1
fcntl(1, 3, 0x636f6c2f7273752f, 0x5570a96d92ba) = -1
fprintf(0x7fad63ece0c0, "%s: ", "hello"hello: )= 7
vfprintf(0x7fad63ece0c0, "%s", 0x7ffdcb755230write error)  = 11
putc(10, 0x7fad63ece0c0, 0x, 0
) = 10
fflush(0x7fad63ece0c0)  = 0
_exit(1 
+++ exited (status 1) +++

On glibc write() happens in atexit:

$ strace ./hello
...
newfstatat(1, "", {st_mode=S_IFCHR|0666, st_rdev=makedev(0x1, 0x7), ...}, 

Re: hello-2.11 fails multibyte-1 and atexit-1 on musl

2022-01-27 Thread Reuben Thomas
On Wed, 26 Jan 2022 at 12:59, Sergei Trofimovich  wrote:
>

Thanks for the report!

> From what I understand at
> 
> https://wiki.musl-libc.org/functional-differences-from-glibc.html#Character-sets-and-locale
> musl deliberately supports any char -> wchar_t (and back) conversion for
> any char in LC_CTYPE=C. And thus the multibyte-1 test will always fail
> there.

I also found this problem on macOS. I tried on my GNU/Linux system
using an ISO-8859 locale for the test, but the multibyte conversion
does not fail! (e.g. in en_GB.iso88591). I also tried with
ru_RU.KOI8-R, same result.

It seems then that it's not going to be possible in general to find a
locale that will fail. These results suggest that on some libc's,
conversion will always succeed.

Hence, I'm removing the test.

> Not sure why atexit-1 fails. It seems to rely on
> close_stdout_set_file_name() call in atexit(), but I don't see where in
> code it's registered to call.

Line 155 registers close_stdout. I can't see anything wrong with the
way this is done; I'd be very grateful if you could investigate
further.

-- 
https://rrt.sc3d.org