Re: s6-test -v does not work for variable "!"

[Laurent Bercot]

 So there are several things.

 - "=" is not a legit envvar name: envvar names cannot include a '='
character. "!" is absolutely a legit envvar name, though, and execline
uses it with commands such as background or pipeline.

 - execline does not understand single quotes. '!' will translate into
the word "'!'". So, don't use single quotes in execline, ever. Only
double quotes.

 - It is a peculiarity, and *the* non-posix behaviour, of s6-test, that
when an argument starts with a backslash, the backslash is ignored,
and the remainder of an argument is used as data, and never as an
operator. s6-test uses this to disambiguate, instead of relying on the
total number of arguments (and doing unspecified, unpredictable things
when given more than 4 arguments). For instance,
"s6-test -r = -r" will fail with a parsing error (which is 
non-conforming

to the "test" specification), because there is a parsing ambiguity with
"-r =", and s6-test does not use the fact that there is another argument
behind "=" to lift the ambiguity.

 - "s6-test -v !" sounds like a simple case, but it's not, because "!"
is a valid operator for s6-test, and s6-test does not look past "!" to
see there are no further arguments and "!" cannot be an operator. So it
fails for the same reason described above. The way to avoid that is to
backslash the argument: "s6-test -v \!"

 - execline uses \ as a quoting character, so when writing an execline
script, you need to double the amount of backslashes that you want to
appear in your argv. Here, you want one backslash before the "!", so
you would write
  s6-test -v \\!
and that should work.

 - I know that the s6-test behaviour is non-conforming when you provide
fewer than 4 arguments, and is non-intuitive and annoying, especially
when "test" just works. But it is *mighty hard* to implement the POSIX
disambiguation method without exploding the code size. Doing it would
take some real serious effort and probably double the size of the 
s6-test

source code - and it would *still* need to provide a different way to
disambiguate for 5 arguments or more, because s6-test explicitly 
supports

an infinite number of arguments. (Or, rather, as many as fit into your
argv.) This is the very reason why I didn't bother making it a 
conforming

"test" implementation: the POSIX requirements require ad-hoc operator
priority rules that do not fit the shift-reduce engine *at all* and that
are super difficult to implement cleanly.

 - I wrote s6-test when I was first reading the Dragon Book on compiler
implementation, specifically the chapters on parsing, as an exercise to
check I understood how pushdown automata worked. s6-test and s6-expr are
prime examples of pushdown automata. So that is why 1. I needed a shift-
reduce disambiguation method that meshed well with the engine, which
the POSIX test disambiguation method is not, and 2. the code may not be
intuitive, because it's the most "computer science academia" thing I've
ever written. :)
 (Yes, that includes the execline/mdevd/... parsers. These parsers are
table-driven parsers, aka deterministic finite automata. That's 
*simple*.
The tables look intimidating, but the code itself is very 
straightforward.

Pushdown automata are a different story, you really need the theoretical
background to wrap your mind around them.)

 TL;DR: s6-test is not test, please use s6-test -v \\!
in your execline scripts, and basically prepend any argument that looks
like a possible test operator with two backslashes (because execline
eats one, and a shell would as well).

--
 Laurent

Re: s6-test -v does not work for variable "!"

[Songbo Wang]

On 9/5/22 22:15, alice wrote:

On Mon Sep 5, 2022 at 10:08 PM CEST, Songbo Wang wrote:


#!/usr/bin/execlineb -P
export ! 0
s6-test -v '\!'

changing this to

s6-test -v \\\!

works as expected.


Thanks for the reply. It indeed works. But there is one thing I find 
counter-intuitive: s6-test expects to see an escaped ! while export 
doesn't. For example this exits 1:


#!/usr/bin/execlineb -P
export \\\! 0
s6-test -v \\\!


quoting is different in execline, i think there was
more written in some documentation for the specifics, but i think it
needs to both escape the ! and the \ , so it's three levels of  >
as a bonus, that also works in normal shell (no quoting in both places)


--
Songbo Wang (汪嵩博)

Re: s6-test -v does not work for variable "!"

[cat æscling via skaware]

When the environment modification takes place with export is unintuitive; try

```
#!/usr/bin/execlineb -P
export ! 0
foreground { exit }
s6-test -v '\!'

```

Re: s6-test -v does not work for variable "!"

[alice]

On Mon Sep 5, 2022 at 10:08 PM CEST, Songbo Wang wrote:
> On 9/5/22 20:57, alice wrote
>  > s6-test -v '\!'
>  > seems to work for me, tested with
>  > env !=1 s6-test -v'\!'
>  > and without the env. maybe i'm misreading it?
>  >
> Your example works for me on bash, but I just tested by the following 
> execline script
>
> #!/usr/bin/execlineb -P
> exec -c s6-test -v !
>
> which gives a parse error to me. Double quoting the ! gives the same 
> error. Single quoting eliminates the error, but doesn't seem to work 
> correctly:
>
> #!/usr/bin/execlineb -P
> export ! 0
> s6-test -v '!'
>
> exits 1. Backslashing ! doesn't work, double quoted or not. Single quote 
> works, but same
>
>
> #!/usr/bin/execlineb -P
> export ! 0
> s6-test -v '\!'
changing this to

s6-test -v \\\!

works as expected. quoting is different in execline, i think there was
more written in some documentation for the specifics, but i think it
needs to both escape the ! and the \ , so it's three levels of \

as a bonus, that also works in normal shell (no quoting in both places)

>
> exits 1.
>
> I wonder if I am stuck in some intricate no-go land of execline or 
> s6-test...
> -- 
> Songbo Wang (汪嵩博)

Re: s6-test -v does not work for variable "!"

[Songbo Wang]

On 9/5/22 20:57, alice wrote
> s6-test -v '\!'
> seems to work for me, tested with
> env !=1 s6-test -v'\!'
> and without the env. maybe i'm misreading it?
>
Your example works for me on bash, but I just tested by the following 
execline script


#!/usr/bin/execlineb -P
exec -c s6-test -v !

which gives a parse error to me. Double quoting the ! gives the same 
error. Single quoting eliminates the error, but doesn't seem to work 
correctly:


#!/usr/bin/execlineb -P
export ! 0
s6-test -v '!'

exits 1. Backslashing ! doesn't work, double quoted or not. Single quote 
works, but same



#!/usr/bin/execlineb -P
export ! 0
s6-test -v '\!'

exits 1.

I wonder if I am stuck in some intricate no-go land of execline or 
s6-test...

--
Songbo Wang (汪嵩博)

Re: s6-test -v does not work for variable "!"

[alice]

On Mon Sep 5, 2022 at 8:42 PM CEST, Songbo Wang wrote:
> Hi,
>
> I've been recently playing with execline scripts and found out that
> s6-test -v !
s6-test -v '\!'
seems to work for me, tested with
env !=1 s6-test -v'\!'
and without the env. maybe i'm misreading it?

> is not able to test if the ! environment variable is set. I guess it is 
> because s6-test would also need to handle expressions like "x != y", but 
> the code is too mystic for me to understand (by the way, s6-test fails 
> to test = too, but I don't even know if it is a legit envvar name).
>
> I think it is reasonable to use ! as an envvar, as it is set by some 
> execline utilities, e.g. background, wait.
>
> Cheers,
> -- 
> Songbo Wang (汪嵩博)

s6-test -v does not work for variable "!"

[Songbo Wang]

Hi,

I've been recently playing with execline scripts and found out that
s6-test -v !
is not able to test if the ! environment variable is set. I guess it is 
because s6-test would also need to handle expressions like "x != y", but 
the code is too mystic for me to understand (by the way, s6-test fails 
to test = too, but I don't even know if it is a legit envvar name).


I think it is reasonable to use ! as an envvar, as it is set by some 
execline utilities, e.g. background, wait.


Cheers,
--
Songbo Wang (汪嵩博)

Test

[Laurent Bercot]

 Test message, please ignore. Apologies for the noise.

--
 Laurent

Re: False positive in skalibs system feature test

[Jens Rehsack]

> Am 26.10.2019 um 03:53 schrieb Guillermo :
> 
> El vie., 25 oct. 2019 a las 17:01, Jens Rehsack escribió:
>> 
>>> [...]
>>> configure:2634: result: no
>>> 
>>> As you can see, only the equivalent of a skarnet.org 'choose cl' is used 
>>> here.
>> 
>> Wasn't that clear enough when I told that weeks before?
>> For any typical library function, that is enough.
> 
> In cases like this, not without precautions.

Wouldn't you please remove parts of the statement?

> This only worked because
> Autoconf happens to know about GNU libc's __stub_* macros, and adds
> garbage to the test source file if the relevant one is defined, so
> that the compile phase fails. If the compile phase had succeeded, the
> link phase would have as well. The configure script would have
> declared that lchown() is available, and one would have ended up with
> a useless lchown() substitute.

Uh - like the handling for __builtin_* stuff of clang.

But there is no need to do that, checking for SYS_getrandom tells
you about the syscall.

Anyway, the knowledge about __sub_* tells whether the libc configure
stage was done correctly.

Probably it's reasonable to do both ... check whether it's no stub
and check whether the syscall is available.

Cheers
--
Jens Rehsack - rehs...@gmail.com



signature.asc
Description: Message signed with OpenPGP

Re: False positive in skalibs system feature test

[Guillermo]

El vie., 25 oct. 2019 a las 17:01, Jens Rehsack escribió:
>
> > [...]
> > configure:2634: result: no
> >
> > As you can see, only the equivalent of a skarnet.org 'choose cl' is used 
> > here.
>
> Wasn't that clear enough when I told that weeks before?
> For any typical library function, that is enough.

In cases like this, not without precautions. This only worked because
Autoconf happens to know about GNU libc's __stub_* macros, and adds
garbage to the test source file if the relevant one is defined, so
that the compile phase fails. If the compile phase had succeeded, the
link phase would have as well. The configure script would have
declared that lchown() is available, and one would have ended up with
a useless lchown() substitute.

G.

Re: False positive in skalibs system feature test

[Guillermo]

El vie., 25 oct. 2019 a las 14:30, Shengjing Zhu escribió:
>
> Not familiar with autoconf, but I found the following snippet in autoconf 
> code.
>
> https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/c.m4?h=v2.69#n179
>
> ```
> #if defined __stub_$1 || defined __stub___$1
> choke me
> #endif

This seems to be indeed what Autoconf currently uses when the
AC_CHECK_FUNC macro is used to check if a function is available. The
__stub_* macros are in  (which on my Gentoo x86_64 system
includes ), an autogenerated GNU libc header included
by . I tried this with getrandom(), which exists, and
lchmod(), which is a libc stub. Here are the results:

$ nm -D /lib64/libc.so.6 | grep -E 'lchmod|getrandom'
0003d980 T getrandom
000f5aa0 T lchmod

$ cat /usr/include/gnu/stubs-64.h
/* This file is automatically generated.
   It defines a symbol `__stub_FUNCTION' for each function
   in the C library which is a stub, meaning it will fail
   every time called, usually setting errno to ENOSYS.  */
#ifdef _LIBC
 #error Applications may not define the macro _LIBC
#endif
[...]
#define __stub_lchmod
[...]

$ cat configure.ac
AC_INIT(example, 1.0)
AC_CHECK_FUNC(getrandom)
AC_CHECK_FUNC(lchmod)
AC_OUTPUT

$ autoconf -V
autoconf (GNU Autoconf) 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+/Autoconf: GNU GPL version 3 or later
, 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by David J. MacKenzie and Akim Demaille.

$ autoconf
$ ./configure
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for getrandom... yes
checking for lchmod... no
configure: creating ./config.status

$ cat config.log
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by example configure 1.0, which was
generated by GNU Autoconf 2.69.
[...]
configure:2634: checking for lchmod
configure:2634: gcc -o conftest -g -O2   conftest.c  >&5
conftest.c:37:1: error: unknown type name 'choke'
 choke me
 ^
conftest.c:37:9: error: expected ';' before 'int'
 choke me
 ^
 ;
conftest.c:40:1:
 int
 ~~~
configure:2634: $? = 1
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "example"
| #define PACKAGE_TARNAME "example"
| #define PACKAGE_VERSION "1.0"
| #define PACKAGE_STRING "example 1.0"
| #define PACKAGE_BUGREPORT ""
| #define PACKAGE_URL ""
| /* end confdefs.h.  */
| /* Define lchmod to an innocuous variant, in case  declares lchmod.
|For example, HP-UX 11i  declares gettimeofday.  */
| #define lchmod innocuous_lchmod
|
| /* System header to define __stub macros and hopefully few prototypes,
| which can conflict with char lchmod (); below.
| Prefer  to  if __STDC__ is defined, since
|  exists even on freestanding compilers.  */
|
| #ifdef __STDC__
| # include 
| #else
| # include 
| #endif
|
| #undef lchmod
|
| /* Override any GCC internal prototype to avoid an error.
|Use char because int might match the return type of a GCC
|builtin and then its argument prototype would still apply.  */
| #ifdef __cplusplus
| extern "C"
| #endif
| char lchmod ();
| /* The GNU C library defines this for functions which it implements
| to always fail with ENOSYS.  Some functions are actually named
| something starting with __ and the normal name is an alias.  */
| #if defined __stub_lchmod || defined __stub___lchmod
| choke me
| #endif
|
| int
| main ()
| {
| return lchmod ();
|   ;
|   return 0;
| }
configure:2634: result: no

As you can see, only the equivalent of a skarnet.org 'choose cl' is used here.

G.

Re: False positive in skalibs system feature test

[Laurent Bercot]

Not familiar with autoconf, but I found the following snippet in autoconf code.

https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/c.m4?h=v2.69#n179

```
#if defined __stub_$1 || defined __stub___$1
choke me
#endif
```

I checked on GNU Hurd, __stub_getrandom is defined.


 Good finding! That seems like the right workaround.
 I mean, no, it relies on an internal ftm, so it's definitely ugly,
and glibc should be fixed, but in the meantime, if that's good enough
for autoconf, that's good enough for me.

 Thanks,

--
 Laurent

Re: False positive in skalibs system feature test

[Shengjing Zhu]

On Fri, Oct 25, 2019 at 5:48 AM Laurent Bercot  wrote:
> Could anyone please:
> * investigate how autoconf deals with it?

Not familiar with autoconf, but I found the following snippet in autoconf code.

https://git.savannah.gnu.org/cgit/autoconf.git/tree/lib/autoconf/c.m4?h=v2.69#n179

```
#if defined __stub_$1 || defined __stub___$1
choke me
#endif
```

I checked on GNU Hurd, __stub_getrandom is defined.

-- 
Shengjing Zhu

Re: False positive in skalibs system feature test

[Jens Rehsack]

> Am 24.10.2019 um 23:41 schrieb Laurent Bercot :
> 
> 
> Excellent analysis, as always. :) Thank you.
> 
> 
>> * If GNU libc is used, a 'choose cl' test is unreliable.
>> * GNU/Hurd and GNU/k*BSD would be 'getrandom: no' OSes, at least until
>> the libc is updated to make a system call on GNU/kFreeBSD [1]
>> * A 'choose clr' test would be reliable.
>> * Any other alternative (passing an LDFLAGS that produces a build
>> failure, overriding the result of tests, or whatever) kind of implies
>> knowledge about what the outcome of the detection is going to be,
>> anyway, doesn't it? It's not really a detection.
> 
> - I will temporarily make it a 'choose clr' test.
> - However, I think this is a misdesign in the glibc, that accomplishes
> nothing else than break autodetection. As such, it should be reported
> to the glibc team.
> - Either this misdesign breaks autoconf too, or autoconf has a way of
> working around it. (Or nobody ever tests for getrandom(), but I doubt
> it - some configure scripts test for strcmp(), so, yeah.)

strcmp is C99 - and it's not "sometimes surprisingly unimplemented".

> - I have no time to take care of the real issue for at least a week.
> Could anyone please:
>   * investigate how autoconf deals with it?

I digged a bit for that. As said, generally autoconf doesn't deal with
glibc or Linux specialities (it does in 90s ... - that causes more trouble
than it solves).

Another project using getrandom() does the run test, too:
* https://github.com/zeromq/libzmq/pull/3085

A hint to LibreSSL in https://lwn.net/Articles/800509/ delivered a reasonable
answer:
sno@beaker:~/prj/libressl-3.0.2$ grep -Rl getrandom *
ChangeLog
crypto/compat/getentropy_linux.c
sno@beaker:~/prj/libressl-3.0.2$ less crypto/compat/getentropy_linux.c
...
#if defined(SYS_getrandom) && defined(GRND_NONBLOCK)
static int getentropy_getrandom(void *buf, size_t len);
#endif
...
sno@beaker:~/prj/libressl-3.0.2$ grep -Rl SYS_getrandom *
crypto/compat/getentropy_linux.c
sno@beaker:~/prj/libressl-3.0.2$ grep -Rl SYS_getrandom /usr/include/
/usr/include/bits/syscall.h
/usr/include/x86_64-linux-gnu/bits/syscall.h
/usr/include/x86_64-linux-gnu/expat_config.h

Seems, there isn't much need in probing at configure stage ...

>   * report the thing to libc-alpha?
> 
> Thanks in advance,

Hope it helps a little bit.

Cheers
--
Jens Rehsack - rehs...@gmail.com



signature.asc
Description: Message signed with OpenPGP

Re: False positive in skalibs system feature test

[Laurent Bercot]

 Excellent analysis, as always. :) Thank you.



* If GNU libc is used, a 'choose cl' test is unreliable.
* GNU/Hurd and GNU/k*BSD would be 'getrandom: no' OSes, at least until
the libc is updated to make a system call on GNU/kFreeBSD [1]
* A 'choose clr' test would be reliable.
* Any other alternative (passing an LDFLAGS that produces a build
failure, overriding the result of tests, or whatever) kind of implies
knowledge about what the outcome of the detection is going to be,
anyway, doesn't it? It's not really a detection.


 - I will temporarily make it a 'choose clr' test.
 - However, I think this is a misdesign in the glibc, that accomplishes
nothing else than break autodetection. As such, it should be reported
to the glibc team.
 - Either this misdesign breaks autoconf too, or autoconf has a way of
working around it. (Or nobody ever tests for getrandom(), but I doubt
it - some configure scripts test for strcmp(), so, yeah.)
 - I have no time to take care of the real issue for at least a week.
Could anyone please:
   * investigate how autoconf deals with it?
   * report the thing to libc-alpha?

 Thanks in advance,

--
 Laurent

Re: False positive in skalibs system feature test

[Guillermo]

El mié., 23 oct. 2019 a las 6:50, Laurent Bercot escribió:
>
> >/usr/bin/ld: src/librandom/random_string.lo: in function `random_string':
> >./src/librandom/random_string.c:26: warning: getrandom is not
> >implemented and will always fail
> [...]
> >ld returns zero, even it knows "xxx is not implemented and will always fail".
>
> That, on the other hand, is completely insane. Why the heck isn't it
> an error? It's *precisely* the kind of thing I want ld to error out on!
> [...]
>
> >I only observe such case on GNU Hurd/K*BSD actually. I'm thinking if
> >only these non-linux systems have such confused behaviour(why it links
> >successfully at all...)
>
>   That would be interesting to explore, yes.

I also think that this is a GNU libc thing. As far as I can tell, it
contains a 'stub' getrandom() that does nothing, returns -1, and sets
errno to ENOSYS, unless it is built for an OS that can provide a
'real' getrandom():

* 
https://sourceware.org/git/?p=glibc.git;a=blob;f=stdlib/getrandom.c;h=9fe6dda34d3f7fe99aaa66fc3a8ad4e540494c4a;hb=56c86f5dd516284558e106d04b92875d5b623b7a

In the upstream libc package, this seems to happen only for Linux,
where it is just a wrapper around the corresponding system call:

* 
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/getrandom.c;h=9d3d919c748d10c7debb6836575c3c44afa0d707;hb=56c86f5dd516284558e106d04b92875d5b623b7a

I would think that for 'pure' GNU (Hurd) and GNU/k*BSD there is no
'real' getrandom() implementation, so the libc gets the 'stub' one,
although the kernel of FreeBSD 12.0, has a getrandom() system call
that the libc could use.

* 
https://svnweb.freebsd.org/base/stable/12/sys/sys/syscall.h?revision=351565=markup

That "stub_warning()" macro turns out to be a GCC trick to display the
"is not implemented and will always fail" message: if an ELF object
file contains .gnu.warning. section, GNU ld knows what it
means, shows a warning, removes the section in the resulting file, and
returns 0, because an excecutable is still produced, although with an
implementation of getrandom() that is not useful for skalibs :P Here's
an example of the trick:

$ cat libtest.c
#include 
#include 

int test_function() {
   fprintf (stderr, "test_function(): I told you! I don't work!\n");
   return errno = ENOSYS, -1;
}

//Magic, create a .gnu.warning.test_function ELF section
static char message[] __attribute__ ((used,
section(".gnu.warning.test_function\n\t#")))
   = "libtest: I provide a \"test_function\" symbol, but I don't
actually implement the function";

$ gcc -shared -Wl,-soname=libtest.so -o libtest.so -fPIC -DPIC libtest.c
$ objdump -h libtest.so | grep .gnu.warning
 23 .gnu.warning.test_function 0059  
  3060  2**5

$ cat test-program.c
#include 
#include 
#include 

int test_function();

int main() {
   if (test_function()) fprintf (stderr, "test-program: ERROR: %s\n",
strerror(errno));
}

$ gcc -o test-program -Wl,-rpath=/home/guillermo -L. test-program.c -ltest
/usr/bin/ld: /tmp/ccu3uP4m.o: in function `main':
test-program.c:(.text+0xa): warning: libtest: I provide a
"test_function" symbol, but I don't actually implement the function

$ objdump -h test-program | grep .gnu.warning
$ ./test-program
test_function(): I told you! I don't work!
test-program: ERROR: Function not implemented

So, it seems that,:for the purposes of detecting whether there is a
usable getrandom(),for skallibs:

* If GNU libc is used, a 'choose cl' test is unreliable.
* GNU/Hurd and GNU/k*BSD would be 'getrandom: no' OSes, at least until
the libc is updated to make a system call on GNU/kFreeBSD [1]
* A 'choose clr' test would be reliable.
* Any other alternative (passing an LDFLAGS that produces a build
failure, overriding the result of tests, or whatever) kind of implies
knowledge about what the outcome of the detection is going to be,
anyway, doesn't it? It's not really a detection.

[1] Is Debian the upstream of the libc port to GNU/kFreeBSD?

G.

Re: False positive in skalibs system feature test

[Dewayne Geraghty]

Thankyou Laurent.

So cat /usr/ports/devel/skalibs/work/skalibs-2.9.1.0/sysdeps.cfg/sysdeps
reveals
getrandom: yes

Now adding --with-sysdep-getrandom=no to the configure line, we get from:
/usr/ports/devel/skalibs/work/skalibs-2.9.1.0/sysdeps.cfg/sysdeps
getrandom: no

and we obtained a 211K stripped image libskarnet.so.2.9.1.0

This on FreeBSD 12.1 using clang 8.0.1 and ld 2.32 (note not the default
lld).  I hope that helps?

However there were some (common) warning messages:
> src/libstddjb/child_spawn1_internal.c:35:23: warning: & has lower
precedence than !=; != will be evaluated first [-Wparentheses]
>   if (p[to & 1] != to & 1)
>   ^
> src/libstddjb/child_spawn1_internal.c:35:23: note: place parentheses
around the '!=' expression to silence this warning
>   if (p[to & 1] != to & 1)
>   ^
>   (  )
> src/libstddjb/child_spawn1_internal.c:35:23: note: place parentheses
around the & expression to evaluate it first
>   if (p[to & 1] != to & 1)
>   ^
>( )
> 1 warning generated.

Re: False positive in skalibs system feature test

[Laurent Bercot]

While building skalibs-2.9.1.0, I notice that the getrandom option isn't
available on FreeBSD 12.1 Stable.  What I have from ./configure --help is:

"...
Sysdeps autodetection override:
  --with-sysdep-K=V assume sysdep K has the value V
[autodetected]
List of mandatory K for cross-compiling: devurandom (V=yes|no)
"


It won't be listed in the help message, but when you look at the
sysdeps file after a compilation, you will see a "getrandom: no" line.
All the keys in that file can be overriden by a configure option on
the model given by the help message.

--
Laurent

Re: False positive in skalibs system feature test

[Dewayne Geraghty]

While building skalibs-2.9.1.0, I notice that the getrandom option isn't
available on FreeBSD 12.1 Stable.  What I have from ./configure --help is:

"...
Sysdeps autodetection override:
  --with-sysdep-K=V assume sysdep K has the value V
[autodetected]
List of mandatory K for cross-compiling: devurandom (V=yes|no)
"

To your question Laurent about what other (non-Linux) people use, as FYI
normal FreeBSD people (on FreeBSD 12.1Stable) use
for cc, clang 8.0.1
# ld -v
LLD 8.0.1 ; an LLVM-based linker

For dev/testing I also use gcc 9.2.0 and ld (GNU Binutils) 2.32 as needed.

On HardenedBSD which uses LLVM tooling, there are additional flags like
-fPIC -fPIE and for linking -pie -z relro -z now; and optional settings
for CFI but that's becoming esoteric. ;)

Cheers, Dewayne.

Re: False positive in skalibs system feature test

[Jens Rehsack]

> Am 23.10.2019 um 20:19 schrieb Laurent Bercot :
> 
>> Overridable for the win ;)
> 
> Are we giving advice again without checking first, hmmm?
> 
> ./configure --with-sysdep-getrandom=no  <-- Try it. :P

Planned in for tomorrow ;)
Finished all other planned tasks so I have some time for that

But I cross-compile for linux only, so especially that one shouldn't
become problematic :P

> My point is that ld *should* be a reliable way to autodetect those
> things, and I want to make sure it works too.

When the entry point is really missing.
When the entry point is there but there is something weird happening
when jumping to it, or it simply points to a "return -1" stub combined
with a ld-warning, than it's not ld's fault.

It's just a smell ...

A look using `objdump -G /path/to/libc.so` should tell us.

A short look into
* binutils-cross-arm/2.32.0-r0/git/ld/testsuite/ld-cris/globsymw1.s
* binutils-cross-arm/2.32.0-r0/git/ld/testsuite/ld-cris/warn3.d
* binutils-cross-arm/2.32.0-r0/git/ld/testsuite/ld-cris/globsymw2.s
* binutils-cross-arm/2.32.0-r0/git/ld/testsuite/ld-cris/warn4.d
confirms the smell ...

Cheers
--
Jens Rehsack - rehs...@gmail.com



signature.asc
Description: Message signed with OpenPGP

Re: False positive in skalibs system feature test

[Laurent Bercot]

Overridable for the win ;)


 Are we giving advice again without checking first, hmmm?

 ./configure --with-sysdep-getrandom=no  <-- Try it. :P

 My point is that ld *should* be a reliable way to autodetect those
things, and I want to make sure it works too.

--
 Laurent

Re: False positive in skalibs system feature test

[Jens Rehsack]

> Am 23.10.2019 um 11:50 schrieb Laurent Bercot :
> 
>> /usr/bin/ld: src/librandom/random_string.lo: in function `random_string':
>> ./src/librandom/random_string.c:26: warning: getrandom is not
>> implemented and will always fail
> 
> And this, ladies and gentlemen, is why 'choose clr' is more reliable
> than 'choose cl'.
> Unfortunately, the former isn't cross-friendly, while the latter is.
> And *some* people insisted on making the build more cross-friendly, so
> here we are.

Choosing cl over clr is not the only option to make something cross-friendly.
Making the command overrideable to use qemu to execute a test or making
each test result overridable is cross-friendly, too.

Do not blame requests for more cross-friendlyness to choose wrong weapon ;)

>> ld returns zero, even it knows "xxx is not implemented and will always fail".
> 
> That, on the other hand, is completely insane. Why the heck isn't it
> an error? It's *precisely* the kind of thing I want ld to error out on!
> 
> 
>> I think the LDFLAGS should be appended with "-Wl,--fatal-warnings", so
>> that ld returns non-zero in such case.
> 
> Yeah, so we have to bully ld for it to do its job. Nice.
> Are there cases where we could get a false negative, though? Are there
> legitimate ld warnings that _should not_ error out? I don't want to make
> a change to accommodate an exotic system if it risks breaking others.

Overridable for the win ;)

>> I only observe such case on GNU Hurd/K*BSD actually. I'm thinking if
>> only these non-linux systems have such confused behaviour(why it links
>> successfully at all...)
> 
> That would be interesting to explore, yes. What version of ld do they
> they use? Is that GNU binutils or something else? And it may be more
> beneficial align ld's behaviour to other systems: skalibs can't be the
> only piece of software that relies on ld to test sysdeps. Starting with
> autoconf. Hm, does autoconf use -Wl,--fatal-warnings ?

Not by default.

But the issue smells more as a glibc issue than an ld issue.
Just the first shot I'd follow when encountering such a one ...

Cheers
--
Jens Rehsack - rehs...@gmail.com



signature.asc
Description: Message signed with OpenPGP

Re: False positive in skalibs system feature test

[Laurent Bercot]

/usr/bin/ld: src/librandom/random_string.lo: in function `random_string':
./src/librandom/random_string.c:26: warning: getrandom is not
implemented and will always fail


And this, ladies and gentlemen, is why 'choose clr' is more reliable
than 'choose cl'.
Unfortunately, the former isn't cross-friendly, while the latter is.
And *some* people insisted on making the build more cross-friendly, so
here we are.



ld returns zero, even it knows "xxx is not implemented and will always fail".


That, on the other hand, is completely insane. Why the heck isn't it
an error? It's *precisely* the kind of thing I want ld to error out on!



I think the LDFLAGS should be appended with "-Wl,--fatal-warnings", so
that ld returns non-zero in such case.


Yeah, so we have to bully ld for it to do its job. Nice.
Are there cases where we could get a false negative, though? Are there
legitimate ld warnings that _should not_ error out? I don't want to make
a change to accommodate an exotic system if it risks breaking others.



I only observe such case on GNU Hurd/K*BSD actually. I'm thinking if
only these non-linux systems have such confused behaviour(why it links
successfully at all...)


 That would be interesting to explore, yes. What version of ld do they
they use? Is that GNU binutils or something else? And it may be more
beneficial align ld's behaviour to other systems: skalibs can't be the
only piece of software that relies on ld to test sysdeps. Starting with
autoconf. Hm, does autoconf use -Wl,--fatal-warnings ?

--
 Laurent

False positive in skalibs system feature test

[Shengjing Zhu]

Hi,

With recent change of skalibs configure script, there's a false positive case.

`choose l` use linker to test function if exists. But actually I observe that,

```
/usr/bin/ld: src/librandom/random_string.lo: in function `random_string':
./src/librandom/random_string.c:26: warning: getrandom is not
implemented and will always fail
```

ld returns zero, even it knows "xxx is not implemented and will always fail".

I think the LDFLAGS should be appended with "-Wl,--fatal-warnings", so
that ld returns non-zero in such case.

I only observe such case on GNU Hurd/K*BSD actually. I'm thinking if
only these non-linux systems have such confused behaviour(why it links
successfully at all...)

-- 
Shengjing Zhu

Re: Call for contributions: test suite

[Dreamcat4]

Hi,
If you make the lifelong habit to write all of your code in a certain way.
Then it will not be ambiguous and hard to understand. Then looking at it
you can see what it is supposed to do. There are other things you can do to
avoid testing too.

When testing is really needed is for certain kinds of projects. Military
Spec. Aircraft, compliance and other kinds of embedded and safety critical
systems. In those fields, the software code itself should be written (at
significantly higher cost) to be 100% verifiable. So then it is treated
somewhat like a state machine / turing machine. Same for VHBL and FPGA /
other hardware designs.

In those cases too, at least the integration testing must always be done
separately. Not by the same software team, or with the same software tools.

So yes - you can write unit tests. But it is largely a duplication of your
own code and isn't guaranteed to prove anything except in the situation
where your APIs are stable. I.e. for large quantities of what is basically
only ever going to be regression testing. (previously implemented
features). That is more needed in large-codebase and multi-developer (or
multi-team) environments. Where one hand may not know it is covering /
breaking the other.

The take away point of all that ^^. Is that such testing typically cannot
help you catch bugs in new features. So it's value is actually somewhat
eroded / less costs are saved vs time spend coding the unit tests. In other
words - you get more bang-for-buck directly fixing bugs in the existing
code that's there or writing entirely new code / new features.

And in software developer time is high cost… so nothing surprising really.


On Sun, Mar 29, 2015 at 10:23 PM, Laurent Bercot ska-skaw...@skarnet.org
wrote:

 On 2015-03-28 15:36, Lorenzo wrote:

 It's been a month, and no replies... but hey, please contact me,
 does it mean there's been some progress in private discussions?


  I wish. But unfortunately, it looks like testing is as unappealing
 to other people as it is to me. :)
  (Not that I blame anyone. I can't be bothered to write tests for
 my own freakin' projects, so why would it be surprising that nobody
 wants to write tests for a project they don't even own ? It's really
 a case of there are all kind of fetishes in the world and I'm hoping
 to find somebody with a compatible one. Which is a numbers game,
 and we don't have the numbers yet.)


  If not, I've spent a few hours looking at possible candidates that
 match your criteria(*), and some candidates actually survived.


  Awesome! I'm all ears.


  Please tell me that someone mailed you in private anyway :)


  The silence in my mailbox was deafening. The community is still too
 small so the odds are against me. :)
  I hope to make the community larger with the projects I'm currently
 working on; I can still keep things under control until then at least.

 --
  Laurent

[TEST] Please ignore...

[Jorge Almeida]

...unless you're Laurent.