Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On Sep 11, 2023, at 00:03, Mark Millard wrote:
> On Sep 10, 2023, at 23:57, Dag-Erling Smørgrav wrote:
>
>> Mark Millard writes:
>>> I'm not aware of there being other documentation for what
>>> is appropriate for setting up such for kyua runs.
>>
>> https://github.com/freebsd/freebsd-ci/blob/master/scripts/build/build-test_image-head.sh#L69-L84
>>
>
> Thanks for the reference that does not involve looking at
> CI log files. Filed away for future references.
>
>
> Side note . . .
>
> Turns out that tcptestsuite does not build for aarch64
> do to alignment problems via packing in net/packetdrill :
>
> In file included from run_packet.c:45:
> In file included from ./tcp_options_iterator.h:31:
> ./tcp_options.h:108:2: error: field within 'struct tcp_option' is less
> aligned than 'union tcp_option::(anonymous at ./tcp_options.h:108:2)' and is
> usually due to 'struct tcp_option' being packed, which can lead to unaligned
> accesses [-Werror,-Wunaligned-access]
> union {
> ^
> --- sctp_iterator.o ---
> cc -O2 -pipe -mcpu=cortex-a7
Looks like I messed up and reported an armv7 context.
aarch64 built net/packetdrill and net/tcptestsuite just
fine. Sorry for the noise.
> -Wno-deprecated -g -fstack-protector-strong -fno-strict-aliasing
> -mcpu=cortex-a7 -Wall -Werror -g -c sctp_iterator.c -o sctp_iterator.o
> --- tcp_options.o ---
> cc -O2 -pipe -mcpu=cortex-a7 -Wno-deprecated -g -fstack-protector-strong
> -fno-strict-aliasing -mcpu=cortex-a7 -Wall -Werror -g -c tcp_options.c -o
> tcp_options.o
> --- run_packet.o ---
> 1 error generated.
> *** [run_packet.o] Error code 1
>
> make[1]: stopped in
> /wrkdirs/usr/ports/net/packetdrill/work/packetdrill-aebdc35/gtests/net/packetdrill
> --- tcp_options.o ---
> In file included from tcp_options.c:25:
> ./tcp_options.h:108:2: error: field within 'struct tcp_option' is less
> aligned than 'union tcp_option::(anonymous at ./tcp_options.h:108:2)' and is
> usually due to 'struct tcp_option' being packed, which can lead to unaligned
> accesses [-Werror,-Wunaligned-access]
> union {
> ^
> 1 error generated.
> *** [tcp_options.o] Error code 1
>
> make[1]: stopped in
> /wrkdirs/usr/ports/net/packetdrill/work/packetdrill-aebdc35/gtests/net/packetdrill
> 2 errors
>
===
Mark Millard
marklmi at yahoo.com
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On Sep 10, 2023, at 23:57, Dag-Erling Smørgrav wrote:
> Mark Millard writes:
>> I'm not aware of there being other documentation for what
>> is appropriate for setting up such for kyua runs.
>
> https://github.com/freebsd/freebsd-ci/blob/master/scripts/build/build-test_image-head.sh#L69-L84
>
Thanks for the reference that does not involve looking at
CI log files. Filed away for future references.
Side note . . .
Turns out that tcptestsuite does not build for aarch64
do to alignment problems via packing in net/packetdrill :
In file included from run_packet.c:45:
In file included from ./tcp_options_iterator.h:31:
./tcp_options.h:108:2: error: field within 'struct tcp_option' is less aligned
than 'union tcp_option::(anonymous at ./tcp_options.h:108:2)' and is usually
due to 'struct tcp_option' being packed, which can lead to unaligned accesses
[-Werror,-Wunaligned-access]
union {
^
--- sctp_iterator.o ---
cc -O2 -pipe -mcpu=cortex-a7 -Wno-deprecated -g -fstack-protector-strong
-fno-strict-aliasing -mcpu=cortex-a7 -Wall -Werror -g -c sctp_iterator.c -o
sctp_iterator.o
--- tcp_options.o ---
cc -O2 -pipe -mcpu=cortex-a7 -Wno-deprecated -g -fstack-protector-strong
-fno-strict-aliasing -mcpu=cortex-a7 -Wall -Werror -g -c tcp_options.c -o
tcp_options.o
--- run_packet.o ---
1 error generated.
*** [run_packet.o] Error code 1
make[1]: stopped in
/wrkdirs/usr/ports/net/packetdrill/work/packetdrill-aebdc35/gtests/net/packetdrill
--- tcp_options.o ---
In file included from tcp_options.c:25:
./tcp_options.h:108:2: error: field within 'struct tcp_option' is less aligned
than 'union tcp_option::(anonymous at ./tcp_options.h:108:2)' and is usually
due to 'struct tcp_option' being packed, which can lead to unaligned accesses
[-Werror,-Wunaligned-access]
union {
^
1 error generated.
*** [tcp_options.o] Error code 1
make[1]: stopped in
/wrkdirs/usr/ports/net/packetdrill/work/packetdrill-aebdc35/gtests/net/packetdrill
2 errors
===
Mark Millard
marklmi at yahoo.com
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
Mark Millard writes: > I'm not aware of there being other documentation for what > is appropriate for setting up such for kyua runs. https://github.com/freebsd/freebsd-ci/blob/master/scripts/build/build-test_image-head.sh#L69-L84 DES -- Dag-Erling Smørgrav - d...@freebsd.org
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On Sep 10, 2023, at 11:21, Warner Losh wrote:
> On Sun, Sep 10, 2023, 11:10 AM Mark Millard wrote:
>> On Sep 10, 2023, at 00:31, Mark Millard wrote:
>>
>> > kyua tests that use the:
>> >
>> > /usr/tests/sys/cddl/zfs/bin/mkfile
>> >
>> > program like so (for example):
>> >
>> > mkfile 500M /testpool.1861/bigfile.0
>> >
>> > (which should be valid) end up with mkfile
>> > instead reporting:
>> >
>> > Standard error:
>> > Usage: mkfile [-nv] [e|p|t|g|m|k|b] ...
>> >
>> > which prevent the kyua test involved from working.
>> >
>> > Turns out this is from expecting char to be always
>> > signed (so a -1 vs. 255 distinction, here in an
>> > aarch64 context):
>> >
>> > . . .
>> > (gdb) list
>> > 179 /* Options. */
>> > 180 while ((ch = getopt(argc, argv, "nv")) != -1) {
>> > 181 switch (ch) {
>> > 182 case 'n':
>> > 183 nofill = 1;
>> > 184 break;
>> > 185 case 'v':
>> > (gdb) print ch
>> > $16 = 255 '\377'
>> > (gdb) print/x -1
>> > $17 = 0x
>> > (gdb) print/x ch
>> > $18 = 0xff
>> > . . .
>> >
>> > With the mix of unsigned and signed it ends up
>> > being a 0xffu != 0xu test, which is
>> > always true.
>> >
>> > So the switch is reached as if a "-" prefix was
>> > present (that is not). Then the "option" is classified
>> > as invalid and the usage message is produced.
>> >
>> > Apparently no one had noticed. That, in turn, suggests a
>> > lack of inspected testing on aarch64, powerpc64,
>> > powerpc64le, armv7, powerpc, and powerpcspe. That, in
>> > turn, suggests that kyua test inspection for the likes
>> > of aarch64 is not historically a part of the release
>> > process for openzfs or for operating systems that include
>> > openzfs.
>> >
>>
>> Looks like the mkfile.c traces back to a former port
>> sysutils/mkfile that was unfetchable as of 2019. And,
>> looking around, it seems the kyua zfs tests may be a
>> FreeBSD only thing, not adopted in openzfs.
>>
>> So various implicit assumptions when I wrote the note
>> do not actually hold.
>>
>> FreeBSD would have to do additional testing via kyua,
>> beyond what openzfs does for testing, to discover the
>> unsigned char related mis-behavior in the mkfile that
>> FreeBSD's kyua tests use. Only FreeBSD variants are
>> likely to have a similar status, not general openzfs
>> including operating systems.
>
> I wonder how hard ot would be to look for the char = getopt() pattern with
> coccinelle
>
Unsure.
But to be sure that the implication that I was also trying
to point out is not lost: kyua testing of zfs (and more?)
for aarch64 (tier 1) is apparently not being done (or at
least the results are not being inspected). Similarly for
armv7 and all the powerpc*'s (not tier 1's, however, so
not as surprising).
Side note:
Via other exchanges that have been going on I learned to
look in the likes of:
https://ci.freebsd.org/job/FreeBSD-main-amd64-testvm/*/consoleText
for what to "pkg install" for kyua test runs to use for
normal runs (at least the subset compatible with
architecture being tested). I'd only figured out a (large)
subset previously for aarch64 and armv7.
I'm not aware of there being other documentation for what
is appropriate for setting up such for kyua runs.
===
Mark Millard
marklmi at yahoo.com
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On Sun, Sep 10, 2023, 11:10 AM Mark Millard wrote:
> On Sep 10, 2023, at 00:31, Mark Millard wrote:
>
> > kyua tests that use the:
> >
> > /usr/tests/sys/cddl/zfs/bin/mkfile
> >
> > program like so (for example):
> >
> > mkfile 500M /testpool.1861/bigfile.0
> >
> > (which should be valid) end up with mkfile
> > instead reporting:
> >
> > Standard error:
> > Usage: mkfile [-nv] [e|p|t|g|m|k|b] ...
> >
> > which prevent the kyua test involved from working.
> >
> > Turns out this is from expecting char to be always
> > signed (so a -1 vs. 255 distinction, here in an
> > aarch64 context):
> >
> > . . .
> > (gdb) list
> > 179 /* Options. */
> > 180 while ((ch = getopt(argc, argv, "nv")) != -1) {
> > 181 switch (ch) {
> > 182 case 'n':
> > 183 nofill = 1;
> > 184 break;
> > 185 case 'v':
> > (gdb) print ch
> > $16 = 255 '\377'
> > (gdb) print/x -1
> > $17 = 0x
> > (gdb) print/x ch
> > $18 = 0xff
> > . . .
> >
> > With the mix of unsigned and signed it ends up
> > being a 0xffu != 0xu test, which is
> > always true.
> >
> > So the switch is reached as if a "-" prefix was
> > present (that is not). Then the "option" is classified
> > as invalid and the usage message is produced.
> >
> > Apparently no one had noticed. That, in turn, suggests a
> > lack of inspected testing on aarch64, powerpc64,
> > powerpc64le, armv7, powerpc, and powerpcspe. That, in
> > turn, suggests that kyua test inspection for the likes
> > of aarch64 is not historically a part of the release
> > process for openzfs or for operating systems that include
> > openzfs.
> >
>
> Looks like the mkfile.c traces back to a former port
> sysutils/mkfile that was unfetchable as of 2019. And,
> looking around, it seems the kyua zfs tests may be a
> FreeBSD only thing, not adopted in openzfs.
>
> So various implicit assumptions when I wrote the note
> do not actually hold.
>
> FreeBSD would have to do additional testing via kyua,
> beyond what openzfs does for testing, to discover the
> unsigned char related mis-behavior in the mkfile that
> FreeBSD's kyua tests use. Only FreeBSD variants are
> likely to have a similar status, not general openzfs
> including operating systems.
>
I wonder how hard ot would be to look for the char = getopt() pattern with
coccinelle
Warner
===
> Mark Millard
> marklmi at yahoo.com
>
>
>
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On Sep 10, 2023, at 00:31, Mark Millard wrote:
> kyua tests that use the:
>
> /usr/tests/sys/cddl/zfs/bin/mkfile
>
> program like so (for example):
>
> mkfile 500M /testpool.1861/bigfile.0
>
> (which should be valid) end up with mkfile
> instead reporting:
>
> Standard error:
> Usage: mkfile [-nv] [e|p|t|g|m|k|b] ...
>
> which prevent the kyua test involved from working.
>
> Turns out this is from expecting char to be always
> signed (so a -1 vs. 255 distinction, here in an
> aarch64 context):
>
> . . .
> (gdb) list
> 179 /* Options. */
> 180 while ((ch = getopt(argc, argv, "nv")) != -1) {
> 181 switch (ch) {
> 182 case 'n':
> 183 nofill = 1;
> 184 break;
> 185 case 'v':
> (gdb) print ch
> $16 = 255 '\377'
> (gdb) print/x -1
> $17 = 0x
> (gdb) print/x ch
> $18 = 0xff
> . . .
>
> With the mix of unsigned and signed it ends up
> being a 0xffu != 0xu test, which is
> always true.
>
> So the switch is reached as if a "-" prefix was
> present (that is not). Then the "option" is classified
> as invalid and the usage message is produced.
>
> Apparently no one had noticed. That, in turn, suggests a
> lack of inspected testing on aarch64, powerpc64,
> powerpc64le, armv7, powerpc, and powerpcspe. That, in
> turn, suggests that kyua test inspection for the likes
> of aarch64 is not historically a part of the release
> process for openzfs or for operating systems that include
> openzfs.
>
Looks like the mkfile.c traces back to a former port
sysutils/mkfile that was unfetchable as of 2019. And,
looking around, it seems the kyua zfs tests may be a
FreeBSD only thing, not adopted in openzfs.
So various implicit assumptions when I wrote the note
do not actually hold.
FreeBSD would have to do additional testing via kyua,
beyond what openzfs does for testing, to discover the
unsigned char related mis-behavior in the mkfile that
FreeBSD's kyua tests use. Only FreeBSD variants are
likely to have a similar status, not general openzfs
including operating systems.
===
Mark Millard
marklmi at yahoo.com
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On Sep 10, 2023, at 05:58, Mike Karels wrote:
> On 10 Sep 2023, at 2:31, Mark Millard wrote:
>
>> kyua tests that use the:
>>
>> /usr/tests/sys/cddl/zfs/bin/mkfile
>>
>> program like so (for example):
>>
>> mkfile 500M /testpool.1861/bigfile.0
>>
>> (which should be valid) end up with mkfile
>> instead reporting:
>>
>> Standard error:
>> Usage: mkfile [-nv] [e|p|t|g|m|k|b] ...
>>
>> which prevent the kyua test involved from working.
>>
>> Turns out this is from expecting char to be always
>> signed (so a -1 vs. 255 distinction, here in an
>> aarch64 context):
>>
>> . . .
>> (gdb) list
>> 179 /* Options. */
>> 180 while ((ch = getopt(argc, argv, "nv")) != -1) {
>> 181 switch (ch) {
>> 182 case 'n':
>> 183 nofill = 1;
>> 184 break;
>> 185 case 'v':
>> (gdb) print ch
>> $16 = 255 '\377'
>> (gdb) print/x -1
>> $17 = 0x
>> (gdb) print/x ch
>> $18 = 0xff
>> . . .
>>
>> With the mix of unsigned and signed it ends up
>> being a 0xffu != 0xu test, which is
>> always true.
>
> mkfile is broken. getopt returns an int, and -1 on end.
> It never returns 0xff. But mkfile declares ch as char,
> which truncates the return value -1. ch is a bad (misleading)
> variable name, although getopt(3) uses it as well (but declared
> as int).
Yep: for char being signed, the code is still wrong
via the char ch use. But the observed behavior is
very different than for char being used but being
unsigned. In this context, consequences of the
unsigned char behavioral results are observable in
the kyua run results but went unnoticed.
I used to run into examples of the use of unsigned
char for holding the getopt result back in my powerpc
days as well and dealt with upstreams for a port or 2
for getting it fixed after finding such was the source
of odd behavior I'd observed. If I remember right,
this is the first example of running into the specific
issue in my aarch64 and armv7 time frame.
> Mike
>
>> So the switch is reached as if a "-" prefix was
>> present (that is not). Then the "option" is classified
>> as invalid and the usage message is produced.
>>
>> Apparently no one had noticed. That, in turn, suggests a
>> lack of inspected testing on aarch64, powerpc64,
>> powerpc64le, armv7, powerpc, and powerpcspe. That, in
>> turn, suggests that kyua test inspection for the likes
>> of aarch64 is not historically a part of the release
>> process for openzfs or for operating systems that include
>> openzfs.
>
===
Mark Millard
marklmi at yahoo.com
Re: Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
On 10 Sep 2023, at 2:31, Mark Millard wrote:
> kyua tests that use the:
>
> /usr/tests/sys/cddl/zfs/bin/mkfile
>
> program like so (for example):
>
> mkfile 500M /testpool.1861/bigfile.0
>
> (which should be valid) end up with mkfile
> instead reporting:
>
> Standard error:
> Usage: mkfile [-nv] [e|p|t|g|m|k|b] ...
>
> which prevent the kyua test involved from working.
>
> Turns out this is from expecting char to be always
> signed (so a -1 vs. 255 distinction, here in an
> aarch64 context):
>
> . . .
> (gdb) list
> 179 /* Options. */
> 180 while ((ch = getopt(argc, argv, "nv")) != -1) {
> 181 switch (ch) {
> 182 case 'n':
> 183 nofill = 1;
> 184 break;
> 185 case 'v':
> (gdb) print ch
> $16 = 255 '\377'
> (gdb) print/x -1
> $17 = 0x
> (gdb) print/x ch
> $18 = 0xff
> . . .
>
> With the mix of unsigned and signed it ends up
> being a 0xffu != 0xu test, which is
> always true.
mkfile is broken. getopt returns an int, and -1 on end.
It never returns 0xff. But mkfile declares ch as char,
which truncates the return value -1. ch is a bad (misleading)
variable name, although getopt(3) uses it as well (but declared
as int).
Mike
> So the switch is reached as if a "-" prefix was
> present (that is not). Then the "option" is classified
> as invalid and the usage message is produced.
>
> Apparently no one had noticed. That, in turn, suggests a
> lack of inspected testing on aarch64, powerpc64,
> powerpc64le, armv7, powerpc, and powerpcspe. That, in
> turn, suggests that kyua test inspection for the likes
> of aarch64 is not historically a part of the release
> process for openzfs or for operating systems that include
> openzfs.
>
>
> ===
> Mark Millard
> marklmi at yahoo.com
Looks like the kyua zfs tests likely are not used on aarch64 or other contexts with unsigned char
kyua tests that use the:
/usr/tests/sys/cddl/zfs/bin/mkfile
program like so (for example):
mkfile 500M /testpool.1861/bigfile.0
(which should be valid) end up with mkfile
instead reporting:
Standard error:
Usage: mkfile [-nv] [e|p|t|g|m|k|b] ...
which prevent the kyua test involved from working.
Turns out this is from expecting char to be always
signed (so a -1 vs. 255 distinction, here in an
aarch64 context):
. . .
(gdb) list
179 /* Options. */
180 while ((ch = getopt(argc, argv, "nv")) != -1) {
181 switch (ch) {
182 case 'n':
183 nofill = 1;
184 break;
185 case 'v':
(gdb) print ch
$16 = 255 '\377'
(gdb) print/x -1
$17 = 0x
(gdb) print/x ch
$18 = 0xff
. . .
With the mix of unsigned and signed it ends up
being a 0xffu != 0xu test, which is
always true.
So the switch is reached as if a "-" prefix was
present (that is not). Then the "option" is classified
as invalid and the usage message is produced.
Apparently no one had noticed. That, in turn, suggests a
lack of inspected testing on aarch64, powerpc64,
powerpc64le, armv7, powerpc, and powerpcspe. That, in
turn, suggests that kyua test inspection for the likes
of aarch64 is not historically a part of the release
process for openzfs or for operating systems that include
openzfs.
===
Mark Millard
marklmi at yahoo.com
