Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Fri, Jan 22, 2021 at 12:25:42PM +, Mark Rutland wrote:
> On Fri, Jan 22, 2021 at 04:03:26AM -0800, Paul E. McKenney wrote:
> > On Thu, Jan 21, 2021 at 11:11:17AM +, Mark Rutland wrote:
> >
> > [ . . . ]
> >
> > > I've given this a spin atop v5.11-rc4, building natively on arm64 on a
> > > Debian 10.7 system, and with the whole series applied I'm able to run
> > > the rcutorture kvm.sh script without issue (the CONFIG warnings are
> > > unrelated to nolibc):
> > >
> > > | [mark@gravadlaks:~/src/linux]%
> > > ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs
> > > "TREE03" --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64"
> > > --duration 1
> > > | Creating a statically linked C-language initrd
> > > | Done creating a statically linked C-language initrd
> > > | Results directory:
> > > /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> > > | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs
> > > TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration
> > > 1
> > > | Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
> > > | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
> > > | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
> > > | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
> > > | TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
> > > | Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
> > > | All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
> > > | TREE03 16: Build/run results:
> > > | --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
> > > | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> > > | :CONFIG_KVM_GUEST=y: improperly set
> >
> > These two (apparently harmless) error messages are due to these lines
> > in CFcommon:
> >
> > CONFIG_HYPERVISOR_GUEST=y
> > CONFIG_KVM_GUEST=y
>
> Yup; I had figured this out, but since this wasn't getting in the way of
> actually running the torture tests I had assumed we could deal with that
> at some indefinite point in the future, or simply ignore it. ;)
>
> > It looks like CONFIG_HYPERVISOR_GUEST is specific to x86, while KVM_GUEST
> > is specific to x86, powerpc, and mips. (It appears that arm64 doesn't
> > need anything here?)
>
> Yup, we don't need any special options -- arm64 doesn't stricly need any
> guest enlightenment to run under a hypervisor, so we never had a need
> for KVM_GUEST or HYPERVISOR_GUEST. We have all the common
> paravritualized drivers (e.g. virtio) in defconfig too, so that should
> all work out of the box.
>
> > Given this variety, I need to make rcutorture know very early on what
> > arch it is building for. My current approach of looking at the
> > vmlinux file won't work because I need to get the config right before
> > building the kernel.
> >
> > One approach would be to look at the initrd/init file, but doing this
> > reliably would mean removing the ability for users to supply their own
> > initrd file trees. Another approach would be to look at the current
> > environment, perhaps using "uname -m", which will work until someone
> > wants to cross-build. Yet another approach would be to parse the target
> > line from the output of "${CROSS_COMPILE}gcc -v".
> >
> > Left to myself, I would parse the output of "${CROSS_COMPILE}gcc -v".
>
> Heh, I hadn't considered parsing the target line from that output, and I
> guess that works for native builds where "${CROSS_COMPILE}" = "". Neat
> trick!
Credit to Willy Tarreau. Me, I just realized that he needed to do
-something- to create rcutorture's initrd. ;-)
> That sounds sensible to me!
Let me see what I can do.
My thought is to add optional CFcommon. files, and pull them in
when there is a match. But I will give it more thought.
Thanx, Paul
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 11:11:17AM +, Mark Rutland wrote:
[ . . . ]
> I've given this a spin atop v5.11-rc4, building natively on arm64 on a
> Debian 10.7 system, and with the whole series applied I'm able to run
> the rcutorture kvm.sh script without issue (the CONFIG warnings are
> unrelated to nolibc):
>
> | [mark@gravadlaks:~/src/linux]%
> ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs "TREE03"
> --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64" --duration 1
> | Creating a statically linked C-language initrd
> | Done creating a statically linked C-language initrd
> | Results directory:
> /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03
> --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> | Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
> | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
> | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
> | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
> | TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
> | Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
> | All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
> | TREE03 16: Build/run results:
> | --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
> | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> | :CONFIG_KVM_GUEST=y: improperly set
These two (apparently harmless) error messages are due to these lines
in CFcommon:
CONFIG_HYPERVISOR_GUEST=y
CONFIG_KVM_GUEST=y
It looks like CONFIG_HYPERVISOR_GUEST is specific to x86, while KVM_GUEST
is specific to x86, powerpc, and mips. (It appears that arm64 doesn't
need anything here?) Given this variety, I need to make rcutorture
know very early on what arch it is building for. My current approach of
looking at the vmlinux file won't work because I need to get the config
right before building the kernel.
One approach would be to look at the initrd/init file, but doing this
reliably would mean removing the ability for users to supply their own
initrd file trees. Another approach would be to look at the current
environment, perhaps using "uname -m", which will work until someone
wants to cross-build. Yet another approach would be to parse the target
line from the output of "${CROSS_COMPILE}gcc -v".
Left to myself, I would parse the output of "${CROSS_COMPILE}gcc -v".
Thoughts?
Thanx, Paul
> | --- Thu 21 Jan 10:56:25 GMT 2021: Starting kernel
> | QEMU 3.1.0 monitor - type 'help' for more information
> | (qemu) Monitoring qemu job at pid 258620
> | Grace period for qemu job at pid 258620
> |
> |
> | --- Thu 21 Jan 10:53:24 GMT 2021 Test summary:
> | Results directory:
> /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03
> --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> | TREE03 --- 475 GPs (7.91667/s) [rcu: g5737 f0x0 total-gps=1713]
> | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> | :CONFIG_KVM_GUEST=y: improperly set
>
> So FWIW:
>
> Tested-by: Mark Rutland [arm64]
>
> It would be great if this could be applied soon so that it's possible to
> use the rcutorture scripts without applying local hacks.
>
> Willy, thanks for sorting this out, especially so quickly!
>
> Thanks,
> Mark.
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Fri, Jan 22, 2021 at 04:03:26AM -0800, Paul E. McKenney wrote:
> On Thu, Jan 21, 2021 at 11:11:17AM +, Mark Rutland wrote:
>
> [ . . . ]
>
> > I've given this a spin atop v5.11-rc4, building natively on arm64 on a
> > Debian 10.7 system, and with the whole series applied I'm able to run
> > the rcutorture kvm.sh script without issue (the CONFIG warnings are
> > unrelated to nolibc):
> >
> > | [mark@gravadlaks:~/src/linux]%
> > ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs
> > "TREE03" --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64"
> > --duration 1
> > | Creating a statically linked C-language initrd
> > | Done creating a statically linked C-language initrd
> > | Results directory:
> > /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24
> > | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs
> > TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1
> > | Start batch 1: Thu 21 Jan 10:53:24 GMT 2021
> > | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021
> > | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021
> > | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021
> > | TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021
> > | Starting kernels. Thu 21 Jan 10:56:25 GMT 2021
> > | All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021
> > | TREE03 16: Build/run results:
> > | --- Thu 21 Jan 10:53:24 GMT 2021: Starting build
> > | :CONFIG_HYPERVISOR_GUEST=y: improperly set
> > | :CONFIG_KVM_GUEST=y: improperly set
>
> These two (apparently harmless) error messages are due to these lines
> in CFcommon:
>
> CONFIG_HYPERVISOR_GUEST=y
> CONFIG_KVM_GUEST=y
Yup; I had figured this out, but since this wasn't getting in the way of
actually running the torture tests I had assumed we could deal with that
at some indefinite point in the future, or simply ignore it. ;)
> It looks like CONFIG_HYPERVISOR_GUEST is specific to x86, while KVM_GUEST
> is specific to x86, powerpc, and mips. (It appears that arm64 doesn't
> need anything here?)
Yup, we don't need any special options -- arm64 doesn't stricly need any
guest enlightenment to run under a hypervisor, so we never had a need
for KVM_GUEST or HYPERVISOR_GUEST. We have all the common
paravritualized drivers (e.g. virtio) in defconfig too, so that should
all work out of the box.
> Given this variety, I need to make rcutorture know very early on what
> arch it is building for. My current approach of looking at the
> vmlinux file won't work because I need to get the config right before
> building the kernel.
>
> One approach would be to look at the initrd/init file, but doing this
> reliably would mean removing the ability for users to supply their own
> initrd file trees. Another approach would be to look at the current
> environment, perhaps using "uname -m", which will work until someone
> wants to cross-build. Yet another approach would be to parse the target
> line from the output of "${CROSS_COMPILE}gcc -v".
>
> Left to myself, I would parse the output of "${CROSS_COMPILE}gcc -v".
Heh, I hadn't considered parsing the target line from that output, and I
guess that works for native builds where "${CROSS_COMPILE}" = "". Neat
trick!
That sounds sensible to me!
Thanks,
Mark.
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 09:17:14PM +0100, Willy Tarreau wrote: > On Thu, Jan 21, 2021 at 11:54:32AM -0800, Paul E. McKenney wrote: > > > > It would be great if this could be applied soon so that it's possible to > > > > use the rcutorture scripts without applying local hacks. > > > > > > Makes sense. I was wondering, should we mark them for stable ? I don't > > > know if anyone relies on these tests to validate stable kernels in > > > fact. > > > > I added Fixes tags that should make this happen, and they are now visible > > at -rcu branch "dev". Could you please check them for me? > > I've just rerun all previous tests from my history and all of them are > OK. Please note however that I only did the manual build test, not through > the whole kvm.sh script, but a diff shows that the involved files are byte > for byte identical to those that Valentin and Mark tested, so for me that's > OK as well. Byte-for-byte identical works for me, thank you! > By the way, thank you for having completed the commit messages, I hope you > didn't spend too much time on this. Not a problem! I definitely had the easy end of this job. ;-) Thanx, Paul
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 11:54:32AM -0800, Paul E. McKenney wrote: > > > It would be great if this could be applied soon so that it's possible to > > > use the rcutorture scripts without applying local hacks. > > > > Makes sense. I was wondering, should we mark them for stable ? I don't > > know if anyone relies on these tests to validate stable kernels in > > fact. > > I added Fixes tags that should make this happen, and they are now visible > at -rcu branch "dev". Could you please check them for me? I've just rerun all previous tests from my history and all of them are OK. Please note however that I only did the manual build test, not through the whole kvm.sh script, but a diff shows that the involved files are byte for byte identical to those that Valentin and Mark tested, so for me that's OK as well. By the way, thank you for having completed the commit messages, I hope you didn't spend too much time on this. Cheers, Willy
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 03:18:09PM +0100, Willy Tarreau wrote: > On Thu, Jan 21, 2021 at 11:11:17AM +, Mark Rutland wrote: > > So FWIW: > > > > Tested-by: Mark Rutland [arm64] Thank you all! > Perfect, thanks! Paul, may I let you copy-paste the tested-by yourself ? > If you prefer I'm fine with resending a series to you, I just don't want > to needlessly spam you :-) Done, with Valentin's and Mark's Tested-by. > > It would be great if this could be applied soon so that it's possible to > > use the rcutorture scripts without applying local hacks. > > Makes sense. I was wondering, should we mark them for stable ? I don't > know if anyone relies on these tests to validate stable kernels in > fact. I added Fixes tags that should make this happen, and they are now visible at -rcu branch "dev". Could you please check them for me? c261145 tools/nolibc: Add the definition for dup() 79f220e tools/nolibc: Make dup2() rely on dup3() when available c0c7c10 tools/nolibc: Make getpgrp() fall back to getpgid(0) be60ca4 tools/nolibc: Implement fork() based on clone() 5b1c827 tools/nolibc: Implement poll() based on ppoll() 70ca7ae tools/nolibc: Get timeval, timespec and timezone from linux/time.h f65d711 tools/nolibc: Remove incorrect definitions of __ARCH_WANT_* 35635d7 tools/nolibc: Emit detailed error for missing alternate syscall number definitions 3c6ce7a tools/nolibc: Fix position of -lgcc in the documented example 26cec81 tools/rcutorture: Fix position of -lgcc in mkinitrd.sh > > Willy, thanks for sorting this out, especially so quickly! > > You're welcome, and thanks to you for the detailed report and explanations. Again, thank you all! On getting this upstream quickly, if all goes well I expect to include this in my pull request for the upcoming merge window. Thanx, Paul
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 11:11:17AM +, Mark Rutland wrote: > So FWIW: > > Tested-by: Mark Rutland [arm64] Perfect, thanks! Paul, may I let you copy-paste the tested-by yourself ? If you prefer I'm fine with resending a series to you, I just don't want to needlessly spam you :-) > It would be great if this could be applied soon so that it's possible to > use the rcutorture scripts without applying local hacks. Makes sense. I was wondering, should we mark them for stable ? I don't know if anyone relies on these tests to validate stable kernels in fact. > Willy, thanks for sorting this out, especially so quickly! You're welcome, and thanks to you for the detailed report and explanations. Willy
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 11:05:48AM +, Valentin Schneider wrote: > Thanks for having a look at this! And kuddos to Mark for nudging the > right people. Likewise to Paul for poking Willy! Thanks all, Mark.
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
Hi Willy, Paul, On Thu, Jan 21, 2021 at 08:20:22AM +0100, Willy Tarreau wrote: > Hi Paul, > > as per the recent discussion with Mark, I've updated the nolibc header to > reflect latest upstream which is needed to build on arm64, and I performed > the few cleanups that Mark rightfully suggested. > > The following patches were taken from the upstream code and this time I > carefully copied the original commit IDs in hope not to miss such fixes > anymore in the future. > > I've build-tested these on x86_64, i586, arm(v5 & v7), arm64, mips and > mipsel, using compilers ranging from gcc 3.4 to gcc 9.3 so I think we're > good for these archs now. I've given this a spin atop v5.11-rc4, building natively on arm64 on a Debian 10.7 system, and with the whole series applied I'm able to run the rcutorture kvm.sh script without issue (the CONFIG warnings are unrelated to nolibc): | [mark@gravadlaks:~/src/linux]% ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs "TREE03" --kmake-arg "CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64" --duration 1 | Creating a statically linked C-language initrd | Done creating a statically linked C-language initrd | Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24 | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1 | Start batch 1: Thu 21 Jan 10:53:24 GMT 2021 | TREE03 16: Starting build. Thu 21 Jan 10:53:24 GMT 2021 | TREE03 16: Waiting for build to complete. Thu 21 Jan 10:53:24 GMT 2021 | TREE03 16: Build complete. Thu 21 Jan 10:56:25 GMT 2021 | TREE03 16: Kernel present. Thu 21 Jan 10:56:25 GMT 2021 | Starting kernels. Thu 21 Jan 10:56:25 GMT 2021 | All kernel runs complete. Thu 21 Jan 10:57:35 GMT 2021 | TREE03 16: Build/run results: | --- Thu 21 Jan 10:53:24 GMT 2021: Starting build | :CONFIG_HYPERVISOR_GUEST=y: improperly set | :CONFIG_KVM_GUEST=y: improperly set | --- Thu 21 Jan 10:56:25 GMT 2021: Starting kernel | QEMU 3.1.0 monitor - type 'help' for more information | (qemu) Monitoring qemu job at pid 258620 | Grace period for qemu job at pid 258620 | | | --- Thu 21 Jan 10:53:24 GMT 2021 Test summary: | Results directory: /home/mark/src/linux/tools/testing/selftests/rcutorture/res/2021.01.21-10.53.24 | ./tools/testing/selftests/rcutorture/bin/kvm.sh --cpus 255 --configs TREE03 --kmake-arg CROSS_COMPILE=aarch64-linux-gnu- ARCH=arm64 --duration 1 | TREE03 --- 475 GPs (7.91667/s) [rcu: g5737 f0x0 total-gps=1713] | :CONFIG_HYPERVISOR_GUEST=y: improperly set | :CONFIG_KVM_GUEST=y: improperly set So FWIW: Tested-by: Mark Rutland [arm64] It would be great if this could be applied soon so that it's possible to use the rcutorture scripts without applying local hacks. Willy, thanks for sorting this out, especially so quickly! Thanks, Mark.
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On Thu, Jan 21, 2021 at 11:05:48AM +, Valentin Schneider wrote: > This lets me run the following invocation without a hitch: > > tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 10 > --configs "4*SRCU-P" --trust-make > > where before I would get some errors building the initrd due to missing > __NR_foo. > > Tested-by: Valentin Schneider Thank you Valentin! Much appreciated! Willy
Re: [PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
On 21/01/21 08:20, Willy Tarreau wrote: > Hi Paul, > > as per the recent discussion with Mark, I've updated the nolibc header to > reflect latest upstream which is needed to build on arm64, and I performed > the few cleanups that Mark rightfully suggested. > > The following patches were taken from the upstream code and this time I > carefully copied the original commit IDs in hope not to miss such fixes > anymore in the future. > > I've build-tested these on x86_64, i586, arm(v5 & v7), arm64, mips and > mipsel, using compilers ranging from gcc 3.4 to gcc 9.3 so I think we're > good for these archs now. > > Just let me know if you prefer a pull request, as I can do that as well. > This lets me run the following invocation without a hitch: tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 10 --configs "4*SRCU-P" --trust-make where before I would get some errors building the initrd due to missing __NR_foo. Tested-by: Valentin Schneider Thanks for having a look at this! And kuddos to Mark for nudging the right people. > Thanks! > Willy > > > Willy Tarreau (9): > tools/nolibc: the definition dup() was missing > tools/nolibc: make dup2() rely on dup3() when available > tools/nolibc: make getpgrp() fall back to getpgid(0) > tools/nolibc: implement fork() based on clone() > tools/nolibc: implement poll() based on ppoll() > tools/nolibc: get timeval, timespec and timezone from linux/time.h > tools/nolibc: remove incorrect definitions of __ARCH_WANT_* > tools/nolibc: emit a detailed error when missing alternate syscall > number definitions > tools/nolibc: fix position of -lgcc in the documented example > > tools/include/nolibc/nolibc.h | 153 +- > 1 file changed, 115 insertions(+), 38 deletions(-) > > -- > 2.28.0
[PATCH 0/9] tools/nolibc: fix build issues on aarch64 after unistd cleanup
Hi Paul, as per the recent discussion with Mark, I've updated the nolibc header to reflect latest upstream which is needed to build on arm64, and I performed the few cleanups that Mark rightfully suggested. The following patches were taken from the upstream code and this time I carefully copied the original commit IDs in hope not to miss such fixes anymore in the future. I've build-tested these on x86_64, i586, arm(v5 & v7), arm64, mips and mipsel, using compilers ranging from gcc 3.4 to gcc 9.3 so I think we're good for these archs now. Just let me know if you prefer a pull request, as I can do that as well. Thanks! Willy Willy Tarreau (9): tools/nolibc: the definition dup() was missing tools/nolibc: make dup2() rely on dup3() when available tools/nolibc: make getpgrp() fall back to getpgid(0) tools/nolibc: implement fork() based on clone() tools/nolibc: implement poll() based on ppoll() tools/nolibc: get timeval, timespec and timezone from linux/time.h tools/nolibc: remove incorrect definitions of __ARCH_WANT_* tools/nolibc: emit a detailed error when missing alternate syscall number definitions tools/nolibc: fix position of -lgcc in the documented example tools/include/nolibc/nolibc.h | 153 +- 1 file changed, 115 insertions(+), 38 deletions(-) -- 2.28.0
