Re: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 progressions on aarch64
> On Dec 4, 2023, at 12:25, Tamar Christina wrote: ... >> - tcwg_bmk-fujitsu_speed-cpu2017speed >> - gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.] >> exp@@GLIBC_2.29 > > This is suspect, it says the slowdown is in exp in glibc, that would be > unrelated > to my patch. Hi Tamar, Linaro benchmarking builds the whole sysroot with the "new" compiler, including glibc. It may be interesting to double-check code-gen differences on glibc's exp() and make sure they are no obvious bad choices. -- Maxim Kuvyrkov https://www.linaro.org > > Tamar > >> >> The link above has the full results. >> >> ci_not...@linaro.org writes: >> >>> Dear contributor, our automatic CI has detected problems related to your >> patch(es). Please >>> find some details below. If you have any questions, please follow up on >>> linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg >>> channel, or >> ping >>> your favourite Linaro toolchain developer on the usual project channel. >>> >>> In master-aarch64 after: >>> >>> | commit gcc-14-5673-g33c2b70dbab >>> | Author: Tamar Christina >>> | Date: Tue Nov 21 13:20:39 2023 + >>> | >>> | AArch64: Add new generic-armv8-a CPU and make it the default. >>> | >>> | This patch adds a new generic scheduling model "generic-armv8-a" and >> makes it >>> | the default for all Armv8 architectures. >>> | >>> | -mcpu=generic and -mtune=generic is kept around for those that >>> really want >> the >>> | previous cost model. >>> | ... 34 lines of the commit log omitted. >>> >>> FAIL: 1 regressions: 8 progressions >>> >>> regressions.sum: >>>=== gcc tests === >>> >>> Running gcc:gcc.target/aarch64/sve/aarch64-sve.exp ... >>> FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test >>> >>>=== Results Summary === >>> >>> progressions.sum: >>>=== gcc tests === >>> >>> Running gcc:gcc.dg/vect/vect.exp ... >>> FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects >>> (test >> for excess errors) >>> UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c compilation failed >> to produce executable >>> UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto- >> objects compilation failed to produce executable >>> FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c (test for excess errors) >>> UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c compilation failed to >> produce executable >>> UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c -flto -ffat-lto-objects >> compilation failed to produce executable >>> FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c (test for excess errors) >>> ... and 3 more entries >>> >>> You can find the failure logs in *.log.1.xz files in >>> - https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64- >> build/1102/artifact/artifacts/00-sumfiles/ . >>> The full lists of regressions and progressions are in >>> - https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64- >> build/1102/artifact/artifacts/notify/ . >>> The list of [ignored] baseline and flaky failures are in >>> - https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64- >> build/1102/artifact/artifacts/sumfiles/xfails.xfail . >>> >>> The configuration of this build is: >>> CI config tcwg_gnu_cross_check_gcc master-aarch64 >>> >>> -8<--8<--8<--- >> --- >>> The information below can be used to reproduce a debug environment: >>> >>> Current build : >>> https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master- >> aarch64-build/1102/artifact/artifacts >>> Reference build : >>> https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master- >> aarch64-build/1101/artifact/artifacts >>> >>> Reproduce last good and first bad builds: https://git- >> us.linaro.org/toolchain/ci/interesting- >> commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcw >> g_gnu_cross_check_gcc/master-aarch64/reproduction_instructions.txt >>> >>> Full commit : https://github.com/gcc- >> mirror/gcc/commit/33c2b70dbabc02788caabcbc66b7baeafeb95bcf >>> >>> Latest bug report status : https://linaro.atlassian.net/browse/GNU-1040 >>> >>> List of configurations that regressed due to this commit : >>> * tcwg_gnu_cross_check_gcc >>> ** master-aarch64 >>> *** FAIL: 1 regressions: 8 progressions >>> *** https://git-us.linaro.org/toolchain/ci/interesting- >> commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcw >> g_gnu_cross_check_gcc/master-aarch64/details.txt >>> *** https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64- >> build/1102/ >> >> >> -- >> Thiago > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose the > contents to any other person, use it for any purpose, or
Re: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 progressions on aarch64
Hello Tamar, Thank you for your clarifications. They're very helpful. Tamar Christina writes: >> - tcwg_bmk-code_size-cpu2017fast >> - gnu-aarch64-master-O2: grew in size by 3% - 508.namd_r >> - gnu-aarch64-master-O2_LTO: grew in size by 2% - 508.namd_r >> >> - tcwg_bmk-code_size-spec2k6 >> - gnu-aarch64-master-O2: grew in size by 2% - 447.dealII >> - gnu-aarch64-master-Os: grew in size by 2% - 456.hmmer >> > > Size gains are expected. The cost model turns on throughput based > optimizations. Since will grow because among other things it unrolls. Ok, that is good to know. >> - tcwg_bmk-code_speed-cpu2017rate >> - gnu-aarch64-master-O2: slowed down by 5% - 519.lbm_r >> - gnu-aarch64-master-O3: slowed down by 5% - 519.lbm_r >> > > As the commit message says, there are outstanding costing issues that will be > resolved at stage 4 when the remaining infrastructure are in place. Nice. >> - tcwg_bmk-fujitsu_speed-cpu2017speed >> - gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.] >> exp@@GLIBC_2.29 > > This is suspect, it says the slowdown is in exp in glibc, that would be > unrelated > to my patch. Indeed, thanks for pointing out. -- Thiago ___ linaro-toolchain mailing list -- linaro-toolchain@lists.linaro.org To unsubscribe send an email to linaro-toolchain-le...@lists.linaro.org
RE: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 progressions on aarch64
Hello Thaigo, > -Original Message- > From: Thiago Jung Bauermann > Sent: Saturday, December 2, 2023 12:15 AM > To: Tamar Christina > Cc: linaro-toolchain@lists.linaro.org; gcc-regress...@gcc.gnu.org; tcwg- > validat...@linaro.org > Subject: Re: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 > progressions on aarch64 > > > Hello Tamar, > > It looks like this failure happens because qemu-aarch64 segfaults when > trying to run the testcase binary: > > Running on qemu: timeout -k 30s 330s setarch x86_64 -R qemu-aarch64 -cpu any > -R 0 -L /home/tcwg- > buildslave/workspace/tcwg_gnu_1/abe/builds/destdir/x86_64-pc-linux- > gnu/aarch64-linux-gnu/libc ./mask_struct_load_3_run.exe > spawn [open ...] > timeout: the monitored command dumped core > FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test > > Which would be a QEMU bug. I'm working on reproducing it and will report > there if appropriate. > > However, our CI also found some size and performance regressions with > this commit, as detailed in https://linaro.atlassian.net/browse/GNU-1040: > > - tcwg_bmk-code_size-cpu2017fast > - gnu-aarch64-master-O2: grew in size by 3% - 508.namd_r > - gnu-aarch64-master-O2_LTO: grew in size by 2% - 508.namd_r > > - tcwg_bmk-code_size-spec2k6 > - gnu-aarch64-master-O2: grew in size by 2% - 447.dealII > - gnu-aarch64-master-Os: grew in size by 2% - 456.hmmer > Size gains are expected. The cost model turns on throughput based optimizations. Since will grow because among other things it unrolls. > - tcwg_bmk-code_speed-cpu2017rate > - gnu-aarch64-master-O2: slowed down by 5% - 519.lbm_r > - gnu-aarch64-master-O3: slowed down by 5% - 519.lbm_r > As the commit message says, there are outstanding costing issues that will be resolved at stage 4 when the remaining infrastructure are in place. > - tcwg_bmk-fujitsu_speed-cpu2017speed > - gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.] > exp@@GLIBC_2.29 This is suspect, it says the slowdown is in exp in glibc, that would be unrelated to my patch. Tamar > > The link above has the full results. > > ci_not...@linaro.org writes: > > > Dear contributor, our automatic CI has detected problems related to your > patch(es). Please > > find some details below. If you have any questions, please follow up on > > linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg > > channel, or > ping > > your favourite Linaro toolchain developer on the usual project channel. > > > > In master-aarch64 after: > > > > | commit gcc-14-5673-g33c2b70dbab > > | Author: Tamar Christina > > | Date: Tue Nov 21 13:20:39 2023 + > > | > > | AArch64: Add new generic-armv8-a CPU and make it the default. > > | > > | This patch adds a new generic scheduling model "generic-armv8-a" and > makes it > > | the default for all Armv8 architectures. > > | > > | -mcpu=generic and -mtune=generic is kept around for those that > > really want > the > > | previous cost model. > > | ... 34 lines of the commit log omitted. > > > > FAIL: 1 regressions: 8 progressions > > > > regressions.sum: > > === gcc tests === > > > > Running gcc:gcc.target/aarch64/sve/aarch64-sve.exp ... > > FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test > > > > === Results Summary === > > > > progressions.sum: > > === gcc tests === > > > > Running gcc:gcc.dg/vect/vect.exp ... > > FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects > > (test > for excess errors) > > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c compilation failed > to produce executable > > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto- > objects compilation failed to produce executable > > FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c (test for excess errors) > > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c compilation failed to > produce executable > > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c -flto -ffat-lto-objects > compilation failed to produce executable > > FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c (test for excess errors) > > ... and 3 more entries > > > > You can find the failure logs in *.log.1.xz files in > > - https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64- > build/1102/artifact/artifacts/00-sumfiles/ . > > The full lists of regressions and progressions are in > > - https://ci.linaro.org/job/tcwg_gnu_cross_chec
Re: [Linaro-TCWG-CI] gcc-14-5673-g33c2b70dbab: FAIL: 1 regressions: 8 progressions on aarch64
Hello Tamar, It looks like this failure happens because qemu-aarch64 segfaults when trying to run the testcase binary: Running on qemu: timeout -k 30s 330s setarch x86_64 -R qemu-aarch64 -cpu any -R 0 -L /home/tcwg-buildslave/workspace/tcwg_gnu_1/abe/builds/destdir/x86_64-pc-linux-gnu/aarch64-linux-gnu/libc ./mask_struct_load_3_run.exe spawn [open ...] timeout: the monitored command dumped core FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test Which would be a QEMU bug. I'm working on reproducing it and will report there if appropriate. However, our CI also found some size and performance regressions with this commit, as detailed in https://linaro.atlassian.net/browse/GNU-1040: - tcwg_bmk-code_size-cpu2017fast - gnu-aarch64-master-O2: grew in size by 3% - 508.namd_r - gnu-aarch64-master-O2_LTO: grew in size by 2% - 508.namd_r - tcwg_bmk-code_size-spec2k6 - gnu-aarch64-master-O2: grew in size by 2% - 447.dealII - gnu-aarch64-master-Os: grew in size by 2% - 456.hmmer - tcwg_bmk-code_speed-cpu2017rate - gnu-aarch64-master-O2: slowed down by 5% - 519.lbm_r - gnu-aarch64-master-O3: slowed down by 5% - 519.lbm_r - tcwg_bmk-fujitsu_speed-cpu2017speed - gnu-aarch64-master-O2: slowed down by 19% - 644.nab_s:[.] exp@@GLIBC_2.29 The link above has the full results. ci_not...@linaro.org writes: > Dear contributor, our automatic CI has detected problems related to your > patch(es). Please > find some details below. If you have any questions, please follow up on > linaro-toolchain@lists.linaro.org mailing list, Libera's #linaro-tcwg > channel, or ping > your favourite Linaro toolchain developer on the usual project channel. > > In master-aarch64 after: > > | commit gcc-14-5673-g33c2b70dbab > | Author: Tamar Christina > | Date: Tue Nov 21 13:20:39 2023 + > | > | AArch64: Add new generic-armv8-a CPU and make it the default. > | > | This patch adds a new generic scheduling model "generic-armv8-a" and > makes it > | the default for all Armv8 architectures. > | > | -mcpu=generic and -mtune=generic is kept around for those that really > want the > | previous cost model. > | ... 34 lines of the commit log omitted. > > FAIL: 1 regressions: 8 progressions > > regressions.sum: > === gcc tests === > > Running gcc:gcc.target/aarch64/sve/aarch64-sve.exp ... > FAIL: gcc.target/aarch64/sve/mask_struct_load_3_run.c execution test > > === Results Summary === > > progressions.sum: > === gcc tests === > > Running gcc:gcc.dg/vect/vect.exp ... > FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto -ffat-lto-objects > (test for excess errors) > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c compilation failed > to produce executable > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c -flto > -ffat-lto-objects compilation failed to produce executable > FAIL: gcc.dg/vect/vect-reduc-pattern-1b-big-array.c (test for excess errors) > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c compilation failed to produce > executable > UNRESOLVED: gcc.dg/vect/vect-reduc-pattern-1b.c -flto -ffat-lto-objects > compilation failed to produce executable > FAIL: gcc.dg/vect/vect-reduc-pattern-1b.c (test for excess errors) > ... and 3 more entries > > You can find the failure logs in *.log.1.xz files in > - > https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/1102/artifact/artifacts/00-sumfiles/ > . > The full lists of regressions and progressions are in > - > https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/1102/artifact/artifacts/notify/ > . > The list of [ignored] baseline and flaky failures are in > - > https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/1102/artifact/artifacts/sumfiles/xfails.xfail > . > > The configuration of this build is: > CI config tcwg_gnu_cross_check_gcc master-aarch64 > > -8<--8<--8<-- > The information below can be used to reproduce a debug environment: > > Current build : > https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/1102/artifact/artifacts > Reference build : > https://ci.linaro.org/job/tcwg_gnu_cross_check_gcc--master-aarch64-build/1101/artifact/artifacts > > Reproduce last good and first bad builds: > https://git-us.linaro.org/toolchain/ci/interesting-commits.git/plain/gcc/sha1/33c2b70dbabc02788caabcbc66b7baeafeb95bcf/tcwg_gnu_cross_check_gcc/master-aarch64/reproduction_instructions.txt > > Full commit : > https://github.com/gcc-mirror/gcc/commit/33c2b70dbabc02788caabcbc66b7baeafeb95bcf > > Latest bug report status : https://linaro.atlassian.net/browse/GNU-1040 > > List of configurations that regressed due to this commit : > * tcwg_gnu_cross_check_gcc > ** master-aarch64 > *** FAIL: 1 regressions: 8 progressions > *** >
