Re: arm64 support for Travis CI testing
On Fri, Jan 10, 2020 at 09:55:24AM -0800, Mike Rumph wrote: > I'm interested in taking a look at this t/apache/expr_string.t > testcase failure. For example, is it the testcase that is in error or > is it the code that it's testing? Maybe this can be discussed in a > separate thread since it is not exclusive to arm64. Has a thread been > started on this already? I don't think there's another thread. I'm guessing this is some kind of timing issue in the test case, which tries to read an error_log entry immediately after the request which generates it. I've added a sleep which will maybe mitigate this - http://svn.apache.org/r1872705 so let's see how this one goes Regards, Joe
Re: arm64 support for Travis CI testing
Okay Job #214 has the arm64 run back close to 11 minutes. On Fri, Jan 10, 2020 at 9:55 AM Mike Rumph wrote: > I'm interested in taking a look at this t/apache/expr_string.t testcase > failure. > For example, is it the testcase that is in error or is it the code that > it's testing? > Maybe this can be discussed in a separate thread since it is not exclusive > to arm64. > Has a thread been started on this already? > > Also, it looks like the arm64 job is taking close to 30 minutes. > - https://travis-ci.org/apache/httpd (Job #213) > But we have seen earlier jobs close to 11 minutes. > Has something changed that can explain this? > > On Thu, Jan 9, 2020 at 6:12 AM Joe Orton wrote: > >> On Thu, Jan 09, 2020 at 03:01:42PM +0100, Luca Toscano wrote: >> > Il giorno gio 9 gen 2020 alle ore 14:19 Joe Orton >> > ha scritto: >> > > >> > > The caching does work on arm64 so the build & test only takes 11 >> minutes >> > > which is reasonable. >> > > >> > > https://travis-ci.org/apache/httpd/jobs/634661216 >> > > >> > > Looks like this one is the next unreliable test to attack - >> > > >> > > 2425# Failed test 17 in t/apache/expr_string.t at line 87 >> > > 2426# Failed test 20 in t/apache/expr_string.t at line 87 fail #2 >> > > 2427t/apache/expr_string.t .. >> > > 2428Failed 2/44 subtests >> > > 2429 >> > > >> > > Can anybody reliably reproduce that failure? >> > >> > Does it happen only for ARM or also for other architectures? Never >> > seen this before.. >> >> Definitely all architectures, it's one Rainer reports as failing >> regularly in release testing. Here's another one and I'm pretty sure it >> has shown up in more of the recent failures too - >> >> https://travis-ci.org/apache/httpd/jobs/633442262 >> >>
Re: arm64 support for Travis CI testing
I'm interested in taking a look at this t/apache/expr_string.t testcase failure. For example, is it the testcase that is in error or is it the code that it's testing? Maybe this can be discussed in a separate thread since it is not exclusive to arm64. Has a thread been started on this already? Also, it looks like the arm64 job is taking close to 30 minutes. - https://travis-ci.org/apache/httpd (Job #213) But we have seen earlier jobs close to 11 minutes. Has something changed that can explain this? On Thu, Jan 9, 2020 at 6:12 AM Joe Orton wrote: > On Thu, Jan 09, 2020 at 03:01:42PM +0100, Luca Toscano wrote: > > Il giorno gio 9 gen 2020 alle ore 14:19 Joe Orton > > ha scritto: > > > > > > The caching does work on arm64 so the build & test only takes 11 > minutes > > > which is reasonable. > > > > > > https://travis-ci.org/apache/httpd/jobs/634661216 > > > > > > Looks like this one is the next unreliable test to attack - > > > > > > 2425# Failed test 17 in t/apache/expr_string.t at line 87 > > > 2426# Failed test 20 in t/apache/expr_string.t at line 87 fail #2 > > > 2427t/apache/expr_string.t .. > > > 2428Failed 2/44 subtests > > > 2429 > > > > > > Can anybody reliably reproduce that failure? > > > > Does it happen only for ARM or also for other architectures? Never > > seen this before.. > > Definitely all architectures, it's one Rainer reports as failing > regularly in release testing. Here's another one and I'm pretty sure it > has shown up in more of the recent failures too - > > https://travis-ci.org/apache/httpd/jobs/633442262 > >
Re: arm64 support for Travis CI testing
On Thu, Jan 09, 2020 at 03:01:42PM +0100, Luca Toscano wrote: > Il giorno gio 9 gen 2020 alle ore 14:19 Joe Orton > ha scritto: > > > > The caching does work on arm64 so the build & test only takes 11 minutes > > which is reasonable. > > > > https://travis-ci.org/apache/httpd/jobs/634661216 > > > > Looks like this one is the next unreliable test to attack - > > > > 2425# Failed test 17 in t/apache/expr_string.t at line 87 > > 2426# Failed test 20 in t/apache/expr_string.t at line 87 fail #2 > > 2427t/apache/expr_string.t .. > > 2428Failed 2/44 subtests > > 2429 > > > > Can anybody reliably reproduce that failure? > > Does it happen only for ARM or also for other architectures? Never > seen this before.. Definitely all architectures, it's one Rainer reports as failing regularly in release testing. Here's another one and I'm pretty sure it has shown up in more of the recent failures too - https://travis-ci.org/apache/httpd/jobs/633442262
Re: arm64 support for Travis CI testing
Il giorno gio 9 gen 2020 alle ore 14:19 Joe Orton ha scritto: > > The caching does work on arm64 so the build & test only takes 11 minutes > which is reasonable. > > https://travis-ci.org/apache/httpd/jobs/634661216 > > Looks like this one is the next unreliable test to attack - > > 2425# Failed test 17 in t/apache/expr_string.t at line 87 > 2426# Failed test 20 in t/apache/expr_string.t at line 87 fail #2 > 2427t/apache/expr_string.t .. > 2428Failed 2/44 subtests > 2429 > > Can anybody reliably reproduce that failure? Does it happen only for ARM or also for other architectures? Never seen this before..
Re: arm64 support for Travis CI testing
The caching does work on arm64 so the build & test only takes 11 minutes which is reasonable. https://travis-ci.org/apache/httpd/jobs/634661216 Looks like this one is the next unreliable test to attack - 2425# Failed test 17 in t/apache/expr_string.t at line 87 2426# Failed test 20 in t/apache/expr_string.t at line 87 fail #2 2427t/apache/expr_string.t .. 2428Failed 2/44 subtests 2429 Can anybody reliably reproduce that failure?
Re: arm64 support for Travis CI testing
On Thu, Jan 09, 2020 at 09:10:31AM +, Pluem, Ruediger, Vodafone Group wrote: > Would > > lscpu -p | grep -c -E ^[0-9] > > work on Ubuntu to deliver the number of cores? > > So > > make -j `lscpu -p | grep -c -E ^[0-9]` I've got no idea if this DTRT for the mix of VMs & containers that travis uses, but it's probably worth trying! Regards, Joe
Re: arm64 support for Travis CI testing
I would like to express my thanks to Luca and Joe and all other people involved in the travis integration! Very nice indeed! > Am 09.01.2020 um 09:15 schrieb Joe Orton : > > On Wed, Jan 08, 2020 at 02:50:17PM -0800, Mike Rumph wrote: >> Okay the first Travis run with arm64 (#201) passed. >> The arm64 job (#201.4) took 33 minutes, 5 seconds. >> Compared tp ppc64le at 4 minutes, 30 seconds. >> This is possibly due to need for extra caching on the first run. >> Otherwise, the jobs seem to have the same results. > > Nice, thanks Mike! From the timings in the log a large chunk of the > time (21m) is building and installing the Perl modules, so if the > caching works it should indeed be much faster second time round. > > I wonder if the non-x86 builds have more CPUs available, we have > hardcoded "make -j2" at the moment but possibly it makes sense to adjust > this. > > Regards, Joe >
Re: arm64 support for Travis CI testing
Hi, On Thu, Jan 9, 2020 at 11:01 AM Joe Orton wrote: > On Wed, Jan 08, 2020 at 02:50:17PM -0800, Mike Rumph wrote: > > Okay the first Travis run with arm64 (#201) passed. > > The arm64 job (#201.4) took 33 minutes, 5 seconds. > > Compared tp ppc64le at 4 minutes, 30 seconds. > > This is possibly due to need for extra caching on the first run. > > Otherwise, the jobs seem to have the same results. > > Nice, thanks Mike! From the timings in the log a large chunk of the > time (21m) is building and installing the Perl modules, so if the > caching works it should indeed be much faster second time round. > > I wonder if the non-x86 builds have more CPUs available, we have > hardcoded "make -j2" at the moment but possibly it makes sense to adjust > this. > https://docs.travis-ci.com/user/multi-cpu-architectures/ says: While available to all Open Source repositories, the concurrency available for multiple CPU arch-based jobs is limited during the alpha period. Not sure what exactly they meant with *limited* here. > Regards, Joe > >
Re: arm64 support for Travis CI testing
On Wed, Jan 08, 2020 at 02:50:17PM -0800, Mike Rumph wrote: > Okay the first Travis run with arm64 (#201) passed. > The arm64 job (#201.4) took 33 minutes, 5 seconds. > Compared tp ppc64le at 4 minutes, 30 seconds. > This is possibly due to need for extra caching on the first run. > Otherwise, the jobs seem to have the same results. Nice, thanks Mike! From the timings in the log a large chunk of the time (21m) is building and installing the Perl modules, so if the caching works it should indeed be much faster second time round. I wonder if the non-x86 builds have more CPUs available, we have hardcoded "make -j2" at the moment but possibly it makes sense to adjust this. Regards, Joe
Re: arm64 support for Travis CI testing
Okay the first Travis run with arm64 (#201) passed. The arm64 job (#201.4) took 33 minutes, 5 seconds. Compared tp ppc64le at 4 minutes, 30 seconds. This is possibly due to need for extra caching on the first run. Otherwise, the jobs seem to have the same results. On Wed, Jan 8, 2020 at 12:13 PM Mike Rumph wrote: > Thanks Joe, > > Since I'm not adding a new feature other than enabling arm64 support, > I'll just add the commit directly so that others can contribute as well > and then revert it if arm64 causes too much noise. > > Mike > > On Wed, Jan 8, 2020 at 1:00 AM Joe Orton wrote: > >> On Tue, Jan 07, 2020 at 01:27:33PM -0800, Mike Rumph wrote: >> > I would like to add support for arm64 to httpd/trunk/.travis.yml. >> > I would then devote some time to getting this support to work. >> > Are there some steps I should take before adding this commit? >> >> That'd be great! If you are familiar with github & Travis, you could >> try it using a fork of the git repo, following the instructions in >> test/README.travis. Otherwise you could just push the change to trunk >> and see ;) >> >> I can't remember if I tested arm before, but it seems the non-x86 VMs in >> Travis don't have the same set of Debian packages installed as x86, so >> some tweaking to the apt: section might be required to get it working. >> >> Regards, Joe >> >>
Re: arm64 support for Travis CI testing
Thanks Joe, Since I'm not adding a new feature other than enabling arm64 support, I'll just add the commit directly so that others can contribute as well and then revert it if arm64 causes too much noise. Mike On Wed, Jan 8, 2020 at 1:00 AM Joe Orton wrote: > On Tue, Jan 07, 2020 at 01:27:33PM -0800, Mike Rumph wrote: > > I would like to add support for arm64 to httpd/trunk/.travis.yml. > > I would then devote some time to getting this support to work. > > Are there some steps I should take before adding this commit? > > That'd be great! If you are familiar with github & Travis, you could > try it using a fork of the git repo, following the instructions in > test/README.travis. Otherwise you could just push the change to trunk > and see ;) > > I can't remember if I tested arm before, but it seems the non-x86 VMs in > Travis don't have the same set of Debian packages installed as x86, so > some tweaking to the apt: section might be required to get it working. > > Regards, Joe > >
Re: arm64 support for Travis CI testing
On Tue, Jan 07, 2020 at 01:27:33PM -0800, Mike Rumph wrote: > I would like to add support for arm64 to httpd/trunk/.travis.yml. > I would then devote some time to getting this support to work. > Are there some steps I should take before adding this commit? That'd be great! If you are familiar with github & Travis, you could try it using a fork of the git repo, following the instructions in test/README.travis. Otherwise you could just push the change to trunk and see ;) I can't remember if I tested arm before, but it seems the non-x86 VMs in Travis don't have the same set of Debian packages installed as x86, so some tweaking to the apt: section might be required to get it working. Regards, Joe
