Re: Failing tests on MSVC in libs-base
I thought the 404 was the error we were trying to correct which is why I placed the file there. Gregory Casamento GNUstep Lead Developer / Black Lotus, Principal Consultant http://www.gnustep.org - http://heronsperch.blogspot.com https://www.patreon.com/bePatron?u=352392 - Become a Patron https://www.openhub.net/languages/objective_c - OpenHub standings On Tue, Apr 9, 2024 at 18:23 Riccardo Mottola wrote: > Hi, > > Ivan Vučica wrote: > > I saw this: this was recently addressed by replacing probe of > > example.org/silly-url-test because example.org started returning 503 > > or similar for that URL instead of 404. > > > > Incoming PR addressed this on every other platform by replacing > > example.org with another piece of infrastructure we don’t fully > > control, gnustep.github.io, so I merged it. (As I mentioned on that > > PR, worse solution would be if it depended on our infrastructure: DNS, > > Nginx, … on gnustep.org. Then any flake on home.gnustep.org or with > > Gandi for DNS would also start people’s tests failing.) > > > > I don't think this was the correct solution. Gregory added a file to the > websites, but that's not the intent of a 404! > Also, I think it is case-sensitive. > > I don't think the PR was a good one? RIght now we fail on windows (just > checked) and FreeBSD, Linux... so it is a consistent "broken" behaviour, > not just windows. > > Riccardo > >
Re: Failing tests on MSVC in libs-base
Hi, Ivan Vučica wrote: I saw this: this was recently addressed by replacing probe of example.org/silly-url-test because example.org started returning 503 or similar for that URL instead of 404. Incoming PR addressed this on every other platform by replacing example.org with another piece of infrastructure we don’t fully control, gnustep.github.io, so I merged it. (As I mentioned on that PR, worse solution would be if it depended on our infrastructure: DNS, Nginx, … on gnustep.org. Then any flake on home.gnustep.org or with Gandi for DNS would also start people’s tests failing.) I don't think this was the correct solution. Gregory added a file to the websites, but that's not the intent of a 404! Also, I think it is case-sensitive. I don't think the PR was a good one? RIght now we fail on windows (just checked) and FreeBSD, Linux... so it is a consistent "broken" behaviour, not just windows. Riccardo
Re: Failing tests on MSVC in libs-base
Hi, Ivan Vučica wrote: I saw this: this was recently addressed by replacing probe of example.org/silly-url-test because example.org started returning 503 or similar for that URL instead of 404. We rely on github there... wonder what happened. But also creating the file doesn't seem to be a good idea if 404 is looked after? Tests should not be trying to access network as part of CI. The right fix would be to spin up a temporary server on localhost etc. That itself requires carefully doing process management on all supported platforms. Richard should chime in here... but before Easter I had all tests running on Linux, FreeBSD, NetBSD... I think there is minimal HTTP test server. Why does it fail for windows+clang? No clue. Maybe CI for windows actually blocks outgoing requests completely? This would be the cause only if we have more tests dependent on the network and they’re also failing. Maybe the CORRECT fix is not to do the test at all, at least in Ci? It could be somehow put as skipped or dashed hope under certain conditions? like a CI flag? Riccardo
Re: Failing tests on MSVC in libs-base
I saw this: this was recently addressed by replacing probe of example.org/silly-url-test because example.org started returning 503 or similar for that URL instead of 404.Incoming PR addressed this on every other platform by replacing example.org with another piece of infrastructure we don’t fully control, gnustep.github.io, so I merged it. (As I mentioned on that PR, worse solution would be if it depended on our infrastructure: DNS, Nginx, … on gnustep.org. Then any flake on home.gnustep.org or with Gandi for DNS would also start people’s tests failing.)Tests should not be trying to access network as part of CI. The right fix would be to spin up a temporary server on localhost etc. That itself requires carefully doing process management on all supported platforms.Or turn it from an integration test into a unit test (verify that sending and reading back HTTP over a Unix socket / windows pipe within the same process works, or similar without using OS primitives at all). Why does it fail for windows+clang? No clue. Maybe CI for windows actually blocks outgoing requests completely? This would be the cause only if we have more tests dependent on the network and they’re also failing. Maybe the CORRECT fix is not to do the test at all, at least in Ci?Sent from my iPhoneOn 3 Apr 2024, at 05:17, Gregory Casamento wrote:Hey,In the MSVC gnustep-2.0 test in libs-base, we are getting the following:base/NSURL/basic.m: 213Failed test: (2024-04-02 22:32:07.070 +) basic.m:63 ... Status of load is 404 for gnustep.github.io/silly-file-name 215base/NSURL/test01.m: 216Failed file: test01.m aborted without running all tests! 217--- Running tests in base/NSURLConnection --- 218--- Running tests in base/NSURLHandle ---I wondering what is causing this and if we can fix it as it is causing a build failure.Thanks, GC-- Gregory CasamentoGNUstep Lead Developer / Black Lotus, Principal Consultanthttp://www.gnustep.org - http://heronsperch.blogspot.comhttps://www.patreon.com/bePatron?u=352392 - Become a Patronhttps://www.openhub.net/languages/objective_c - OpenHub standings
