RE: Windows testsuite failures
Simon Peyton Jones via ghc-devs writes: > Both Tamar and Omer are right. > > * Not doing CI on Windows is bad. It means that bugs get introduced, > and not discovered until later. This is Tamer’s point, and it is > valid. > * Holding up MRs because a failures in Windows CI that is unrelated > to the MR is also bad. It a frustrating waste of time, and > discourages all authors. (In contrast, holding up an MR because it > introduces a bug on Windows is fine, indeed desirable.) This is > Omer’s point, and it is valid. > > The obvious solution is: let’s fix Windows CI, so that it doesn’t fail > except when the MR genuinely introduces a bug. > > How hard would it be to do that? Do we even know what the problem is? > This latest issue was quite tricky since the root cause was in an unexpected place (it seems that some of the Windows GitLab runners somehow no longer had symlink permission, perhaps due to an operating system update; I had expected the problem to be in the testsuite driver due to previous issues in that area [1]). Given the relative scarcity of Windows CI capacity and the difficulty of hitting the issue to begin with, it took quite a while to realized the problem. However, this morning I identified the issue and, as a workaround, temporarily disabled forced usage of symlinks on Windows CI. I have also opened #17706, which should allow us to use symlinks without fear of this potential breakage. Cheers, - Ben [1] https://gitlab.haskell.org/ghc/ghc/commit/e35fe8d58f18bd179efdc848c617dc9eddf4478b signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
Ömer Sinan Ağacan writes: >> Now we have rewritten the CI and it's pointing out actual issues in the >> compiler. And your suggestion is well let's just ignore it. > > When is the last time Windows CI caught an actual bug? All I see is random > system failures [1, 2, 3]. > > It must be catching *some* bugs, but that's a rare event in my experience. > It's unfortunately not nearly as rare as you would hope. However, it is likely that these cases aren't widely seen as I have been working on marking broken tests as expect_broken over the last few months. Only recently (for roughly a month now, IIRC) has Windows been a mandatory-green platform and it took a significant amount of effort and several false-starts to get to this point. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
Ömer Sinan Ağacan writes: > Hi Ben, > > Can we please disable Windows CI? I've spent more time fighting the CI than > doing useful work this week, it's really frustrating. > Yes, this recent spate is issues indeed took too long to solve. Unfortunately this particular issue took quite a while to isolate since I had trouble reproducing it as it depended upon which CI builder the job ran on. However, I eventually found and pushed a patch to fix the root cause this morning. > Since we have no idea how to fix it maybe we should test Windows only before a > release, manually (and use bisect in case of regressions). > As pointed out by Tamar, this really is not a viable option. In truth, the pain we are experiencing now is precisely because we neglected Windows support for so long. I am now making a concerted effort to fix this and, while it has been painful, we are gradually approaching a half-way decent story for Windows. x86-64 is, as of this morning, believed to be mostly stable; I'm now working on i386. Cheers, - Ben signature.asc Description: PGP signature ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
RE: Windows testsuite failures
Both Tamar and Omer are right. * Not doing CI on Windows is bad. It means that bugs get introduced, and not discovered until later. This is Tamer’s point, and it is valid. * Holding up MRs because a failures in Windows CI that is unrelated to the MR is also bad. It a frustrating waste of time, and discourages all authors. (In contrast, holding up an MR because it introduces a bug on Windows is fine, indeed desirable.) This is Omer’s point, and it is valid. The obvious solution is: let’s fix Windows CI, so that it doesn’t fail except when the MR genuinely introduces a bug. How hard would it be to do that? Do we even know what the problem is? Simon From: ghc-devs On Behalf Of Phyx Sent: 17 January 2020 06:49 To: Ömer Sinan Ağacan Cc: ghc-devs Subject: Re: Windows testsuite failures Oh I spent a non-insignificant amount of time back in the phabricator days to make the CI stable. Now because people were committing to master directly without going through CI it was always a cat and mouse game and I gave up eventually. Now we have rewritten the CI and it's pointing out actual issues in the compiler. And your suggestion is well let's just ignore it. How about you use some of that energy to help I stead of taking the easy way? And I bet you're going to say you don't care about Windows to which I would say I don't care about the non-threaded runtime and wish we would get rid of it. But can't always get what you want. And to say we'll actually fix anything before release doesn't align with what I've seen so far, which had me scrambling last minute to ensure we can release Windows instead of making releases without it. Quite frankly I don't need you to tell me to submit MRs to fix it since that's what I spent again a lot of time doing. Or maybe you would like to pay my paycheck so I can spend more than a considerable amount of my free time on it. Kind regards, Tamar Sent from my Mobile On Fri, Jan 17, 2020, 06:17 Ömer Sinan Ağacan mailto:omeraga...@gmail.com>> wrote: We release more often than once in 6 months. We clearly have no idea how to test on Windows. If you know how to do it then feel free to submit a MR. Otherwise blocking every MR indefinitely is worse than testing Windows less frequently. Ömer Phyx mailto:loneti...@gmail.com>>, 17 Oca 2020 Cum, 09:10 tarihinde şunu yazdı: > > Sure because only testing once every 6 months is a very very good idea... > > Sent from my Mobile > > On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan > mailto:omeraga...@gmail.com>> wrote: >> >> Hi Ben, >> >> Can we please disable Windows CI? I've spent more time fighting the CI than >> doing useful work this week, it's really frustrating. >> >> Since we have no idea how to fix it maybe we should test Windows only before >> a >> release, manually (and use bisect in case of regressions). >> >> Ömer >> >> Ben Gamari mailto:b...@smart-cactus.org>>, 14 Oca >> 2020 Sal, 14:30 tarihinde şunu yazdı: >> > >> > Hi all, >> > >> > Currently Windows CI is a bit flaky due to some unfortunately rather >> > elusive testsuite driver bugs. Progress in resolving this has been a bit >> > slow due to travel over the last week but I will be back home tomorrow and >> > should be able to resolve the issue soon thereafter. >> > >> > Cheers, >> > >> > - Ben >> > ___ >> > ghc-devs mailing list >> > ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs=02%7C01%7Csimonpj%40microsoft.com%7C20b2daaf0e674142e41508d79b1968ad%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637148405761605935=VmLpBxMbcYWYHaa34KpF1ju2SURx%2BOb5C5g5Gi7YoGE%3D=0> >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs<https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs=02%7C01%7Csimonpj%40microsoft.com%7C20b2daaf0e674142e41508d79b1968ad%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637148405761610933=z28IhdWnHfBKvYTakK6lh8qhr4VPMUBzl71RXBrYkqA%3D=0> ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
On Fri, Jan 17, 2020 at 7:02 AM Ömer Sinan Ağacan wrote: > > Now we have rewritten the CI and it's pointing out actual issues in the > > compiler. And your suggestion is well let's just ignore it. > > When is the last time Windows CI caught an actual bug? All I see is random > system failures [1, 2, 3]. > [1]: Symbolic link privileges are missing from the CI user, something has gone wrong with the permissions on that slave. There's code in the testsuite to symlink or copy. Should fix the permissions, or add permission detection to the python code or switch to copy. [2]: git checkout error, disk probably full. Testsuite runs tend to create a lot of temp files which aren't cleaned up. Over time the disk fills and you get errors such as these. There's a cron job to periodically clean these, but of course that is prone to a race condition. This can be made more reliable by using OS event triggers instead of a cron job. i.e. monitor disk 80% full events and run the cleanup. [3]: It's either trying to execute a non-executable file or something it executed loaded a shared library for a different architecture. Hard to tell which one by just that output. Will need more logs. Now to answer your question [4] and [5] are issues the CI caught that were quite important. [4] https://gitlab.haskell.org/ghc/ghc/issues/17480 [5] https://gitlab.haskell.org/ghc/ghc/issues/17691 just to name two, I can go on, plugin test failures, which pointed out someone submitted an patch tested only on ELF that broke loading on plugins as non-shared objects, etc. The list is quite long. > > It must be catching *some* bugs, but that's a rare event in my experience. > Sure if you go "ahh it's just Windows that's broken" and don't look at the underlying issues. > Sure, I don't write Windows-specific code (e.g. IO manager, or library > code), > but then why am I fighting the Windows CI literally every day, it makes no > sense. Give an option to skip Windows CI for my patches. > > > How about you use some of that energy to help I stead of taking the easy > way? > > And I bet you're going to say you don't care about Windows to which I > would > > say I don't care about the non-threaded runtime and wish we would get > rid of > > it. But can't always get what you want. > > I'm not suggesting we release buggy GHCs for Windows or stop Windows > support. > I'm sorry, how is disabling the Windows CI not exactly that? If you Disabling the CI just means you test it even less. You test it even less means by the time you get to testing it the issues are too many to fix. Over time you just stop trying and stop releasing it. So sorry, how *exactly* is your suggestion not exactly that. > > > And to say we'll actually fix anything before release doesn't align with > what > > I've seen so far, which had me scrambling last minute to ensure we can > release > > Windows instead of making releases without it. > > Are you saying we skip a platform we support when it's buggy? That makes no > sense. I don't know when did Windows become a first-tier platform but > since it > is now we should be releasing Windows binaries similar to Linux and OSX > binaries. > It's *always* been a tier one platform as far as I can tell. It's certainly been for the past 6 years. > It's not uncommon to do some testing for every patch, and do more > comprehensive > testing before releases. We did this many times in other projects in the > past > and I know some other compilers do this today. > Yes, but a project that doesn't test a tier one platform during development, which is what your want to do means it's not tier one. Which means you won't fix it for release. > > > Quite frankly I don't need you to tell me to submit MRs to fix it since > that's > > what I spent again a lot of time doing. Or maybe you would like to pay my > > paycheck so I can spend more than a considerable amount of my free time > on it. > > I wish someone paid me for the time I wasted because I'm only paid by the > time I > spend productively. I'd be happier waiting for the CI then. > Yeah, not waiting for CI is how we got in this mess in the first place. Tamar. > Ömer > > [1]: https://gitlab.haskell.org/ghc/ghc/-/jobs/237457 > [2]: https://gitlab.haskell.org/osa1/ghc/-/jobs/238236 > [3]: https://gitlab.haskell.org/osa1/ghc/-/jobs/237279 > > Phyx , 17 Oca 2020 Cum, 09:49 tarihinde şunu yazdı: > > > > Oh I spent a non-insignificant amount of time back in the phabricator > days to make the CI stable. Now because people were committing to master > directly without going through CI it was always a cat and mouse game and I > gave up eventually. > > > > Now we have rewritten the CI and it's pointing out actual issues in the > compiler. And your suggestion is well let's just ignore it. > > > > How about you use some of that energy to help I stead of taking the easy > way? And I bet you're going to say you don't care about Windows to which I > would say I don't care about the
Re: Windows testsuite failures
> Now we have rewritten the CI and it's pointing out actual issues in the > compiler. And your suggestion is well let's just ignore it. When is the last time Windows CI caught an actual bug? All I see is random system failures [1, 2, 3]. It must be catching *some* bugs, but that's a rare event in my experience. Sure, I don't write Windows-specific code (e.g. IO manager, or library code), but then why am I fighting the Windows CI literally every day, it makes no sense. Give an option to skip Windows CI for my patches. > How about you use some of that energy to help I stead of taking the easy way? > And I bet you're going to say you don't care about Windows to which I would > say I don't care about the non-threaded runtime and wish we would get rid of > it. But can't always get what you want. I'm not suggesting we release buggy GHCs for Windows or stop Windows support. > And to say we'll actually fix anything before release doesn't align with what > I've seen so far, which had me scrambling last minute to ensure we can release > Windows instead of making releases without it. Are you saying we skip a platform we support when it's buggy? That makes no sense. I don't know when did Windows become a first-tier platform but since it is now we should be releasing Windows binaries similar to Linux and OSX binaries. It's not uncommon to do some testing for every patch, and do more comprehensive testing before releases. We did this many times in other projects in the past and I know some other compilers do this today. > Quite frankly I don't need you to tell me to submit MRs to fix it since that's > what I spent again a lot of time doing. Or maybe you would like to pay my > paycheck so I can spend more than a considerable amount of my free time on it. I wish someone paid me for the time I wasted because I'm only paid by the time I spend productively. I'd be happier waiting for the CI then. Ömer [1]: https://gitlab.haskell.org/ghc/ghc/-/jobs/237457 [2]: https://gitlab.haskell.org/osa1/ghc/-/jobs/238236 [3]: https://gitlab.haskell.org/osa1/ghc/-/jobs/237279 Phyx , 17 Oca 2020 Cum, 09:49 tarihinde şunu yazdı: > > Oh I spent a non-insignificant amount of time back in the phabricator days to > make the CI stable. Now because people were committing to master directly > without going through CI it was always a cat and mouse game and I gave up > eventually. > > Now we have rewritten the CI and it's pointing out actual issues in the > compiler. And your suggestion is well let's just ignore it. > > How about you use some of that energy to help I stead of taking the easy way? > And I bet you're going to say you don't care about Windows to which I would > say I don't care about the non-threaded runtime and wish we would get rid of > it. But can't always get what you want. > > And to say we'll actually fix anything before release doesn't align with what > I've seen so far, which had me scrambling last minute to ensure we can > release Windows instead of making releases without it. > > Quite frankly I don't need you to tell me to submit MRs to fix it since > that's what I spent again a lot of time doing. Or maybe you would like to pay > my paycheck so I can spend more than a considerable amount of my free time on > it. > > Kind regards, > Tamar > > > Sent from my Mobile > > On Fri, Jan 17, 2020, 06:17 Ömer Sinan Ağacan wrote: >> >> We release more often than once in 6 months. >> >> We clearly have no idea how to test on Windows. If you know how to do it then >> feel free to submit a MR. Otherwise blocking every MR indefinitely is worse >> than >> testing Windows less frequently. >> >> Ömer >> >> Phyx , 17 Oca 2020 Cum, 09:10 tarihinde şunu yazdı: >> > >> > Sure because only testing once every 6 months is a very very good idea... >> > >> > Sent from my Mobile >> > >> > On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan wrote: >> >> >> >> Hi Ben, >> >> >> >> Can we please disable Windows CI? I've spent more time fighting the CI >> >> than >> >> doing useful work this week, it's really frustrating. >> >> >> >> Since we have no idea how to fix it maybe we should test Windows only >> >> before a >> >> release, manually (and use bisect in case of regressions). >> >> >> >> Ömer >> >> >> >> Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde şunu >> >> yazdı: >> >> > >> >> > Hi all, >> >> > >> >> > Currently Windows CI is a bit flaky due to some unfortunately rather >> >> > elusive testsuite driver bugs. Progress in resolving this has been a >> >> > bit slow due to travel over the last week but I will be back home >> >> > tomorrow and should be able to resolve the issue soon thereafter. >> >> > >> >> > Cheers, >> >> > >> >> > - Ben >> >> > ___ >> >> > ghc-devs mailing list >> >> > ghc-devs@haskell.org >> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> >> ___ >> >> ghc-devs mailing list >> >> ghc-devs@haskell.org
Re: Windows testsuite failures
Oh I spent a non-insignificant amount of time back in the phabricator days to make the CI stable. Now because people were committing to master directly without going through CI it was always a cat and mouse game and I gave up eventually. Now we have rewritten the CI and it's pointing out actual issues in the compiler. And your suggestion is well let's just ignore it. How about you use some of that energy to help I stead of taking the easy way? And I bet you're going to say you don't care about Windows to which I would say I don't care about the non-threaded runtime and wish we would get rid of it. But can't always get what you want. And to say we'll actually fix anything before release doesn't align with what I've seen so far, which had me scrambling last minute to ensure we can release Windows instead of making releases without it. Quite frankly I don't need you to tell me to submit MRs to fix it since that's what I spent again a lot of time doing. Or maybe you would like to pay my paycheck so I can spend more than a considerable amount of my free time on it. Kind regards, Tamar Sent from my Mobile On Fri, Jan 17, 2020, 06:17 Ömer Sinan Ağacan wrote: > We release more often than once in 6 months. > > We clearly have no idea how to test on Windows. If you know how to do it > then > feel free to submit a MR. Otherwise blocking every MR indefinitely is > worse than > testing Windows less frequently. > > Ömer > > Phyx , 17 Oca 2020 Cum, 09:10 tarihinde şunu yazdı: > > > > Sure because only testing once every 6 months is a very very good idea... > > > > Sent from my Mobile > > > > On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan > wrote: > >> > >> Hi Ben, > >> > >> Can we please disable Windows CI? I've spent more time fighting the CI > than > >> doing useful work this week, it's really frustrating. > >> > >> Since we have no idea how to fix it maybe we should test Windows only > before a > >> release, manually (and use bisect in case of regressions). > >> > >> Ömer > >> > >> Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde > şunu yazdı: > >> > > >> > Hi all, > >> > > >> > Currently Windows CI is a bit flaky due to some unfortunately rather > elusive testsuite driver bugs. Progress in resolving this has been a bit > slow due to travel over the last week but I will be back home tomorrow and > should be able to resolve the issue soon thereafter. > >> > > >> > Cheers, > >> > > >> > - Ben > >> > ___ > >> > ghc-devs mailing list > >> > ghc-devs@haskell.org > >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > >> ___ > >> ghc-devs mailing list > >> ghc-devs@haskell.org > >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
We release more often than once in 6 months. We clearly have no idea how to test on Windows. If you know how to do it then feel free to submit a MR. Otherwise blocking every MR indefinitely is worse than testing Windows less frequently. Ömer Phyx , 17 Oca 2020 Cum, 09:10 tarihinde şunu yazdı: > > Sure because only testing once every 6 months is a very very good idea... > > Sent from my Mobile > > On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan wrote: >> >> Hi Ben, >> >> Can we please disable Windows CI? I've spent more time fighting the CI than >> doing useful work this week, it's really frustrating. >> >> Since we have no idea how to fix it maybe we should test Windows only before >> a >> release, manually (and use bisect in case of regressions). >> >> Ömer >> >> Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde şunu >> yazdı: >> > >> > Hi all, >> > >> > Currently Windows CI is a bit flaky due to some unfortunately rather >> > elusive testsuite driver bugs. Progress in resolving this has been a bit >> > slow due to travel over the last week but I will be back home tomorrow and >> > should be able to resolve the issue soon thereafter. >> > >> > Cheers, >> > >> > - Ben >> > ___ >> > ghc-devs mailing list >> > ghc-devs@haskell.org >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs >> ___ >> ghc-devs mailing list >> ghc-devs@haskell.org >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
Sure because only testing once every 6 months is a very very good idea... Sent from my Mobile On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan wrote: > Hi Ben, > > Can we please disable Windows CI? I've spent more time fighting the CI than > doing useful work this week, it's really frustrating. > > Since we have no idea how to fix it maybe we should test Windows only > before a > release, manually (and use bisect in case of regressions). > > Ömer > > Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde şunu > yazdı: > > > > Hi all, > > > > Currently Windows CI is a bit flaky due to some unfortunately rather > elusive testsuite driver bugs. Progress in resolving this has been a bit > slow due to travel over the last week but I will be back home tomorrow and > should be able to resolve the issue soon thereafter. > > > > Cheers, > > > > - Ben > > ___ > > ghc-devs mailing list > > ghc-devs@haskell.org > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
Hi Ben, Can we please disable Windows CI? I've spent more time fighting the CI than doing useful work this week, it's really frustrating. Since we have no idea how to fix it maybe we should test Windows only before a release, manually (and use bisect in case of regressions). Ömer Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde şunu yazdı: > > Hi all, > > Currently Windows CI is a bit flaky due to some unfortunately rather elusive > testsuite driver bugs. Progress in resolving this has been a bit slow due to > travel over the last week but I will be back home tomorrow and should be able > to resolve the issue soon thereafter. > > Cheers, > > - Ben > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Windows testsuite failures
Hi Simon, Sure I'll do that. I'll create the patches later tonight. Kind regards, Tamar On Mon, Sep 24, 2018, 13:35 Simon Peyton Jones wrote: > Tamar > > > > Thank you, that’s great! > > > > For the ones that need more work, can we mark them as expect-broken, so > that they don’t pollute the testsuite output? > > > Simon > > > > > > *From:* loneti...@gmail.com > *Sent:* 24 September 2018 07:13 > > > *To:* Simon Peyton Jones > *Subject:* RE: Windows testsuite failures > > > > > > Hi Simon, > > > > I created some patches to fix the majority of these > > > > https://phabricator.haskell.org/D5174 > > https://phabricator.haskell.org/D5175 > > https://phabricator.haskell.org/D5176 > > > > The remaining ones I’ve either pinged the patches that caused the issues > or created tickets > > For them because they’re actual bugs that require a bit more time to find > the cause and fix. > > > > https://ghc.haskell.org/trac/ghc/ticket/15668 > > https://ghc.haskell.org/trac/ghc/ticket/15669 > > https://ghc.haskell.org/trac/ghc/ticket/15670 > > https://ghc.haskell.org/trac/ghc/ticket/15671 > > > > these should bring down the amount of failing tests to about 5. > > > > Kind Regards, > > Tamar > > > > *From: *Simon Peyton Jones > *Sent: *Thursday, September 20, 2018 12:05 > *To: *Phyx > *Subject: *RE: Windows testsuite failures > > > > Thanks Tamar. I’ll look forward to hearing back > > > > S > > > > *From:* Phyx > *Sent:* 20 September 2018 12:02 > *To:* Simon Peyton Jones > *Cc:* ghc-devs@haskell.org > *Subject:* Re: Windows testsuite failures > > > > Hi Simon, > > > > Thanks for the email. I haven't been building head much as I'm working on > top of some older commits. From a quick look it seems like the plugin ones > are probably testisms, the plugins aren't found so likely a missing path > entry somewhere. > > > > The linker ones are are weird, I'll need to take a closer look at those, > likely culprit is my recent patch, I had been testing in the 32 bit build > and didn't notice these. > > > > There are a few worrying segfault that shouldn't be there on some random > tests so I'll take a closer look at those too. > > > > And the stat changes need to be updated. > > > > The framework failures I don't see on harbormaster, so think they are > again a threading artifact. Need to figure out a more effective way to > debug these to find a permanent fix. The ones that harbormaster does see > are encoding related. touch is failing on non-ascii names. > > > > I will take a look this weekend. > > > > Kind regards, > > Tamar > > > > On Thu, Sep 20, 2018, 11:33 Simon Peyton Jones > wrote: > > Hi Tamar > > The list of testsuite failure on Windows has grown quite long – see > below. Most seem to concern plugins or linking. > > Do you know what is going on here? If they can’t be fixed, can we mark > them as expect_broken on Windows, so that it’s easier (when developing) to > know when I’ve introduced a regression. Currently I have to do a manual > diff against a rather long list. > > Thanks! > > Simon > > > > *SUMMARY for test run started at Thu Sep 20 00:13:20 2018 GMTST* > > *1:03:11 spent to go through* > > *6530 total tests, which gave rise to* > > * 18728 test cases, of which* > > * 12206 were skipped* > > > > * 33 had missing libraries* > > *6278 expected passes* > > * 173 expected failures* > > > > * 9 caused framework failures* > > * 1 caused framework warnings* > > * 0 unexpected passes* > > * 31 unexpected failures* > > * 7 unexpected stat failures* > > > > *Unexpected failures:* > > * ghci/linking/dyn/T10955dyn.run T10955dyn [bad exit code] > (normal)* > > * ghci/linking/dyn/T10955.run T10955 [bad stderr] (ghci)* > > * ghci/linking/dyn/T11072gcc.run T11072gcc [bad exit code] > (normal)* > > * numeric/should_run/FloatFnInverses.run FloatFnInverses [bad stdout] > (normal)* > > * plugins/T11244.run T11244 [bad stderr] (normal)* > > * plugins/plugin-recomp-change.runplugin-recomp-change [bad exit > code] (normal)* > > * rts/T7040_ghci.run T7040_ghci [bad stdout] (ghci)* > > * rts/linker_unload.run linker_unload [bad exit code] > (normal)* > > * rts/linker_error1.run linker_error1 [bad exit
RE: Windows testsuite failures
Tamar Thank you, that’s great! For the ones that need more work, can we mark them as expect-broken, so that they don’t pollute the testsuite output? Simon From: loneti...@gmail.com Sent: 24 September 2018 07:13 To: Simon Peyton Jones Subject: RE: Windows testsuite failures Hi Simon, I created some patches to fix the majority of these https://phabricator.haskell.org/D5174 https://phabricator.haskell.org/D5175 https://phabricator.haskell.org/D5176 The remaining ones I’ve either pinged the patches that caused the issues or created tickets For them because they’re actual bugs that require a bit more time to find the cause and fix. https://ghc.haskell.org/trac/ghc/ticket/15668 https://ghc.haskell.org/trac/ghc/ticket/15669 https://ghc.haskell.org/trac/ghc/ticket/15670 https://ghc.haskell.org/trac/ghc/ticket/15671 these should bring down the amount of failing tests to about 5. Kind Regards, Tamar From: Simon Peyton Jones<mailto:simo...@microsoft.com> Sent: Thursday, September 20, 2018 12:05 To: Phyx<mailto:loneti...@gmail.com> Subject: RE: Windows testsuite failures Thanks Tamar. I’ll look forward to hearing back S From: Phyx mailto:loneti...@gmail.com>> Sent: 20 September 2018 12:02 To: Simon Peyton Jones mailto:simo...@microsoft.com>> Cc: ghc-devs@haskell.org<mailto:ghc-devs@haskell.org> Subject: Re: Windows testsuite failures Hi Simon, Thanks for the email. I haven't been building head much as I'm working on top of some older commits. From a quick look it seems like the plugin ones are probably testisms, the plugins aren't found so likely a missing path entry somewhere. The linker ones are are weird, I'll need to take a closer look at those, likely culprit is my recent patch, I had been testing in the 32 bit build and didn't notice these. There are a few worrying segfault that shouldn't be there on some random tests so I'll take a closer look at those too. And the stat changes need to be updated. The framework failures I don't see on harbormaster, so think they are again a threading artifact. Need to figure out a more effective way to debug these to find a permanent fix. The ones that harbormaster does see are encoding related. touch is failing on non-ascii names. I will take a look this weekend. Kind regards, Tamar On Thu, Sep 20, 2018, 11:33 Simon Peyton Jones mailto:simo...@microsoft.com>> wrote: Hi Tamar The list of testsuite failure on Windows has grown quite long – see below. Most seem to concern plugins or linking. Do you know what is going on here? If they can’t be fixed, can we mark them as expect_broken on Windows, so that it’s easier (when developing) to know when I’ve introduced a regression. Currently I have to do a manual diff against a rather long list. Thanks! Simon SUMMARY for test run started at Thu Sep 20 00:13:20 2018 GMTST 1:03:11 spent to go through 6530 total tests, which gave rise to 18728 test cases, of which 12206 were skipped 33 had missing libraries 6278 expected passes 173 expected failures 9 caused framework failures 1 caused framework warnings 0 unexpected passes 31 unexpected failures 7 unexpected stat failures Unexpected failures: ghci/linking/dyn/T10955dyn.run T10955dyn [bad exit code] (normal) ghci/linking/dyn/T10955.run T10955 [bad stderr] (ghci) ghci/linking/dyn/T11072gcc.run T11072gcc [bad exit code] (normal) numeric/should_run/FloatFnInverses.run FloatFnInverses [bad stdout] (normal) plugins/T11244.run T11244 [bad stderr] (normal) plugins/plugin-recomp-change.runplugin-recomp-change [bad exit code] (normal) rts/T7040_ghci.run T7040_ghci [bad stdout] (ghci) rts/linker_unload.run linker_unload [bad exit code] (normal) rts/linker_error1.run linker_error1 [bad exit code] (normal) rts/linker_error2.run linker_error2 [bad exit code] (normal) rts/T12771/T12771.run T12771 [bad exit code] (normal) rts/T13082/T13082_good.run T13082_good [bad exit code] (normal) rts/T14611/T14611.run T14611 [bad exit code] (normal) simplCore/should_compile/T7702.run T7702 [exit code non-0] (normal) rts/T10672/T10672_x64.run T10672_x64 [bad exit code] (normal) libraries/Win32/tests/T4452.run T4452 [bad exit code] (normal) plugins/plugins01.run plugins01 [bad exit code] (normal) plugins/plugins07.run plugins07 [bad exit code] (normal) plugins/plugins09.run plugins09 [bad exit code] (normal) plugins/plugins11.run plugins11 [bad exit code] (normal) plugins/plugins12.run plugins12 [bad exit code] (normal) plugins/plugins13.run plugins13
Re: Windows testsuite failures
Hi Simon, Thanks for the email. I haven't been building head much as I'm working on top of some older commits. From a quick look it seems like the plugin ones are probably testisms, the plugins aren't found so likely a missing path entry somewhere. The linker ones are are weird, I'll need to take a closer look at those, likely culprit is my recent patch, I had been testing in the 32 bit build and didn't notice these. There are a few worrying segfault that shouldn't be there on some random tests so I'll take a closer look at those too. And the stat changes need to be updated. The framework failures I don't see on harbormaster, so think they are again a threading artifact. Need to figure out a more effective way to debug these to find a permanent fix. The ones that harbormaster does see are encoding related. touch is failing on non-ascii names. I will take a look this weekend. Kind regards, Tamar On Thu, Sep 20, 2018, 11:33 Simon Peyton Jones wrote: > Hi Tamar > > The list of testsuite failure on Windows has grown quite long – see > below. Most seem to concern plugins or linking. > > Do you know what is going on here? If they can’t be fixed, can we mark > them as expect_broken on Windows, so that it’s easier (when developing) to > know when I’ve introduced a regression. Currently I have to do a manual > diff against a rather long list. > > Thanks! > > Simon > > > > *SUMMARY for test run started at Thu Sep 20 00:13:20 2018 GMTST* > > *1:03:11 spent to go through* > > *6530 total tests, which gave rise to* > > * 18728 test cases, of which* > > * 12206 were skipped* > > > > * 33 had missing libraries* > > *6278 expected passes* > > * 173 expected failures* > > > > * 9 caused framework failures* > > * 1 caused framework warnings* > > * 0 unexpected passes* > > * 31 unexpected failures* > > * 7 unexpected stat failures* > > > > *Unexpected failures:* > > * ghci/linking/dyn/T10955dyn.run T10955dyn [bad exit code] > (normal)* > > * ghci/linking/dyn/T10955.run T10955 [bad stderr] (ghci)* > > * ghci/linking/dyn/T11072gcc.run T11072gcc [bad exit code] > (normal)* > > * numeric/should_run/FloatFnInverses.run FloatFnInverses [bad stdout] > (normal)* > > * plugins/T11244.run T11244 [bad stderr] (normal)* > > * plugins/plugin-recomp-change.runplugin-recomp-change [bad exit > code] (normal)* > > * rts/T7040_ghci.run T7040_ghci [bad stdout] (ghci)* > > * rts/linker_unload.run linker_unload [bad exit code] > (normal)* > > * rts/linker_error1.run linker_error1 [bad exit code] > (normal)* > > * rts/linker_error2.run linker_error2 [bad exit code] > (normal)* > > * rts/T12771/T12771.run T12771 [bad exit code] > (normal)* > > * rts/T13082/T13082_good.run T13082_good [bad exit code] > (normal)* > > * rts/T14611/T14611.run T14611 [bad exit code] > (normal)* > > * simplCore/should_compile/T7702.run T7702 [exit code non-0] > (normal)* > > * rts/T10672/T10672_x64.run T10672_x64 [bad exit code] > (normal)* > > * libraries/Win32/tests/T4452.run T4452 [bad exit code] (normal)* > > * plugins/plugins01.run plugins01 [bad exit code] > (normal)* > > * plugins/plugins07.run plugins07 [bad exit code] > (normal)* > > * plugins/plugins09.run plugins09 [bad exit code] > (normal)* > > * plugins/plugins11.run plugins11 [bad exit code] > (normal)* > > * plugins/plugins12.run plugins12 [bad exit code] > (normal)* > > * plugins/plugins13.run plugins13 [bad exit code] > (normal)* > > * plugins/plugins14.run plugins14 [bad exit code] > (normal)* > > * plugins/plugins15.run plugins15 [bad exit code] > (normal)* > > * plugins/T10420.run T10420 [bad exit code] > (normal)* > > * plugins/T10294.run T10294 [bad exit code] > (normal)* > > * plugins/T10294a.run T10294a [bad exit code] > (normal)* > > * plugins/T12567a.run T12567a [bad exit code] > (normal)* > > * plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit > code] (normal)* > > * plugins/plugin-recomp-impure.runplugin-recomp-impure [bad exit > code] (normal)* > > * plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit > code] (normal)* > > > > *Unexpected stat failures:* > > * perf/compiler/T9872d.run T9872d [stat not good enough] > (normal)* > > * perf/compiler/T12425.run T12425 [stat not good enough] > (optasm)* > > * perf/compiler/T12234.run T12234 [stat not good enough] > (optasm)* > > * perf/compiler/T12150.run T12150 [stat not good enough] >
