Re: Windows/R issue with FFI

[Phyx]

Thanks, that does indeed look dynamically linked.

Could you also paste on the ticket the contents of hR.buildinfo?

Cheers,
Tamar

Sent from my Mobile

On Mon, Mar 27, 2023, 15:18 Dominick Samperi  wrote:

> Yes, everything else stays the  same, including x <- r_NilValue.
>
> I opened a ticket here where more details are provided
> https://gitlab.haskell.org/ghc/ghc/-/issues/23183
>
> After initializing an R instance, if you fetch R_NilValue and
> peek at its value (using FFI peek) you get a bad address. But if
> you add a trace statement before the peek the address is valid.
>
> A "race condition" should not be possible in a single-threaded
> application, so I am not sure what is going on. I tried to come
> up with a simple reproducible example where a library module does
> nothing but fetch R_NilValue, and the client also uses FFI to fetch
> R_NilValue, but in this example both addresses are valid and equal.
>
>
>
>
> <http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
> Virus-free.www.avg.com
> <http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
> <#m_7940054291392109769_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> On Mon, Mar 27, 2023 at 9:48 AM Phyx  wrote:
>
>> Hi,
>>
>> I'm missing some details here here as I'm having trouble following the
>> flow.
>>
>> What provides the symbol for that import? As in where does R_NilValue
>> come from? As in, how is it defined. Are you linking against a library or C
>> sources?
>>
>> When you say you replace the trace statement, do you keep the x <-
>> r_NilValue?
>>
>> The address to R_NilValue should never change during initialization so
>> I'm more suspicious of how it's declared. Unless you're linking to a symbol
>> in a shared library, in which case that could be possible due to ASLR.
>>
>> Kind regards,
>> Tamar
>>
>> Sent from my Mobile
>>
>> On Sun, Mar 26, 2023, 14:15 Dominick Samperi  wrote:
>>
>>> Thanks Ben, I'll see what I can do to reliably reproduce and open a
>>> ticket.
>>>
>>> One theory I'm investigating is that this might have something to do
>>> with my anti-virus software (AVG), since it sometimes interacts with
>>> Windows in strange ways (for example, an extra instance of a terminal
>>> app pops up, then disappears after a few seconds). But disabling this
>>> software does not seem to solve the problem.
>>>
>>> On Sat, Mar 25, 2023 at 11:18 PM Ben Gamari 
>>> wrote:
>>>
>>>> This sounds like a bug. Could you open a ticket, ideally with a fairly
>>>> standalone reproducer?
>>>>
>>>> Cheer,
>>>>
>>>> - Ben
>>>>
>>>> On March 25, 2023 6:49:09 PM EDT, Dominick Samperi 
>>>> wrote:
>>>>>
>>>>> Hello,
>>>>> FFI code that used to work now fails under Windows (still seems to work
>>>>> under Ubuntu), and I wonder if anybody has seen anything like this and
>>>>> can provide some pointers...
>>>>>
>>>>> The code uses FFI to fetch information from the R side like R_NilValue,
>>>>> using something like this;
>>>>>
>>>>> -- Fetch R's R_NilValue...
>>>>> foreign import ccall unsafe "_NilValue" r_NilValue_ptr :: Ptr R_EXP
>>>>> r_NilValue :: IO R_EXP
>>>>> r_NilValue = peek r_NilValue_ptr
>>>>> rNilValue1 :: IO REXP
>>>>> rNilValue1 = do
>>>>> x <- r_NilValue
>>>>> traceShow("addr=",x) extREXP x
>>>>>
>>>>> Under Windows the address displayed is obviously bad, and this causes
>>>>> the app to crash. This does not happen under Linux (Ubuntu).
>>>>>
>>>>> Now, replace the line containing peek with
>>>>>
>>>>> r_NilValue = trace "PEEK" peek r_NilValue_ptr
>>>>>
>>>>> The address is now valid! It seems that adding the trace "PEEK" adds
>>>>> some delay and somehow resolves the problem.
>>>>>
>>>>> This problem is intermittent, so it is hard to come up with a
>>>>> simple example that fails every time.
>>>>>
>>>>> A little background: R_NilValue is a pointer to a SEXP that is not
>>>>> initialized until an embedded instance of R is initialized, and the
>>>>> code above is not triggered until this happens. Perhaps there is
>>>>> a race condition between the time R initializes itself and Haskell
>>>>> performs the peek? I don't think R_NilValue is garbage collected
>>>>> once initialized.
>>>>>
>>>>> Any tips would be appreciated.
>>>>> Dominick
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ___
>>> ghc-devs mailing list
>>> ghc-devs@haskell.org
>>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>>
>>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows/R issue with FFI

[Phyx]

Hi,

I'm missing some details here here as I'm having trouble following the
flow.

What provides the symbol for that import? As in where does R_NilValue come
from? As in, how is it defined. Are you linking against a library or C
sources?

When you say you replace the trace statement, do you keep the x <-
r_NilValue?

The address to R_NilValue should never change during initialization so I'm
more suspicious of how it's declared. Unless you're linking to a symbol in
a shared library, in which case that could be possible due to ASLR.

Kind regards,
Tamar

Sent from my Mobile

On Sun, Mar 26, 2023, 14:15 Dominick Samperi  wrote:

> Thanks Ben, I'll see what I can do to reliably reproduce and open a ticket.
>
> One theory I'm investigating is that this might have something to do
> with my anti-virus software (AVG), since it sometimes interacts with
> Windows in strange ways (for example, an extra instance of a terminal app
> pops up, then disappears after a few seconds). But disabling this software
> does not seem to solve the problem.
>
> On Sat, Mar 25, 2023 at 11:18 PM Ben Gamari  wrote:
>
>> This sounds like a bug. Could you open a ticket, ideally with a fairly
>> standalone reproducer?
>>
>> Cheer,
>>
>> - Ben
>>
>> On March 25, 2023 6:49:09 PM EDT, Dominick Samperi 
>> wrote:
>>>
>>> Hello,
>>> FFI code that used to work now fails under Windows (still seems to work
>>> under Ubuntu), and I wonder if anybody has seen anything like this and
>>> can provide some pointers...
>>>
>>> The code uses FFI to fetch information from the R side like R_NilValue,
>>> using something like this;
>>>
>>> -- Fetch R's R_NilValue...
>>> foreign import ccall unsafe "_NilValue" r_NilValue_ptr :: Ptr R_EXP
>>> r_NilValue :: IO R_EXP
>>> r_NilValue = peek r_NilValue_ptr
>>> rNilValue1 :: IO REXP
>>> rNilValue1 = do
>>> x <- r_NilValue
>>> traceShow("addr=",x) extREXP x
>>>
>>> Under Windows the address displayed is obviously bad, and this causes
>>> the app to crash. This does not happen under Linux (Ubuntu).
>>>
>>> Now, replace the line containing peek with
>>>
>>> r_NilValue = trace "PEEK" peek r_NilValue_ptr
>>>
>>> The address is now valid! It seems that adding the trace "PEEK" adds
>>> some delay and somehow resolves the problem.
>>>
>>> This problem is intermittent, so it is hard to come up with a
>>> simple example that fails every time.
>>>
>>> A little background: R_NilValue is a pointer to a SEXP that is not
>>> initialized until an embedded instance of R is initialized, and the
>>> code above is not triggered until this happens. Perhaps there is
>>> a race condition between the time R initializes itself and Haskell
>>> performs the peek? I don't think R_NilValue is garbage collected
>>> once initialized.
>>>
>>> Any tips would be appreciated.
>>> Dominick
>>>
>>>
>>>
>>>
>>>
>>> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Buy-in for technical proposal 47 which affect GHC devs

[Phyx]

> What am I missing?


Hi Ben,

I'm talking about dynamic-too support, which I am still working on getting
back in Windows.

In the grand scheme of things it doesn't matter much, as it doesn't work
today. But a forwarder library in that scenario still will have a compile
time cost in the current scheme for how dynamic-too works on Windows.

So I wanted to see if the implementation cost isn't high if we could do the
library split now.

But if it is then no problem, I'll find a work around.

Thanks,
Tamar

On Sat, Mar 25, 2023, 16:54 Ben Gamari  wrote:

> Phyx  writes:
>
> >> I highly doubt that this split will have any measurable overhead.
> >> Reexporting a definition defined in one module from another module via
> >> an export list does not produce any code at all; importing such a
> >> declaration is equivalent to importing the definition from the defining
> >> module.
> >
> > Ah right, I can see how that's true at the Haskell level but..
> >
> >> If for some reason we can't in some cases directly reexport then we
> >> would likely rather have a some very trivial bindings that GHC would be
> >> quite eager to inline.
> >
> > Sure, I can see how you'd inline based on the haskell contract, I can't
> see
> > how you avoid the compile time overhead when compiling the library. If
> you
> > have a haskell library
> >
>
> > module Test (Control.Monad.when, Control.Applicative.many) where
> >
> > import Control.Monad(when)
> > import Control.Applicative(many)
> >
> > compiling it:
> >
> >  ghc test.hs
> > [1 of 1] Compiling Test ( test.hs, test.o )
> >
> > which still contains the closure for the library.
>
> I'm a bit lost. When I compile this module I get an object which
> contains no references to `when` or `many`. The only references to these
> two symbols should be in the interface file. What am I missing?
>
> Cheers,
>
> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Buy-in for technical proposal 47 which affect GHC devs

[Phyx]

> I highly doubt that this split will have any measurable overhead.
> Reexporting a definition defined in one module from another module via
> an export list does not produce any code at all; importing such a
> declaration is equivalent to importing the definition from the defining
> module.

Ah right, I can see how that's true at the Haskell level but..

> If for some reason we can't in some cases directly reexport then we
> would likely rather have a some very trivial bindings that GHC would be
> quite eager to inline.

Sure, I can see how you'd inline based on the haskell contract, I can't see
how you avoid the compile time overhead when compiling the library. If you
have a haskell library

module Test (Control.Monad.when, Control.Applicative.many) where

import Control.Monad(when)
import Control.Applicative(many)

compiling it:

 ghc test.hs
[1 of 1] Compiling Test ( test.hs, test.o )

which still contains the closure for the library.  On Windows where GHC
forces the use of --*export*-*all*-symbols with dynamic-too this will not
result in no code.
in fact, it will result in exactly the *same* copy of code as in base
inside the shared library:

--export-all-symbols

Treat all global and weak defined symbols found in the input object files
as symbols to be exported. There is a small list of symbols which are not
exported by default; see the --no-default-excludes option. You may add to
the list of symbols to not export by using the --exclude-symbols option.
At runtime you're right that you can avoid the extra calls (forgot about
re-exportation through module definition) because the library becomes
unused,
but you don't avoid it at compile and link time in all cases.

Yes, --*export*-*all*-symbols is horrible but that's how it works today
because GHC does not support symbol visibility correctly.

So unless there's a very good reason, I still think that it's better for
*all* platforms to just move the code as opposed to re-export them, less we
make it even
harder still to support dynamic-too on Windows (though maybe that's ok and
GHC should be fixed).

Thanks,
Tamar

On Fri, Mar 24, 2023, 21:18 Ben Gamari  wrote:

> Phyx  writes:
>
> > Hi,
> >
> > Though I'm no longer a very active GHC developer I do have some
> questions.
> >
> > Overall I'm in support of this but I think I'd rather see an outright
> split
> > from the start rather than first having a forwarder library.
> >
> > The reason for this is that I'm afraid of the performance impact of
> having
> > this intermediate layer.
> >
> > For statically linked programs this means at least an additional load and
> > branch on every call to a standard library. This would for instance
> affect
> > Windows quite heavily. I'm not sure the impact is mitigated by branch
> > prediction and prefetching. At the least it'll polute your L2 cache much
> > more than before.
> >
> > For dynamically linked we could potentially use symbol preemption to
> remove
> > the forwarding or on Windows redirect using import libraries.
> >
> > Now maybe I'm overestimating the impact this would have, but I'd very
> much
> > like to see some numbers on a small-ish experiment to see what impact (if
> > any) there are and what mitigation we can do.
> >
> > Typically it's quite hard to optimize after the fact. Maybe I've missed
> it
> > in there. Proposal, but can the compiler remove the forwarding? i.e. Can
> > the calls be specialized directly to the definition one? If so it'll
> break
> > having alternative standard libs at runtime?
> >
> I highly doubt that this split will have any measurable overhead.
> Reexporting a definition defined in one module from another module via
> an export list does not produce any code at all; importing such a
> declaration is equivalent to importing the definition from the defining
> module.
>
> If for some reason we can't in some cases directly reexport then we
> would likely rather have a some very trivial bindings that GHC would be
> quite eager to inline.
>
> Cheers,
>
> - Ben
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Buy-in for technical proposal 47 which affect GHC devs

[Phyx]

Hi,

Though I'm no longer a very active GHC developer I do have some questions.

Overall I'm in support of this but I think I'd rather see an outright split
from the start rather than first having a forwarder library.

The reason for this is that I'm afraid of the performance impact of having
this intermediate layer.

For statically linked programs this means at least an additional load and
branch on every call to a standard library. This would for instance affect
Windows quite heavily. I'm not sure the impact is mitigated by branch
prediction and prefetching. At the least it'll polute your L2 cache much
more than before.

For dynamically linked we could potentially use symbol preemption to remove
the forwarding or on Windows redirect using import libraries.

Now maybe I'm overestimating the impact this would have, but I'd very much
like to see some numbers on a small-ish experiment to see what impact (if
any) there are and what mitigation we can do.

Typically it's quite hard to optimize after the fact. Maybe I've missed it
in there. Proposal, but can the compiler remove the forwarding? i.e. Can
the calls be specialized directly to the definition one? If so it'll break
having alternative standard libs at runtime?

Kind regards,
Tamar

On Fri, Mar 24, 2023, 04:47 Ben Gamari  wrote:

> "Adam Sandberg Eriksson"  writes:
>
> > I switched all the HADDOCK hide to not-home in base a couple of years
> > ago, but I see a couple of new ones have snuck in in the meantime. I
> > would suggest adding a lint against hiding in GHC CI.
> >
> Sounds reasonable to me.
> Perhaps someone could open a ticket to track this?
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: The GHC(i)/RTS linker and Template Haskell

[Phyx]

Hi Alexis,

Most information on this can be found on the Wiki,  where a lot of these
design decisions are made. e.g.
https://gitlab.haskell.org/ghc/ghc/-/wikis/dynamic-ghc-programs

The points you've figured out are correct so far, to answer some of your
questions:

> But I don’t actually understand what interpreterDynamic means! The
Haddock comment just says that it determines whether or not the
“interpreter uses the Dynamic way”, but I don’t see why that matters. My
understanding was that GHCi *always* requires dynamic linking, since it is,
after all, loading code dynamically.

DynamicWay essentially means whether or not the runtime linker uses the
platform linker under the hood. When dynamic way your object files will be
linked into a shared library by the RTS linker and that shared library
loaded.
This means that the linker itself doesn't have to do a bunch of work such
as relocation processing etc.  For most Unix platforms this is the default.

The downside of this approach is that on every change, i.e. if you load a
new object file into scope, you have to relink the shared library, unload
the old one, and link the new one in. This brings with it its own set of
problems, such as what happens to references you already hold to symbols on
the old shared library etc.

> Under what circumstances would interpreterDynamic ever be False?

For instance, on Windows. Linking on Windows using the system linker is
generally slower, so creating multiple shared libraries on the fly is time
consuming. There are also some practical issues, for instance base is so
big that it doesn't fit into a single DLL.
or how Windows handles data and code accesses to symbols Shared libraries,
etc.  This means that on Windows we load object files and internally do all
relocation processing, run initializers etc.  Everything you would need to
do to be able to run the code inside the object file.

There are several other platforms as well, such as Android, where there's
no system linker to call etc.

> In the case that interpreterDynamic is True, GHC appears to convert the
desired dyn_o object into a shared library by calling the system linker,
then loads that, which can be very slow but otherwise works. However, when
interpreterDynamic is False, it loads the object directly. Both paths
eventually call into “the RTS linker”, implemented in rts/Linker.c, to
actually load the resulting object.

Yes, the end goal is to be able to resolve a function name to an address.
So whichever strategy is chosen, we must in the end register the functions
with the RTS.  Though loading a shared lib is much less error prone than
loading the object files directly. It also uses less memory and can benefit
from linker level optimizations that we don't implement in the RTS linker.
Also loading a shared library has additional benefits such as that the
system loader deals with running initializers, registering exception
tables, etc.

Hope this clarified it somewhat, but if you have any more questions feel
free to ask.

Regards,
Tamar

On Wed, Jun 1, 2022 at 2:38 AM Alexis King  wrote:

> Hi all,
>
> I’ve recently been trying to better understand how and where time is spent
> at compile-time when running Template Haskell splices, and one of the areas
> I’ve been struggling to figure out is the operation of the linker. From
> reading the source code, here’s a summary of what I think I’ve figured out
> so far:
>
>- TH splices are executed using the GHCi interpreter, though it may be
>internal or external (if -fexternal-interpreter is used).
>
>- Regardless of which mode is used, TH splices need their dependencies
>loaded into the interpreter context before they can be run. This is handled
>by the call to loadDecls in hscCompileCoreExpr', which in turn calls
>loadDependencies in GHC.Linker.Loader.
>
>- loadDependencies loads packages and modules in different ways.
>Package dependencies are just loaded via the appropriate built shared
>libraries, but modules from the current package have to be loaded a
>different way, via loadObjects (also in GHC.Linker.Loader).
>
> Here, however, is where I get a bit lost. GHC has two strategies for
> loading individual objects, which it chooses between depending on whether
> the current value of interpreterDynamic is True. But I don’t actually
> understand what interpreterDynamic means! The Haddock comment just says
> that it determines whether or not the “interpreter uses the Dynamic way”,
> but I don’t see why that matters. My understanding was that GHCi *always*
> requires dynamic linking, since it is, after all, loading code dynamically.
> Under what circumstances would interpreterDynamic ever be False?
>
> Furthermore, I don’t actually understand precisely how and why this
> influences the choice of loading strategy. In the case that
> interpreterDynamic is True, GHC appears to convert the desired dyn_o object
> into a shared library by calling the system linker, then loads that, 

Install WIP app in haskell org

[Phyx]

Hi Ben,

Is it possible to install the WIP app into the github Haskell org?
https://github.com/apps/wip

It's quite handy to mark things as in progress.

Regards,
Tamar
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 9.2.1-alpha1 now available

[Phyx]

Hi,

Typically a GHC release is tied to a new cabal release as well. I however
can't find a new tag for Cabal.

Does this mean I need to use cabal-head or can I use Cabal 3.4?

Thanks,
Tamar

On Thu, Apr 1, 2021 at 5:44 PM Ben Gamari  wrote:

> Hi all,
>
> The GHC developers are very happy to announce the availability of the
> first alpha release in the 9.2.1 series. Binary distributions, source
> distributions, and documentation are available at
>
>  https://downloads.haskell.org/ghc/9.2.1-alpha1
>
> GHC 9.2 will bring a number of exciting features including:
>
>  * Many changes in the area of records, including the new
>`RecordDotSyntax` and `NoFieldSelectors` language extensions, as well
>as Support for `DuplicateRecordFields` with `PatternSynonyms`.
>
>  * Introduction of the new `GHC2021` language extension set, giving
>users convenient access to a larger set of language extensions which
>have been long considered stable.
>
>  * Merge of `ghc-exactprint` into the GHC tree, providing infrastructure
>for source-to-source program rewriting out-of-the-box.
>
>  * Introduction of a `BoxedRep` `RuntimeRep`, allowing for polymorphism
>over levity of boxed objects (#17526)
>
>  * Implementation of the `UnliftedDataTypes` extension, allowing users
>to define types which do not admit lazy evaluation ([proposal])
>
>  * The new [-hi profiling] mechanism which provides significantly
>improved insight into thunk leaks.
>
>  * Support for the `ghc-debug` out-of-process heap inspection library
>[ghc-debug]
>
>  * Support for profiling of pinned objects with the cost-centre profiler
>(#7275)
>
>  * Introduction of Haddock documentation support in TemplateHaskell (#5467)
>
> In addition, the final 9.2.1 release will bring a new native code
> generator for ARM, providing fast, first-class for Haskell on Apple
> ARM hardware [apple-m1], although this functionality is not yet
> present in this alpha.
>
> As always, do give this a try and open a [ticket] if you see anything
> amiss.
>
> Happy testing,
>
> - Ben
>
>
> [apple-m1]: https://www.haskell.org/ghc/blog/20210309-apple-m1-story.html
> [proposal]:
> https://github.com/ghc-proposals/ghc-proposals/blob/master/proposals/0265-unlifted-datatypes.rst
> [-hi
> 
> profiling]:
> https://well-typed.com/blog/2021/01/first-look-at-hi-profiling-mode/
> [ghc-debug
> ]:
> http://ghc.gitlab.haskell.org/ghc-debug/
> [ticket]: https://gitlab.haskell.org/ghc/ghc/-/issues/new
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Options for targeting Windows XP?

[Phyx]

Hi,

>  XP. GHCJS seems to at least have a compiler based on GHC 8.6.
> 2. Patch GHC with an additional command line argument to produce XP/Vista
compatible executables, perhaps by looking at the changes between 7.10 ->
8.0, and re-introducing the XP approach as an option.

This would be somewhat hard but not impossible for 8.0.. Which If I
recalled drop XP for some linker functionality. The higher you go the more
difficult it would become though.

When you get to 9.0 you don't have much hope as there it's not just the
linker, but the RTS itself heavily relies on functionality not available in
XP, including how we manage memory and do synchronization.

It's however not just GHC that would need patching but libraries such as
process as well. That is not to say it's impossible, just you'd have to
find ways to work around the bugs that caused us to change APIs to begin
with...

I can't speak for the community, but I wouldn't want to re-introduce XP as
a supported options in mainline. Parts of e.g. 9.0 (like winio) just won't
work on XP. The design itself is centered around new APIs. So supporting XP
means essentially a new design.

Kind regards,
Tamar

Sent from my Mobile

On Wed, Mar 24, 2021, 14:09 Clinton Mead  wrote:

> I'm currently trying to bring my company around to using a bit of Haskell.
> One issue is that a number of our clients are based in South East Asia and
> need software that runs on Windows XP.
>
> Unfortunately it seems the last version of GHC that produces executables
> that run on Windows XP is GHC 7.10. Whilst this table
>  suggests
> the issue may only running GHC 8.0+ on Windows XP, I've confirmed that GHC
> 8.0 executables (even "Hello World") will not run on Windows XP, presumably
> because a non-XP WinAPI call in the runtime.
>
> My first thought would be to restrict myself to GHC 7.10 features (i.e.
> 2015). This would be a slight annoyance but GHC 7.10 still presents a
> reasonable language. But my concern would be that increasingly I'll run
> into issues with libraries that use extensions post GHC 7.10, particularly
> libraries with large dependency lists.
>
> So there's a few options I've considered at this point:
>
> 1. Use GHCJS to compile to Javascript, and then dig out a version of
> NodeJS that runs on Windows XP. GHCJS seems to at least have a compiler
> based on GHC 8.6.
> 2. Patch GHC with an additional command line argument to produce XP/Vista
> compatible executables, perhaps by looking at the changes between 7.10 ->
> 8.0, and re-introducing the XP approach as an option.
>
> The issue with 1 is that is that as well as being limited by how up to
> date GHCJS is, this will increase install size, memory usage and decrease
> performance on Windows XP machines, which are often in our environments
> quite old and resource and memory constrained.
>
> Approach 2 is something I'd be willing to put some work into if it was
> practical, but my thought is that XP support was removed for a reason,
> presumably by using newer WinAPI functions simplified things significantly.
> By re-adding in XP support I'd be complicating GHC once again, and GHC will
> effectively have to maintain two approaches. In addition, in the long term,
> whenever a new WinAPI call is added one would now have to check whether
> it's available in Windows XP, and if it's not produce a Windows XP
> equivalent. That might seem like just an extra burden of support for
> already busy GHC developers. But on the other hand, if the GHC devs would
> be happy to merge a patch and keep up XP support this would be the cleanest
> option.
>
> But then I had a thought. If GHC Core isn't supposed to change much
> between versions is it? Which made me come up with these approaches:
>
> 3. Hack up a script to compile programs using GHC 9 to Core, then feed
> that Core output into GHC 7.10. OR
> 4. Produce a chimera style GHC by importing the GHC 9.0 API and the GHC
> 7.10 API, and making a version of GHC that does Haskell -> Core in GHC 9.0
> and the rest of the code generation in GHC 7.10.
>
> One issue with 4 will be that presumably that because I'm importing GHC
> 9.0 API and the 7.10 API separately, all their data types will technically
> be separate, so I'll need to basically deep copy the GHC 9.0 core datatype
> (and perhaps others) to GHC 7.10 datatypes. But presuming their largely
> similar this should be fairly mechanical.
>
> So are any of these approaches (well, particularly 2 and 4) reasonable? Or
> am I going to run into big problems with either of them? Is there another
> approach I haven't thought of?
>
> Thanks,
> Clinton
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.10 backports?

[Phyx]

Hi,

I currently have https://gitlab.haskell.org/ghc/ghc/-/merge_requests/5055
marked for backports but don't know if it was done or not.

Thanks,
Tamar

Sent from my Mobile

On Mon, Mar 22, 2021, 04:33 Moritz Angermann 
wrote:

> Hi there!
>
> Does anyone have any backports they'd like to see for consideration for
> 8.10.5?
>
> Cheers,
>  Moritz
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: What changed between GHC 8.8 and 8.10 that could cause this?

[Phyx]

Hi,

> But we don't _want_ the shared state, it's simply there.
> This whole issue arises from the fact that we were oblivious to the
shared RTS state, resulting in Clash doing GHC API calls where the RTS
loads/links an object file twice.

The RTS should under no circumstances be actually loading an object file
twice as there's only one linker map and should result in a symbol
collision.

Looking at the error you posted at
https://github.com/clash-lang/clash-compiler/issues/1686 is actually the
linker doing the right thing.

GHC runtime linker: fatal error: I found a duplicate definition for symbol
   Lib2_plots2_closure
whilst processing object file
   
.stack-work/dist/x86_64-linux-tinfo6/Cabal-3.2.1.0/build/exe2/_clashilator/clash-syn/Lib2.o
The symbol was previously defined in
   
.stack-work/dist/x86_64-linux-tinfo6/Cabal-3.2.1.0/build/exe1/_clashilator/clash-syn/Lib2.o

You're loading the same object file twice from different build folders and
the linker has no guarantee that these two are the same symbol at all.
This however is indeed a shortcoming of M388 that we can't split the C
linker map easily.

> And we're not even explicitly linking/loading object files twice,
something to do with the GHC type-checker seems to do that.

Yes but you have a new object file, in a different path. This can't be
resolved by the linker cache.  This looks like it accidentally worked
before as the shared Haskell Linker state resolves based on the Close name
itself.
So it never asked the C linker. I say accidental because there's no
guarantee that the closure in exe1 and exe2 are the same, despite them
having the same name..

> I don't see how I can avoid this issue without being forced to run within
a single `runGhc` session.

As I mentioned below, you can override the hsc_dynLinker in a wrapper
around runGhc.

i.e.

runClashGhc ::  -> ..
  do shared_linker <- ...
 runGhc .. $ do
   setSession $ hsc_env { hsc_dynLinker = shared_linker }


Should restore the behavior. You don't need to run inside a single runGhc,
you just need to provide a single hsc_dynLinker.

That should work.

Kind Regards,
Tamar

On Tue, Mar 9, 2021, 11:46 Christiaan Baaij 
wrote:

> But we don't _want_ the shared state, it's simply there.
> This whole issue arises from the fact that we were oblivious to the shared
> RTS state, resulting in Clash doing GHC API calls where the RTS loads/links
> an object file twice.
> And we're not even explicitly linking/loading object files twice,
> something to do with the GHC type-checker seems to do that.
> I don't see how I can avoid this issue without being forced to run within
> a single `runGhc` session.
>
> On Tue, 9 Mar 2021 at 12:22, Phyx  wrote:
>
>> Hi,
>>
>> Hmm... I don't agree..
>>
>> This isn't about grounds of truth or anything like that.. and in fact, an
>> object being in the linker map, doesn't mean its usable at all or meant to
>> be used at all.
>> It can be temporary state (symbol redirection or supporting of deprecated
>> symbols are two that come to mind).  So this is also a case of.. be careful.
>>
>> The change introduced in the MR simply decoupled the top level user
>> interface and the C linker.
>> The reason for this is simply because the majority of projects do not
>> require shared state here, but infact benefit from unshared state.
>>
>> i.e. interpreters, IDEs etc.  Where you want to be able to process
>> multiple separate files at the same time without needing to create new
>> processes for each.
>>
>> Now back to your point about runGhc needing to use a shared state.. In my
>> opinion that would be wrong.
>>
>> Here's the documentation for GHC 8.6.5
>> https://hackage.haskell.org/package/ghc-8.6.5/docs/GHC.html
>>
>> specifically:
>>
>> 
>>
>> runGhc
>>   :: Maybe FilePath- See argument to initGhcMonad.
>>   -> Ghc a - The action to perform.
>>   -> IO a - Run function for the Ghc monad.
>>
>> It initialises the GHC session and warnings via initGhcMonad.
>> Each call to this function will create a new session which should not be
>> shared among several threads.
>>
>> Any errors not handled inside the Ghc action are propagated as IO
>> exceptions.
>>
>> ---
>>
>> And if the session isn't guaranteed there's no guarantee about the
>> underlying state.
>> This explicit declaration that runGhc will not share state has been in
>> the API for for decades (going as far back as I stopped looking at 7.2).
>>
>> That Clash is relying on behavior we explicitly stated is not the case is
>> a bug in Clash.
>>
>> If you require shared state you should not be using the top level runGhc
>> wrapp

Re: What changed between GHC 8.8 and 8.10 that could cause this?

[Phyx]

Hi,

Hmm... I don't agree..

This isn't about grounds of truth or anything like that.. and in fact, an
object being in the linker map, doesn't mean its usable at all or meant to
be used at all.
It can be temporary state (symbol redirection or supporting of deprecated
symbols are two that come to mind).  So this is also a case of.. be careful.

The change introduced in the MR simply decoupled the top level user
interface and the C linker.
The reason for this is simply because the majority of projects do not
require shared state here, but infact benefit from unshared state.

i.e. interpreters, IDEs etc.  Where you want to be able to process multiple
separate files at the same time without needing to create new processes for
each.

Now back to your point about runGhc needing to use a shared state.. In my
opinion that would be wrong.

Here's the documentation for GHC 8.6.5
https://hackage.haskell.org/package/ghc-8.6.5/docs/GHC.html

specifically:



runGhc
  :: Maybe FilePath- See argument to initGhcMonad.
  -> Ghc a - The action to perform.
  -> IO a - Run function for the Ghc monad.

It initialises the GHC session and warnings via initGhcMonad.
Each call to this function will create a new session which should not be
shared among several threads.

Any errors not handled inside the Ghc action are propagated as IO
exceptions.

---

And if the session isn't guaranteed there's no guarantee about the
underlying state.
This explicit declaration that runGhc will not share state has been in the
API for for decades (going as far back as I stopped looking at 7.2).

That Clash is relying on behavior we explicitly stated is not the case is a
bug in Clash.

If you require shared state you should not be using the top level runGhc
wrapper but instead call unGhc yourself (or call setSession yourself).

There is perhaps a case to be made for a runGhcShared which does this, but
runGhc itself never guaranteed one session or one state.

Kind regards,
Tamar

On Tue, Mar 9, 2021, 10:27 Christiaan Baaij 
wrote:

> Even if MR388 ( https://gitlab.haskell.org/ghc/ghc/-/merge_requests/388 )
> is the cause of the issue we're seeing with the API exposed by Clash, I
> still think MR388 is wrong.
> My reasoning is the following:
>
> In 8.8 and earlier we had:
> - RTS C-code contains the ground truth of what is linked. The API it
> provides are set-membership, insert, lookup, and delete. Notably it does
> not allow you to get the set of linked objects.
> - There is a globally shared MVar (using NOINLINE, sharedCaf,
> unsafePerformIO newIORef "tricks") to what is basically a log/view of the
> linked-objects state kept by the RTS C-code.
>
> With MR388, in 8.10 and later we get:
> - RTS C-code contains the ground truth of what is linked. The API it
> provides are set-membership, insert, lookup, and delete. Notably it does
> not allow you to get the set of linked objects.
> - A _new_ MVar for every call to `runGhc` which is a log/view of the
> linked-object state kept by the RTS C-code. But that means these MVar get
> out-of-sync with the ground truth that is the RTS C-code! And since the RTS
> C-code does not expose an API to get the set of linked objects, there's no
> way to sync these MVars either!
>
> I'm building a ghc-8.10.2 with MR388 reverted to see whether it is indeed
> what is causing the issue we're seeing in Clash.
> Given my analysis above of what I think is wrong with MR388, I'm not
> saying we should completely revert MR388, but simply ensure that every
> HscEnv created through `runGhc` gets the globally shared MVar; as opposed
> to the current call to `newMVar`.
>
> On Sun, 7 Mar 2021 at 04:02, ÉRDI Gergő  wrote:
>
>> Thanks Matthew and Julian! Unfortunately, trying out GHC before/after
>> this
>> change didn't turn out to be as easy as I hoped: to do my testing, I
>> need to build a given GHC commit, and then use that via Stack to install
>> ~140 dependencies so that I can then test the problem I have initially
>> seen. And it turns out doing that with a random GHC commit is quite
>> painful because in any given Stackage snapshot there will be packages
>> with
>> which the GHC-bundled libraries are incompatible... :/
>>
>>
>>
>> On Thu, 4 Mar 2021, Julian Leviston wrote:
>>
>> > Hi,I don’t know enough about what Clash does to comment really, but it
>> sounds like
>> > it’s to do with my work on enabling multiple linker instances
>> > in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/388 — maybe
>> reading through
>> > that or the plan I outlined at
>> https://gitlab.haskell.org/ghc/ghc/-/issues/3372 might
>> > help, though I’m not sure.
>> >
>> > Strange, though, as this work was to isolate state in GHC — to change
>> it from using a
>> > global IORef to use a per-process MVar . But it definitely did change
>> the way state is
>> > handled, so it might be the related to these issues somehow?
>> >
>> > I realise this isn’t much help, but maybe it points you in a direction
>> where you can
>> > begin to understand 

Re: GHC 9.1?

[Phyx]

I am also against not using the odd/even versioning scheme.

My objections are similar to what Edward mentioned in that adding "junks"
at the end of the build number is problematic for packagers of the
toolchain where the packaging has its own way to mark something
pre-release.

If GHC were to invent its own thing, especially if it's alpha numeric this
would be a huge pain for no real benefit.

A beginner can quickly see on Wikipedia or other places that the compiler
only does even numbered releases, but the changes has a lot of wide
spreading implications.

Kind regards,
Tamar

Sent from my Mobile

On Tue, Mar 2, 2021, 09:34 Edward Kmett  wrote:

> In the past I've gained non-zero utility from having the spacer there to
> allow me to push patches in to allow HEAD builds while features are still
> in flux. Some of those in flux changes -- to my mild chagrin -- made it out
> to hackage, but were handled robustly because I wasn't claiming in the code
> that it worked on the next major release of GHC. Admittedly this was in the
> before-times, when it was much harder to vendor specific versions of
> packages for testing. Now with stack.yaml and cabal.project addressing that
> detail it is much reduced concern.
>
> That isn't to say there is zero cost to losing every other version number,
> but if we want to allow GHC versions and PVP versions to mentally "fit in
> the same type" the current practice has the benefit that it doesn't require
> us either doing something like bolting tags back into Data.Version to
> handle the "x.y.nightly" or forcing everyone to move to the real next
> release the moment the new compiler ships with a bunch of a jump, or
> generally forcing more string-processing nonsense into build systems. Right
> now version numbers go up and you can use some numerical shenanigans to
> approximate them with a single integer for easy ifdefs.
>
> I'm ever so slightly against recoloring the bikeshed on the way we manage
> the GHC  version number, just because I know my tooling is robust around
> what we have, and I don't see marked improvement in the status quo being
> gained, while I do foresee a bit of complication around the consumption of
> ghc as a tool if we change
>
> -Edward
>
> On Mon, Mar 1, 2021 at 8:30 PM Richard Eisenberg  wrote:
>
>> Hi devs,
>>
>> I understand that GHC uses the same version numbering system as the Linux
>> kernel did until 2003(*), using odd numbers for unstable "releases" and
>> even ones for stable ones. I have seen this become a point of confusion, as
>> in: "Quick Look just missed the cutoff for GHC 9.0, so it will be out in
>> GHC 9.2" "Um, what about 9.1?"
>>
>> Is there a reason to keep this practice? Linux moved away from it 18
>> years ago and seems to have thrived despite. Giving this convention up on a
>> new first-number change (the change from 8 to 9) seems like a good time.
>>
>> I don't feel strongly about this, at all -- just asking a question that
>> maybe no one has asked in a long time.
>>
>> Richard
>>
>> (*) I actually didn't know that Linux stopped doing this until writing
>> this email, wondering why we needed to tie ourselves to Linux. I
>> coincidentally stopped using Linux full-time (and thus administering my own
>> installation) in 2003, when I graduated from university.
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Stop holding hadrian back with backwards compatibility

[Phyx]

Hi, Just leaving my two cents feel free to ignore..

> I almost suggested that this had to be the reason for the back-compat
design

You're right, but not for backwards compat of Hadrian vs Make, but for
compat with RTS versions.
I could be wrong, but my understanding is the current design in Make is
just an artifact of getting something that works on all OSes without much
pain, but has proven to be suboptimal in a very important use case (slight
detour time):

You have to make a choice of which RTS to use at compile time.  Which is
quite bad.  Because it means that you can't swap between two RTS flavors
with the same ABI. It also means building presents a problem, you want your
compiler at the end of stage1 to use your new rts, not the one of the
stage0 compiler.

You can't have multiple versions of the RTS in one library, but if you have
the full name as a dependency the dynamic loader happily loads you multiple
copies.

To solve this issue the design was made to not declare the RTS as a
dependency on any haskell library. i.e. there's not DT_NEEDED entry for it
on ELF operating systems.  Which means before you load a Haskell produced
dynamic library on Linux you need to LD_PRELOAD an rts. It's clunky, but it
works, it allows you to switch between debug and non-debug rts at
initialization time.

On Windows, this problem was punted, because everything is statically
linked.  But the problem exists that you can have multiple DLLs with
different RTS and ABIs.  This is fine as long as the DLLs have no
dependencies on each other. Once they do... you have a big problem.  This
is one of the primary blockers of shared library support on Windows.

I.. don't know whatever wacky solution MacOS uses so can't comment there.

Now back to the original question about version 1.0, this has nothing to do
with Make at all. Make based system only implemented the scheme that was
wanted. It's not like any Make system design issues forced this scheme. Now
over the years, assumptions that the RTS is always version 1.0 could have
krept into the build system.  But I don't believe this to have been design,
just convenience. Right now, the design only requires you to know the GHC
version, which is available in all makefiles.  Knowing the RTS version
would be difficult, but the point is that in a proper design you don't need
to know the version.

Almost half a decade ago a plan was made to replace this scheme with one
that would work on all OSes and would allow us to solve these issues. The
design was made and debated here
https://gitlab.haskell.org/ghc/ghc/-/issues/10352

The actual solution isn't as simple as just adding the rts version to the
library name or add it only to the build system, in fact this would be the
wrong approach as it makes it impossible to observe backwards compatibility
between GHC releases.
i.e. without it, you'd need to have GHC 9.0.1 installed to run GHC 9.0.1
programs, you can't run using GHC 9.2.x rts if the version changed.

Typically ELF based platforms solve this by a combination of SONAME and
symbol versioning.  Windows solves this by a combination of SxS Assembly
versioning or mingw style SONAME.

All of which require you to have the same filename for the libraries, but
use a different path to disambiguate:

lib/ghc-${ver}/rts-1.0/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/thr/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/debug/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/l/libHSrts-ghc${ver}.so

lib/ghc-${ver}/rts-1.0/thr_l/libHSrts-ghc${ver}.so

for each RTS with the same ABI. profiling libs for instance have a
different ABI and can't use this scheme.

So what has taken so long to implement this? Well.. time. As it turns out,
getting this scheme to work required a lot of foundational work in GHC
(Particularly on Windows where dynamic linking design wasn't optimal, but
both GHC and the dynamic linker are happy now).

On Linux it took a while to get SONAME support in cabal
https://github.com/haskell/cabal/issues/4052 so we don't have to hack
around in the build system.

But anyway this is why the current scheme exists, and why just adding an
rts version isn't really sufficient, especially if the name propagates to
the shared lib.

TL;DR;

If we are going to change the build system, we should do it properly.

The current scheme exists because GHC does not observe any mechanism to
support multiple runtimes with the same ABI and does not really have a
backwards compatibility story.

Kind Regards,

Tamar

On Wed, Feb 10, 2021 at 11:00 PM Richard Eisenberg  wrote:

>
>
> On Feb 10, 2021, at 8:50 AM, Simon Peyton Jones 
> wrote:
>
> build with hadrian, and then continue using make with the artifacts
> (partially) built by Hadrian
>
>
> I almost suggested that this had to be the reason for the back-compat
> design, but I assumed I had to be wrong. I also agree this is a non-goal;
> I'm quite happy to be forced to pick one or the other and stick with that
> choice until blasting away all build products.

Re: Has ghc-9.0 for windows changed to require installation?

[Phyx]

No, there's no change at all in the portability.
This looks like a fallout from switching from Make to Hadrian.

ghcii.sh was created as an artifact of make install. Hadrian seems to lack
this step.

Note that this script is nothing magical, it's just a hack around how
signal handlers in Native windows and Cygwin processes are handled.

I'm not entirely sure it's still needed with the new I/O manager, however
that's not on yet by default..

Anyways support should probably be added to Hadrian.

Kind regards,
Tamar

Sent from my Mobile

On Mon, Feb 8, 2021, 14:43 Moritz Angermann 
wrote:

> Thanks for flagging this. This would be the opposite direction of what
> I’ve been advocating for. That we get bindists for Linux and macOS that
> work by simply unpacking them.
>
> On Mon, 8 Feb 2021 at 10:05 PM, Takenobu Tani 
> wrote:
>
>> Hi devs,
>>
>> The ghc-binary for windows needs to `make install` since ghc-9.0 [1].
>> Is this an intended change?
>>
>> Previously, ghc-8.10.4 binary for windows [2] doesn't need to `make
>> install`.
>> We only expand the tar-file and then we can execute `bin/ghcii.sh`.
>>
>> [1]:
>> https://downloads.haskell.org/ghc/9.0.1/ghc-9.0.1-x86_64-unknown-mingw32.tar.xz
>> [2]:
>> https://downloads.haskell.org/ghc/8.10.4/ghc-8.10.4-x86_64-unknown-mingw32.tar.xz
>>
>> Regards,
>> Takenobu
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Debugging annotation machinery

[Phyx]

Hi,

I am changing the hashing algorithm used to create Fingerprints but somehow
the annrun01.hs test has started failing and I don't quite understand why.

>From what I can tell the failure happens during deserialization but the
module itself is still found.
The calls findGlobalAnns deserializeWithData target seem to return nothing.

Calling ~/ghc/inplace/bin/ghc-stage2.exe --show-iface Annrun01_Help.hi
shows that GHC itself understands the module fine though.

Does someone have an idea why deserialization might be failing? but also
how does one debug this? Almost none of the structures have a Show or
Outputable instance...

Thanks,
Tamar
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] Glasgow Haskell Compiler 8.10.3 released

[Phyx]

Hi Ben,

The package ghc-8.10.3-x86_64-unknown-mingw32-integer-simple.tar.xz reports

Tamar@PowPow /t/ghc> ./ghc-8.10.3/bin/ghc.exe -v
Glasgow Haskell Compiler, Version 8.10.3, stage 2 booted by GHC version
8.8.3
...
wired-in package integer-wired-in mapped to integer-gmp-1.0.3.0
...

Tamar@PowPow /t/ghc> ./ghc-8.10.3/bin/ghc-pkg list | grep integer
integer-gmp-1.0.3.0

Is this package correct? The naming seems different from the one in the
9.0.1-alpha as well (that one had an explicit -integer-simple in the folder
name when uncompressed)

Thanks,
Tamar


On Sun, Dec 20, 2020 at 12:23 AM Ben Gamari  wrote:

> Hello all,
>
> The GHC team is happy to announce the release of GHC 8.10.3. Source
> and binary distributions are available at the usual place:
>
> https://downloads.haskell.org/ghc/8.10.3/
>
> GHC 8.10.3 fixes a number of issues in present in GHC 8.10.2 including:
>
>  * Numerous stability improves on Windows
>
>  * More robust support for architectures with weak memory ordering
>guarantees (e.g. modern ARM hardware).
>
>  * GHC can now split dynamic objects to accomodate macOS' RPATH size
>limitation when building large projects (#1)
>
>  * Several correctness bugs in the new low-latency garbage collector
>
>  * Many, many other bug-fixes
>
> Note that at the moment we still require that macOS Catalina users
> exempt the binary distribution from the notarization requirement by
> running `xattr -cr .` on the unpacked tree before running `make install`.
> This situation will hopefully be improved for GHC 9.0.1 with the
> resolution of #17418 [1].
>
> Cheers,
>
> - Ben
>
> [1] https://gitlab.haskell.org/ghc/ghc/issues/17418
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: msys woes

[Phyx]

> I'm not sure what you mean by "the tag won't move". Could you clarify?

I mean that the tag called 8.8.4-release will still point to the a broken
hash no? Since it represents the hash of the commit the release was made
from.

Sent from my Mobile

On Thu, Oct 8, 2020, 21:28 Ben Gamari  wrote:

> Phyx  writes:
>
> > Right,
> >
> > I think seems to be a bug in the script that wasn't noticed before.
> >
> > On line 40 there are () missing around the condition. So it's checking
> both
> > URLs i think.
> >
> > The download to Haskell.org succeeds but the msys2 one fails since that
> > package is gone.
> >
> > The whole thing after the || needs to be in ().
> >
> > This would need to be fixed in that branch but the tag won't move.
> >
> > Any ideas here Ben?
> >
> I'm not sure what you mean by "the tag won't move". Could you clarify?
>
> Regardless, I do see that some parens are needed here. I have opened
> #18821 to track this.
>
> For what it's worth, the download script was revamped in GHC 8.10 so I
> don't think there is anything needed on master or ghc-8.10.
>
> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: msys woes

[Phyx]

>From the looks of it the repo.msys2.org system doesn't have enough space to
restore all these old packages.

So we'll have to handle it on our end. The easiest way to get your builds
working again is by applying a patch with the fix to the source tree.

Sent from my Mobile

On Thu, Oct 8, 2020, 17:55 Phyx  wrote:

> Right,
>
> I think seems to be a bug in the script that wasn't noticed before.
>
> On line 40 there are () missing around the condition. So it's checking
> both URLs i think.
>
> The download to Haskell.org succeeds but the msys2 one fails since that
> package is gone.
>
> The whole thing after the || needs to be in ().
>
> This would need to be fixed in that branch but the tag won't move.
>
> Any ideas here Ben?
>
> Sent from my Mobile
>
> On Thu, Oct 8, 2020, 17:42 Shayne Fletcher 
> wrote:
>
>>
>>
>> On Thu, Oct 8, 2020 at 12:19 PM Shayne Fletcher <
>> shayne.fletcher...@gmail.com> wrote:
>>
>>>
>>>
>>> On Thu, Oct 8, 2020 at 12:13 PM Phyx  wrote:
>>>
>>>> Hi Shayne,
>>>>
>>>> 8.8.1 had a different mechanism but the primary is still haskell.org.
>>>>
>>>>
>>> I think we are looking at ghc-8.4.3 here (being used on 8.8.1 sources).
>>>
>>>
>>>> You don't need to run the entire pipeline to test this. Just run
>>>>
>>>> ./mk/get-win32-tarballs.sh download x86_64
>>>>
>>>>
>>> I've thrown that into the build script. Stand-by. Let's see what we get.
>>>
>>>
>> Here we go!
>>
>> ```
>>
>> 2020-10-08T16:35:14.8128041Z Downloading 
>> mingw-w64-x86_64-crt-git-5.0.0.4795.e3d96cb1-1-any.pkg.tar.xz to 
>> ghc-tarballs/mingw-w64/x86_64...
>> 2020-10-08T16:35:15.3839177Z #=#=#
>> 2020-10-08T16:35:15.3841316Z
>> 2020-10-08T16:35:15.4836014Z 
>>   22.6%
>> 2020-10-08T16:35:15.4918420Z 
>> ##
>> 92.9%
>> 2020-10-08T16:35:15.4921362Z 
>>  
>> 100.0%
>> 2020-10-08T16:35:15.7741545Z #=#=#
>> 2020-10-08T16:35:15.8742272Z ##O#- #
>> 2020-10-08T16:35:15.9760569Z ##O=#  #
>> 2020-10-08T16:35:16.9768720Z #=#=-#  #
>> 2020-10-08T16:35:17.4160288Z -#O#- #   #
>> 2020-10-08T16:35:17.4162806Z -=#=#   #   #
>> 2020-10-08T16:35:17.4164112Z curl: (22) The requested URL returned error: 
>> 404 Not Found
>> 2020-10-08T16:35:17.4407190Z
>> 2020-10-08T16:35:17.4425090Z ERROR: Download failed.
>> 2020-10-08T16:35:17.4564686Z ghc-lib-gen.EXE: Failed when running system 
>> command: stack --stack-yaml hadrian/stack.yaml exec -- bash -c 
>> "./mk/get-win32-tarballs.sh download x86_64"
>> ```
>>
>> Hope this helps!
>>
>> --
>> Shayne Fletcher
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: msys woes

[Phyx]

Right,

I think seems to be a bug in the script that wasn't noticed before.

On line 40 there are () missing around the condition. So it's checking both
URLs i think.

The download to Haskell.org succeeds but the msys2 one fails since that
package is gone.

The whole thing after the || needs to be in ().

This would need to be fixed in that branch but the tag won't move.

Any ideas here Ben?

Sent from my Mobile

On Thu, Oct 8, 2020, 17:42 Shayne Fletcher 
wrote:

>
>
> On Thu, Oct 8, 2020 at 12:19 PM Shayne Fletcher <
> shayne.fletcher...@gmail.com> wrote:
>
>>
>>
>> On Thu, Oct 8, 2020 at 12:13 PM Phyx  wrote:
>>
>>> Hi Shayne,
>>>
>>> 8.8.1 had a different mechanism but the primary is still haskell.org.
>>>
>>>
>> I think we are looking at ghc-8.4.3 here (being used on 8.8.1 sources).
>>
>>
>>> You don't need to run the entire pipeline to test this. Just run
>>>
>>> ./mk/get-win32-tarballs.sh download x86_64
>>>
>>>
>> I've thrown that into the build script. Stand-by. Let's see what we get.
>>
>>
> Here we go!
>
> ```
>
> 2020-10-08T16:35:14.8128041Z Downloading 
> mingw-w64-x86_64-crt-git-5.0.0.4795.e3d96cb1-1-any.pkg.tar.xz to 
> ghc-tarballs/mingw-w64/x86_64...
> 2020-10-08T16:35:15.3839177Z #=#=#
> 2020-10-08T16:35:15.3841316Z
> 2020-10-08T16:35:15.4836014Z  
>  22.6%
> 2020-10-08T16:35:15.4918420Z 
> ##
> 92.9%
> 2020-10-08T16:35:15.4921362Z 
>  
> 100.0%
> 2020-10-08T16:35:15.7741545Z #=#=#
> 2020-10-08T16:35:15.8742272Z ##O#- #
> 2020-10-08T16:35:15.9760569Z ##O=#  #
> 2020-10-08T16:35:16.9768720Z #=#=-#  #
> 2020-10-08T16:35:17.4160288Z -#O#- #   #
> 2020-10-08T16:35:17.4162806Z -=#=#   #   #
> 2020-10-08T16:35:17.4164112Z curl: (22) The requested URL returned error: 404 
> Not Found
> 2020-10-08T16:35:17.4407190Z
> 2020-10-08T16:35:17.4425090Z ERROR: Download failed.
> 2020-10-08T16:35:17.4564686Z ghc-lib-gen.EXE: Failed when running system 
> command: stack --stack-yaml hadrian/stack.yaml exec -- bash -c 
> "./mk/get-win32-tarballs.sh download x86_64"
> ```
>
> Hope this helps!
>
> --
> Shayne Fletcher
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: msys woes

[Phyx]

Hi Shayne,

8.8.1 had a different mechanism but the primary is still haskell.org.

You don't need to run the entire pipeline to test this. Just run

./mk/get-win32-tarballs.sh download x86_64

This should echo to stdout more information.

Kind regards,
Tamar

Sent from my Mobile

On Thu, Oct 8, 2020, 15:54 Shayne Fletcher 
wrote:

>
>
> On Thu, Oct 8, 2020 at 10:51 AM Shayne Fletcher <
> shayne.fletcher...@gmail.com> wrote:
>
>>
>> Hi Phyx,
>>
>> On Thu, Oct 8, 2020 at 9:10 AM Phyx  wrote:
>>
>>> > `./configure --enable-tarballs-autodownload` GHC build step on Windows
>>> has been failing because repo.msys2.org
>>>
>>> Afaik GHC doesn't rely ok repo.msys2.org for builds, only for
>>> mirroring. The primary url is haskell.org
>>> https://downloads.haskell.org/ghc/mingw/
>>>
>>> So it's down time shouldn't have affected you (and works for me).
>>>
>>>
>> [...]
>>
>> A procedure to reproduce it would be,
>> ```
>> cd ghc
>> git fetch --tags && git checkout ghc-8.8.1-release
>> git submodule update --init --recursive
>> stack --stack-yaml hadrian/stack.yaml exec -- bash -c "./configure
>> --enable-tarballs-autodownload"
>> ```
>>
>
> Oops, obviously, there should be a boot in there first.
> ```
> cd ghc
> git fetch --tags && git checkout ghc-8.8.1-release
> git submodule update --init --recursive
> stack --stack-yaml hadrian/stack.yaml exec -- bash -c ./boot
> stack --stack-yaml hadrian/stack.yaml exec -- bash -c "./configure
> --enable-tarballs-autodownload"
> ```
>
> --
> Shayne Fletcher
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: msys woes

[Phyx]

> `./configure --enable-tarballs-autodownload` GHC build step on Windows
has been failing because repo.msys2.org

Afaik GHC doesn't rely ok repo.msys2.org for builds, only for mirroring.
The primary url is haskell.org https://downloads.haskell.org/ghc/mingw/

So it's down time shouldn't have affected you (and works for me).

Which url does it say is inaccessible?

Sent from my Mobile

On Thu, Oct 8, 2020, 13:50 Shayne Fletcher 
wrote:

> For the last 5 days, the `./configure --enable-tarballs-autodownload` GHC
> build step on Windows has been failing because repo.msys2.org has been
> unavailable (https://github.com/msys2/MSYS2-packages/issues/2171).
> Apparently that's fixed today and the server can now be reached but still
> the configure fails now with 'curl: (22) The requested URL returned
> error: 404 Not Found'  It would be great if someone with an understanding
> of what precisely `--enable-tarballs-autodownload` translates to was
> to file a bug-report with the right folks!
>
> --
> Shayne Fletcher
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: New Windows I/O manager in GHC 8.12

[Phyx]

Thanks Simon, cheers :)

Sent from my Mobile

On Mon, Jul 20, 2020, 15:28 Simon Peyton Jones 
wrote:

> Tamar, I salute you!  This is a big piece of work – thank you!
>
>
> Simon
>
>
>
> *From:* ghc-devs  *On Behalf Of *Phyx
> *Sent:* 17 July 2020 16:04
> *To:* ghc-devs@haskell.org Devs 
> *Subject:* New Windows I/O manager in GHC 8.12
>
>
>
> Hi All,
>
> In case you've missed it, about 150 or so commits were committed to master
> yesterday.  These commits add WinIO (Windows I/O) to GHC.  This is a new
> I/O
> manager that is designed for the native Windows I/O subsystem instead of
> relying on the broken posix-ish compatibility layer that MIO used.
>
> This is one of 3 big patches I have been working on for years now..
>
> So before I continue on why WinIO was made I'll add a TL;DR;
>
> WinIO adds an internal API break compared to previous GHC releases.  That
> is
> the internal code was modified to support a completely asynchronous I/O
> system.
>
> What this means is that we have to keep track of the file pointer offset
> which
> previously was done by the C runtime.  This is because in async I/O you
> cannot
> assume the offset to be at any given location.
>
> What does this mean for you? Very little. If you did not use internal GHC
> I/O code.
> In particular if you haven't used Buffer, BufferIO and RawIO. If you have
> you will
> to explicitly add support for GHC 8.12+.
>
> Because FDs are a Unix concept and don't behave as you would expect on
> Windows, the
> new I/O manager also uses HANDLE instead of FD. This means that any
> library that has
> used the internal GHC Fd type won't work with WinIO. Luckily the number of
> libraries
> that have seems quite low. If you can please stick to the external Handle
> interface
> for I/O functions.
>
> The boot libraries have been updated, and in particular process *requires*
> the version
> that is shipped with GHC.  Please respect the version bounds here!  I will
> be writing
> a migration guide for those that need to migrate code.  The amount of work
> is usually
> trivial as Base provides shims to do most of the common things you would
> have used Fd for.
>
> Also if I may make a plea to GHC developers.. Do not add non-trivial
> implementations
> in the external exposed modules (e.g. System.xxx, Data.xxx) but rather add
> them to internal
> modules (GHC.xxx) and re-export them from the external modules.  This
> allows us to avoid
> import cycles inside the internal modules :)
>
> --
>
> So why WinIO? Over the years a number of hard to fix issues popped up on
> Windows, including
> proper Unicode console I/O, cooked inputs, ability to cancel I/O requests.
> This also allows libraries like Brick to work on Windows without
> re-inventing the wheel or have to hide their I/O from the I/O manager.
>
> In order to attempt to do some of these with MIO layer upon layers of
> hacks were added.  This means that things sometimes worked.., but when it
> didn't was rather unpredictable.  Some of the issues were simply unfixable
> with MIO.  I will be making some posts about how WinIO works (and also
> archiving them on the wiki don't worry :)) but for now some highlights:
>
> WinIO is 3 years of work, First started by Joey Hess, then picked up by
> Mikhail Glushenkov before landing at my feet.  While the majority has been
> rewritten their work did provide a great jumping off point so thanks!  Also
> thanks to Ben and AndreasK for helping me get it over the line.. As you can
> imagine I was exhausted by this point :).
>
> Some stats: ~8000 new lines and ~1100 removed ones spread over 130+
> commits (sorry this was the smallest we could get it while not losing some
> historical context) and with over 153 files changed not counting the
> changes to boot libraries.
>
> It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359,
> #12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797,
> #5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189
> (but only on native windows consoles, so no msys shells) and #806 which is
> 14 years old!
>
> WinIO is a dynamic choice, so you can switch between I/O managers using
> the RTS flag --io-manager=[native|posix].
>
> On non-Windows native is the same as posix.
>
> The chosen Async interface for this implementation is using Completion
> Ports.
>
> The I/O manager uses a new interface added in Windows Vista called
> GetQueuedCompletionStatusEx which allows us to service multiple
> request interrupts in one go.
>
> Some highlights:
>
> * Drops Windows Vista support
>   Vista is out of extended support as of 2017. The new minimum is Windows
> 7.  This 

Re: New Windows I/O manager in GHC 8.12

[Phyx]

Hi Mikhail,

Thanks! And thanks for the jump start :)

Cheers,
Tamar

Sent from my Mobile

On Sun, Jul 19, 2020, 09:05 Mikhail Glushenkov 
wrote:

> Hi Tamar,
>
> Congratulations on getting WinIO merged, this is a really impressive
> effort.
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: New Windows I/O manager in GHC 8.12

[Phyx]

Oops just a correction here, should be Joseph Adams instead of Joey Hess in
the attribution.

Regards,
Tamar

Sent from my Mobile

On Fri, Jul 17, 2020, 16:03 Phyx  wrote:

> Hi All,
>
> In case you've missed it, about 150 or so commits were committed to master
> yesterday.  These commits add WinIO (Windows I/O) to GHC.  This is a new
> I/O
> manager that is designed for the native Windows I/O subsystem instead of
> relying on the broken posix-ish compatibility layer that MIO used.
>
> This is one of 3 big patches I have been working on for years now..
>
> So before I continue on why WinIO was made I'll add a TL;DR;
>
> WinIO adds an internal API break compared to previous GHC releases.  That
> is
> the internal code was modified to support a completely asynchronous I/O
> system.
>
> What this means is that we have to keep track of the file pointer offset
> which
> previously was done by the C runtime.  This is because in async I/O you
> cannot
> assume the offset to be at any given location.
>
> What does this mean for you? Very little. If you did not use internal GHC
> I/O code.
> In particular if you haven't used Buffer, BufferIO and RawIO. If you have
> you will
> to explicitly add support for GHC 8.12+.
>
> Because FDs are a Unix concept and don't behave as you would expect on
> Windows, the
> new I/O manager also uses HANDLE instead of FD. This means that any
> library that has
> used the internal GHC Fd type won't work with WinIO. Luckily the number of
> libraries
> that have seems quite low. If you can please stick to the external Handle
> interface
> for I/O functions.
>
> The boot libraries have been updated, and in particular process *requires*
> the version
> that is shipped with GHC.  Please respect the version bounds here!  I will
> be writing
> a migration guide for those that need to migrate code.  The amount of work
> is usually
> trivial as Base provides shims to do most of the common things you would
> have used Fd for.
>
> Also if I may make a plea to GHC developers.. Do not add non-trivial
> implementations
> in the external exposed modules (e.g. System.xxx, Data.xxx) but rather add
> them to internal
> modules (GHC.xxx) and re-export them from the external modules.  This
> allows us to avoid
> import cycles inside the internal modules :)
>
> --
>
> So why WinIO? Over the years a number of hard to fix issues popped up on
> Windows, including
> proper Unicode console I/O, cooked inputs, ability to cancel I/O requests.
> This also allows libraries like Brick to work on Windows without
> re-inventing the wheel or have to hide their I/O from the I/O manager.
>
> In order to attempt to do some of these with MIO layer upon layers of
> hacks were added.  This means that things sometimes worked.., but when it
> didn't was rather unpredictable.  Some of the issues were simply unfixable
> with MIO.  I will be making some posts about how WinIO works (and also
> archiving them on the wiki don't worry :)) but for now some highlights:
>
> WinIO is 3 years of work, First started by Joey Hess, then picked up by
> Mikhail Glushenkov before landing at my feet.  While the majority has been
> rewritten their work did provide a great jumping off point so thanks!  Also
> thanks to Ben and AndreasK for helping me get it over the line.. As you can
> imagine I was exhausted by this point :).
>
> Some stats: ~8000 new lines and ~1100 removed ones spread over 130+
> commits (sorry this was the smallest we could get it while not losing some
> historical context) and with over 153 files changed not counting the
> changes to boot libraries.
>
> It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359,
> #12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797,
> #5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189
> (but only on native windows consoles, so no msys shells) and #806 which is
> 14 years old!
>
> WinIO is a dynamic choice, so you can switch between I/O managers using
> the RTS flag --io-manager=[native|posix].
>
> On non-Windows native is the same as posix.
>
> The chosen Async interface for this implementation is using Completion
> Ports.
>
> The I/O manager uses a new interface added in Windows Vista called
> GetQueuedCompletionStatusEx which allows us to service multiple
> request interrupts in one go.
>
> Some highlights:
>
> * Drops Windows Vista support
>   Vista is out of extended support as of 2017. The new minimum is Windows
> 7.  This allows us to use much more efficient OS provided abstractions.
>
> * Replace Events and Monitor locks with much faster and efficient
> Conditional Variables and SlimReaderWriterLocks.
> * Change GHC's B

Re: New Windows I/O manager in GHC 8.12

[Phyx]

Hi Travis,

Thanks for the kind words :) Hopefully once network support gets done and
it's the default that all the work pays off :)

Cheers,
Tamar

Sent from my Mobile

On Sat, Jul 18, 2020, 06:38 Travis Whitaker  wrote:

> Hello Tamar,
>
> Just wanted to send a quick note expressing my gratitude for these
> improvements! My first professional work in Haskell targeted Windows
> platforms, and I'm sure there are many others out there who appreciate
> the tireless work you've done to keep GHC's Windows support chugging
> along. I've been kicking the tires on your WinIO branch for a few
> months and this is absolutely a game-changer for Windows support.
>
> Thanks!
>
> Travis
>
> On Fri, Jul 17, 2020 at 8:06 AM Phyx  wrote:
> >
> > Hi All,
> >
> > In case you've missed it, about 150 or so commits were committed to
> master
> > yesterday.  These commits add WinIO (Windows I/O) to GHC.  This is a new
> I/O
> > manager that is designed for the native Windows I/O subsystem instead of
> > relying on the broken posix-ish compatibility layer that MIO used.
> >
> > This is one of 3 big patches I have been working on for years now..
> >
> > So before I continue on why WinIO was made I'll add a TL;DR;
> >
> > WinIO adds an internal API break compared to previous GHC releases.
> That is
> > the internal code was modified to support a completely asynchronous I/O
> system.
> >
> > What this means is that we have to keep track of the file pointer offset
> which
> > previously was done by the C runtime.  This is because in async I/O you
> cannot
> > assume the offset to be at any given location.
> >
> > What does this mean for you? Very little. If you did not use internal
> GHC I/O code.
> > In particular if you haven't used Buffer, BufferIO and RawIO. If you
> have you will
> > to explicitly add support for GHC 8.12+.
> >
> > Because FDs are a Unix concept and don't behave as you would expect on
> Windows, the
> > new I/O manager also uses HANDLE instead of FD. This means that any
> library that has
> > used the internal GHC Fd type won't work with WinIO. Luckily the number
> of libraries
> > that have seems quite low. If you can please stick to the external
> Handle interface
> > for I/O functions.
> >
> > The boot libraries have been updated, and in particular process
> *requires* the version
> > that is shipped with GHC.  Please respect the version bounds here!  I
> will be writing
> > a migration guide for those that need to migrate code.  The amount of
> work is usually
> > trivial as Base provides shims to do most of the common things you would
> have used Fd for.
> >
> > Also if I may make a plea to GHC developers.. Do not add non-trivial
> implementations
> > in the external exposed modules (e.g. System.xxx, Data.xxx) but rather
> add them to internal
> > modules (GHC.xxx) and re-export them from the external modules.  This
> allows us to avoid
> > import cycles inside the internal modules :)
> >
> > --
> >
> > So why WinIO? Over the years a number of hard to fix issues popped up on
> Windows, including
> > proper Unicode console I/O, cooked inputs, ability to cancel I/O
> requests. This also allows libraries like Brick to work on Windows without
> re-inventing the wheel or have to hide their I/O from the I/O manager.
> >
> > In order to attempt to do some of these with MIO layer upon layers of
> hacks were added.  This means that things sometimes worked.., but when it
> didn't was rather unpredictable.  Some of the issues were simply unfixable
> with MIO.  I will be making some posts about how WinIO works (and also
> archiving them on the wiki don't worry :)) but for now some highlights:
> >
> > WinIO is 3 years of work, First started by Joey Hess, then picked up by
> Mikhail Glushenkov before landing at my feet.  While the majority has been
> rewritten their work did provide a great jumping off point so thanks!  Also
> thanks to Ben and AndreasK for helping me get it over the line.. As you can
> imagine I was exhausted by this point :).
> >
> > Some stats: ~8000 new lines and ~1100 removed ones spread over 130+
> commits (sorry this was the smallest we could get it while not losing some
> historical context) and with over 153 files changed not counting the
> changes to boot libraries.
> >
> > It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359,
> #12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797,
> #5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189
> > (but only on native windows consoles, so no msys shells) 

New Windows I/O manager in GHC 8.12

[Phyx]

Hi All,

In case you've missed it, about 150 or so commits were committed to master
yesterday.  These commits add WinIO (Windows I/O) to GHC.  This is a new I/O
manager that is designed for the native Windows I/O subsystem instead of
relying on the broken posix-ish compatibility layer that MIO used.

This is one of 3 big patches I have been working on for years now..

So before I continue on why WinIO was made I'll add a TL;DR;

WinIO adds an internal API break compared to previous GHC releases.  That is
the internal code was modified to support a completely asynchronous I/O
system.

What this means is that we have to keep track of the file pointer offset
which
previously was done by the C runtime.  This is because in async I/O you
cannot
assume the offset to be at any given location.

What does this mean for you? Very little. If you did not use internal GHC
I/O code.
In particular if you haven't used Buffer, BufferIO and RawIO. If you have
you will
to explicitly add support for GHC 8.12+.

Because FDs are a Unix concept and don't behave as you would expect on
Windows, the
new I/O manager also uses HANDLE instead of FD. This means that any library
that has
used the internal GHC Fd type won't work with WinIO. Luckily the number of
libraries
that have seems quite low. If you can please stick to the external Handle
interface
for I/O functions.

The boot libraries have been updated, and in particular process *requires*
the version
that is shipped with GHC.  Please respect the version bounds here!  I will
be writing
a migration guide for those that need to migrate code.  The amount of work
is usually
trivial as Base provides shims to do most of the common things you would
have used Fd for.

Also if I may make a plea to GHC developers.. Do not add non-trivial
implementations
in the external exposed modules (e.g. System.xxx, Data.xxx) but rather add
them to internal
modules (GHC.xxx) and re-export them from the external modules.  This
allows us to avoid
import cycles inside the internal modules :)

--

So why WinIO? Over the years a number of hard to fix issues popped up on
Windows, including
proper Unicode console I/O, cooked inputs, ability to cancel I/O requests.
This also allows libraries like Brick to work on Windows without
re-inventing the wheel or have to hide their I/O from the I/O manager.

In order to attempt to do some of these with MIO layer upon layers of hacks
were added.  This means that things sometimes worked.., but when it didn't
was rather unpredictable.  Some of the issues were simply unfixable with
MIO.  I will be making some posts about how WinIO works (and also archiving
them on the wiki don't worry :)) but for now some highlights:

WinIO is 3 years of work, First started by Joey Hess, then picked up by
Mikhail Glushenkov before landing at my feet.  While the majority has been
rewritten their work did provide a great jumping off point so thanks!  Also
thanks to Ben and AndreasK for helping me get it over the line.. As you can
imagine I was exhausted by this point :).

Some stats: ~8000 new lines and ~1100 removed ones spread over 130+ commits
(sorry this was the smallest we could get it while not losing some
historical context) and with over 153 files changed not counting the
changes to boot libraries.

It Fixes #18307, #17035, #16917, #15366, #14530, #13516, #13396, #13359,
#12873, #12869, #11394, #10542, #10484, #10477, #9940, #7593, #7353, #5797,
#5305, #4471, #3937, #3081, #12117, #2408, #10956, #2189
(but only on native windows consoles, so no msys shells) and #806 which is
14 years old!

WinIO is a dynamic choice, so you can switch between I/O managers using the
RTS flag --io-manager=[native|posix].

On non-Windows native is the same as posix.

The chosen Async interface for this implementation is using Completion
Ports.

The I/O manager uses a new interface added in Windows Vista called
GetQueuedCompletionStatusEx which allows us to service multiple
request interrupts in one go.

Some highlights:

* Drops Windows Vista support
  Vista is out of extended support as of 2017. The new minimum is Windows
7.  This allows us to use much more efficient OS provided abstractions.

* Replace Events and Monitor locks with much faster and efficient
Conditional Variables and SlimReaderWriterLocks.
* Change GHC's Buffer and I/O structs to support asynchronous operation by
not relying on the OS managing File Offset.
* Implement a new command line flag +RTS --io-manager=[native|posix] to
control which I/O manager is used.
* Implement a new Console I/O interface supporting much faster reads/writes
and unicode output correctly.  Also supports things like cooked input etc.
* In new I/O manager if the user still has their code-page set to OEM, then
we use UTF-8 by default. This allows Unicode to work correctly out of the
box.
* Add Atomic Exchange PrimOp and implement Atomic Ptr exchanges.
* Flush event logs eagerly as to not rely on finalizers running.
* A lot of refactoring and more use of hsc2hs to 

Re: HEAD doesn't build. Totally stalled.

[Phyx]

But, where do you actually check for __SSP__

The guard just checks for not windows and not dynamic
https://github.com/ghc/ghc/commit/686e72253aed3880268dd6858eadd8c320f09e97#diff-03f5bc5a50fd8ae13e902782c4392c38R1157

shouldn't it just be checking for defined(__SSP__) instead? This check is
currently only correct if the distro has turned stack protector on by
default.


Regards,
Tamar

On Thu, Jul 16, 2020 at 3:46 PM Moritz Angermann 
wrote:

> I’ve tried to reproduce this and it turns out, I fail to. You are somehow
> building the rts either with _FORTYFY_SOURCE or __SSP__, but then your
> linker ends up not passing -lssp or the equivalent for your tool chain.
>
> At this point I’m tempted to add an additional ARM arch guard. While that
> would be conceptually wrong, it would reduce the cases where this could go
> wrong to a rarely used platform. Maybe @Ben Gamari has an idea?
>
> On Thu, 16 Jul 2020 at 10:25 PM, Simon Peyton Jones 
> wrote:
>
>> Moritz
>>
>> How’s it going getting this patch committed?
>>
>> It’s painful manually applying a fix, but then NOT committing that to
>> master by mistake
>>
>>
>>
>> Thanks
>>
>> s
>>
>>
>>
>> *From:* Moritz Angermann 
>> *Sent:* 14 July 2020 12:14
>> *To:* Simon Peyton Jones 
>> *Cc:* ghc-devs@haskell.org
>> *Subject:* Re: HEAD doesn't build. Totally stalled.
>>
>>
>>
>> For some reason, you end up in the defined RTS_SSP_SYMBOLS, I believe and
>> then the RTS wants __stack_chk symbols. Which it can’t find when linking.
>>
>>
>>
>> Replacing
>>
>> #if !defined(mingw32_HOST_OS) && !defined(DYNAMIC)
>>
>> #define RTS_SSP_SYMBOLS\
>>
>>   SymI_NeedsProto(__stack_chk_guard)   \
>>
>>   SymI_NeedsProto(__stack_chk_fail)
>>
>> #else
>>
>> #define RTS_SSP_SYMBOLS
>>
>> #endif
>>
>> With just
>>
>>
>>
>> #define RTS_SSP_SYMBOLS
>>
>>
>>
>> Should do. I hope.
>>
>>
>>
>> Currently only on mobile phone :-/
>>
>>
>>
>> Cheers,
>>
>>  Moritz
>>
>>
>>
>> On Tue, 14 Jul 2020 at 7:06 PM, Simon Peyton Jones 
>> wrote:
>>
>> thanks.  What specifically do I comment out?
>>
>>
>>
>> *From:* Moritz Angermann 
>> *Sent:* 14 July 2020 12:00
>> *To:* Simon Peyton Jones 
>> *Cc:* ghc-devs@haskell.org
>> *Subject:* Re: HEAD doesn't build. Totally stalled.
>>
>>
>>
>> This was my fault. Not sure why this wasn’t caught in CI.
>>
>> It’s due to the addition of the symbols here
>>
>>
>>
>>
>> https://github.com/ghc/ghc/commit/686e72253aed3880268dd6858eadd8c320f09e97#diff-03f5bc5a50fd8ae13e902782c4392c38R1159
>> 
>>
>>
>>
>> You should be able to just comment them out. I’ll prepare a proper fix.
>>
>>
>>
>> Cheers,
>>
>>  Moritz
>>
>>
>>
>> On Tue, 14 Jul 2020 at 6:41 PM, Simon Peyton Jones via ghc-devs <
>> ghc-devs@haskell.org> wrote:
>>
>> I’m getting this failure in a clean HEAD build. Any ideas?I’m totally
>> stalled because I can’t build GHC any more.
>>
>> I’m using Windows Subsystem for Linux (WSL).
>>
>> Help help!
>>
>> Thanks
>>
>> Simon
>>
>> /home/simonpj/code/HEAD-9/rts/dist/build/libHSrts_thr_p.a(RtsSymbols.thr_p_o):
>> RtsSymbols.c:rtsSyms: error: undefined reference to '__stack_chk_guard'
>>
>> collect2: error: ld returned 1 exit status
>>
>> `cc' failed in phase `Linker'. (Exit code: 1)
>>
>> utils/iserv/ghc.mk:105
>> :
>> recipe for target 'utils/iserv/stage2_p/build/tmp/ghc-iserv-prof' failed
>>
>> make[1]: *** [utils/iserv/stage2_p/build/tmp/ghc-iserv-prof] Error 1
>>
>> make[1]: *** Waiting for unfinished jobs
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>> 
>>
>> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Evac.c

[Phyx]

Hi Ben,

Just FYI, gcc 9 added -fdiagnostics-format=json so that may be an option to
make this sort of thing easier to handle.

Kind regards
Tamar

Sent from my Mobile

On Thu, Jun 11, 2020, 17:50 Ben Gamari  wrote:

> Simon Peyton Jones via ghc-devs  writes:
>
> > I'm getting this in HEAD.  Should I worry?  It says "error"!
> > Simon
> >
> Indeed we probably ought to fix the error message handling here. The
> claim that this is an "error" is from GHC, which parses GCC's error
> output. You see, however, that GCC is merely giving us a warning.
>
> Regardless, the issue itself is non-critical. It's merely telling us
> that we asked for a function to be inlined yet GCC felt it wouldn't be
> wise to do so. This usually happens in the debug runtime due to
> additional assertion code. It's safe to ignore.
>
> Cheers,
>
> - Ben
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Github mirror

[Phyx]

Hi Ben,

It looks like the github mirror for ghc hasn't updated in a month.

Kind regards,
Tamar

Sent from my Mobile
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows testsuite failures

[Phyx]

On Fri, Jan 17, 2020 at 7:02 AM Ömer Sinan Ağacan 
wrote:

> > Now we have rewritten the CI and it's pointing out actual issues in the
> > compiler. And your suggestion is well let's just ignore it.
>
> When is the last time Windows CI caught an actual bug? All I see is random
> system failures [1, 2, 3].
>

[1]: Symbolic link privileges are missing from the CI user, something has
gone wrong with the permissions on that slave.
  There's code in the testsuite to symlink or copy. Should fix the
permissions, or add permission detection to the python code or switch to
copy.
[2]: git checkout error, disk probably full. Testsuite runs tend to create
a lot of temp files which aren't cleaned up. Over time the disk fills and
you get errors such as these.
  There's a cron job to periodically clean these, but of course that is
prone to a race condition. This can be made more reliable by using OS event
triggers instead of a cron job.
  i.e. monitor disk 80% full events and run the cleanup.
[3]: It's either trying to execute a non-executable file or something it
executed loaded a shared library for a different architecture. Hard to tell
which one by just that output. Will need more logs.

Now to answer your question [4] and [5] are issues the CI caught that were
quite important.

[4] https://gitlab.haskell.org/ghc/ghc/issues/17480
[5] https://gitlab.haskell.org/ghc/ghc/issues/17691

just to name two, I can go on, plugin test failures, which pointed out
someone submitted an patch tested only on ELF that broke loading on plugins
as non-shared objects, etc.
The list is quite long.


>
> It must be catching *some* bugs, but that's a rare event in my experience.
>

Sure if you go "ahh it's just Windows that's broken" and don't look at the
underlying issues.


> Sure, I don't write Windows-specific code (e.g. IO manager, or library
> code),
> but then why am I fighting the Windows CI literally every day, it makes no
> sense. Give an option to skip Windows CI for my patches.
>
> > How about you use some of that energy to help I stead of taking the easy
> way?
> > And I bet you're going to say you don't care about Windows to which I
> would
> > say I don't care about the non-threaded runtime and wish we would get
> rid of
> > it. But can't always get what you want.
>
> I'm not suggesting we release buggy GHCs for Windows or stop Windows
> support.
>

I'm sorry, how is disabling the Windows CI not exactly that? If you
Disabling the CI just means you test it even less.
You test it even less means by the time you get to testing it the issues
are too many to fix. Over time you just stop
trying and stop releasing it.  So sorry, how *exactly* is your suggestion
not exactly that.


>
> > And to say we'll actually fix anything before release doesn't align with
> what
> > I've seen so far, which had me scrambling last minute to ensure we can
> release
> > Windows instead of making releases without it.
>
> Are you saying we skip a platform we support when it's buggy? That makes no
> sense. I don't know when did Windows become a first-tier platform but
> since it
> is now we should be releasing Windows binaries similar to Linux and OSX
> binaries.
>

It's *always* been a tier one platform as far as I can tell. It's certainly
been for the past 6 years.


> It's not uncommon to do some testing for every patch, and do more
> comprehensive
> testing before releases. We did this many times in other projects in the
> past
> and I know some other compilers do this today.
>

Yes, but a project that doesn't test a tier one platform during
development, which is what your want to do
means it's not tier one. Which means you won't fix it for release.


>
> > Quite frankly I don't need you to tell me to submit MRs to fix it since
> that's
> > what I spent again a lot of time doing. Or maybe you would like to pay my
> > paycheck so I can spend more than a considerable amount of my free time
> on it.
>
> I wish someone paid me for the time I wasted because I'm only paid by the
> time I
> spend productively. I'd be happier waiting for the CI then.
>

Yeah, not waiting for CI is how we got in this mess in the first place.

Tamar.


> Ömer
>
> [1]: https://gitlab.haskell.org/ghc/ghc/-/jobs/237457
> [2]: https://gitlab.haskell.org/osa1/ghc/-/jobs/238236
> [3]: https://gitlab.haskell.org/osa1/ghc/-/jobs/237279
>
> Phyx , 17 Oca 2020 Cum, 09:49 tarihinde şunu yazdı:
> >
> > Oh I spent a non-insignificant amount of time back in the phabricator
> days to make the CI stable. Now because people were committing to master
> directly without going through CI it was always a cat and mouse game and I
> gave up eventually.
> >
> > Now we have rewritten the CI and it's pointing out actual issues 

Re: Windows testsuite failures

[Phyx]

Oh I spent a non-insignificant amount of time back in the phabricator days
to make the CI stable. Now because people were committing to master
directly without going through CI it was always a cat and mouse game and I
gave up eventually.

Now we have rewritten the CI and it's pointing out actual issues in the
compiler. And your suggestion is well let's just ignore it.

How about you use some of that energy to help I stead of taking the easy
way? And I bet you're going to say you don't care about Windows to which I
would say I don't care about the non-threaded runtime and wish we would get
rid of it. But can't always get what you want.

And to say we'll actually fix anything before release doesn't align with
what I've seen so far, which had me scrambling last minute to ensure we can
release Windows instead of making releases without it.

Quite frankly I don't need you to tell me to submit MRs to fix it since
that's what I spent again a lot of time doing. Or maybe you would like to
pay my paycheck so I can spend more than a considerable amount of my free
time on it.

Kind regards,
Tamar


Sent from my Mobile

On Fri, Jan 17, 2020, 06:17 Ömer Sinan Ağacan  wrote:

> We release more often than once in 6 months.
>
> We clearly have no idea how to test on Windows. If you know how to do it
> then
> feel free to submit a MR. Otherwise blocking every MR indefinitely is
> worse than
> testing Windows less frequently.
>
> Ömer
>
> Phyx , 17 Oca 2020 Cum, 09:10 tarihinde şunu yazdı:
> >
> > Sure because only testing once every 6 months is a very very good idea...
> >
> > Sent from my Mobile
> >
> > On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan 
> wrote:
> >>
> >> Hi Ben,
> >>
> >> Can we please disable Windows CI? I've spent more time fighting the CI
> than
> >> doing useful work this week, it's really frustrating.
> >>
> >> Since we have no idea how to fix it maybe we should test Windows only
> before a
> >> release, manually (and use bisect in case of regressions).
> >>
> >> Ömer
> >>
> >> Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde
> şunu yazdı:
> >> >
> >> > Hi all,
> >> >
> >> > Currently Windows CI is a bit flaky due to some unfortunately rather
> elusive testsuite driver bugs. Progress in resolving this has been a bit
> slow due to travel over the last week but I will be back home tomorrow and
> should be able to resolve the issue soon thereafter.
> >> >
> >> > Cheers,
> >> >
> >> > - Ben
> >> > ___
> >> > ghc-devs mailing list
> >> > ghc-devs@haskell.org
> >> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> >> ___
> >> ghc-devs mailing list
> >> ghc-devs@haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows testsuite failures

[Phyx]

Sure because only testing once every 6 months is a very very good idea...

Sent from my Mobile

On Fri, Jan 17, 2020, 06:03 Ömer Sinan Ağacan  wrote:

> Hi Ben,
>
> Can we please disable Windows CI? I've spent more time fighting the CI than
> doing useful work this week, it's really frustrating.
>
> Since we have no idea how to fix it maybe we should test Windows only
> before a
> release, manually (and use bisect in case of regressions).
>
> Ömer
>
> Ben Gamari , 14 Oca 2020 Sal, 14:30 tarihinde şunu
> yazdı:
> >
> > Hi all,
> >
> > Currently Windows CI is a bit flaky due to some unfortunately rather
> elusive testsuite driver bugs. Progress in resolving this has been a bit
> slow due to travel over the last week but I will be back home tomorrow and
> should be able to resolve the issue soon thereafter.
> >
> > Cheers,
> >
> > - Ben
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: More failure

[Phyx]

Hi Andrey,

I'm not sure what the original issue here is (should probably find the
original message) but

> The Make build system happens to do the right thing, somehow. I believe
we should be able to express the same logic in Shake, but it's not easy.

I believe this typically works because GCC and GHC support dumping the
dependencies that a command would have caused to a file. So your dynamic
dependencies don't matter as their static to the build system after this
invocation.

See
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/separate_compilation.html#dependency-generation

These Compilers are able to dump out make rules which enabled better
dependency handling..

Kind regards,
Tamar

Sent from my Mobile

On Tue, Dec 10, 2019, 00:58 Andrey Mokhov 
wrote:

> Hi Simon,
>
>
>
> (Re-sending from the email address that’s allowed on the mailing list.)
>
>
>
> > Ugh.  That's not a very happy state of affairs, is it?  It didn't happen
> with 'make'.
>
> > Is it a fundamental problem, or just not yet fixed?
>
>
>
> I think this is not a fundamental problem, but the problem of getting
> dependencies right.
>
>
>
> In this case, the complexity comes from the fact that a single invocation
> of GHC produces a set of files, and which set depends on the command line
> flags, which are in turn determined dynamically by reading environment
> settings (specifically, `platformSupportsSharedLibs`).
>
>
>
> Such rules are hard to describe precisely, because build systems are tuned
> to the typical case where we statically know, for every output file, which
> rule produces it -- recall the Tasks = k -> Maybe Task function from our
> paper. In this case, we deal with something like k -> f (Maybe Task)
> instead, i.e. with `f` around the Maybe.
>
>
>
> The Make build system happens to do the right thing, somehow. I believe we
> should be able to express the same logic in Shake, but it's not easy.
>
>
>
> (I never really had a chance to look at dynamic builds, since they are not
> supported on Windows. I guess I should finally find a Linux box for
> Hadrian.)
>
>
>
> Cheers,
>
> Andrey
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Tracking Progress of #7353 (New Windows IO Manager)

[Phyx]

Hi Travis,

The current MR with my WIP is at
https://gitlab.haskell.org/ghc/ghc/merge_requests/1224

This should build without a problem (on Windows, haven't sorted the Linux
build failures yet). The threaded version passes the testsuite (i.e.
-threaded +RTS --io-manager=native) but the non-threaded
version has proven tricky to support correctly. That one still has a
deadlock that I am debugging (though I have made significant progress since
I last updated that MR).

I will push the updated code this weekend. Though I'm afraid that based on
it not getting any review of the design so far and my current work
commitments I won't be able to
get this done for GHC 8.10. So I will punt it to GHC 8.12.

The MR contains a list of TODOs in case you're wondering what's left on it.

Thanks,
Tamar

On Tue, Sep 24, 2019 at 11:39 PM Travis Whitaker 
wrote:

> Hello Haskell friends,
>
> Even though I no longer work at a Windows shop, I've been watching
> https://gitlab.haskell.org/ghc/ghc/issues/7353 with great interest. Last
> I knew Tamar Christina was heroically hacking away on a new IO manager for
> Windows.
>
> Although my low-level Windows knowledge is likely insufficient for me to
> contribute significantly, I'd be very interested in testing the new IO
> manager and helping with debugging in any way I can. How might one go about
> building a GHC with this new Windows IO manager?
>
> Thanks, as always, for all your hard work,
>
> Travis Whitaker
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

margebot.

[Phyx]

Hello,

margebot seems down, or did I miss a memo on how to commit now?

Cheers,
Tamar
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: gitlab upgrade - change in approval system

[Phyx]

I have noticed that you can work around this by editing the Mr and adding
yourself explicitly to the approvals list for it.

It then allows you to approve it.

Sent from my Mobile

On Fri, May 17, 2019, 13:54 Ryan Scott  wrote:

> I would definitely appreciate changing Marge's behavior back to the old
> one. As things stand currently, I am unable to approve anything that I'm
> not explicitly listed as a reviewer for. See here [1] for an example of
> where this has happened.
>
> Ryan S.
> -
> [1] https://gitlab.haskell.org/ghc/ghc/merge_requests/559#note_198714
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Old build system broken

[Phyx]

That looks like stage1 has been improperly configured.

Does /home/simonpj/code/HEAD/inplace/bin/ghc-stage1 --info

*Return anything sensible for target arch? *

*I still use the make system exclusively and haven't noticed a failure. *

*But my nightlies haven't kicked off yet today. *

*Thanks, *

*Tamar *
Sent from my Mobile

On Wed, May 8, 2019, 16:24 Simon Peyton Jones via ghc-devs <
ghc-devs@haskell.org> wrote:

> I know we are supposed to be using Hadrian now, but is the old build
> system supposed to be broken?
>
> A clean build fails with
>
> "inplace/bin/ghc-cabal" check libraries/ghc-prim
>
> "inplace/bin/ghc-cabal" configure libraries/ghc-prim dist-install
> --with-ghc="/home/simonpj/code/HEAD/inplace/bin/ghc-stage1"
> --with-ghc-pkg="/home/simonpj/code/HEAD/inplace/bin/ghc-pkg"
> --disable-library-for-ghci --enable-library-vanilla
> --enable-library-for-ghci --disable-library-profiling --enable-shared
> --configure-option=CFLAGS="-Wall -Werror=unused-but-set-variable
> -Wno-error=inline" --configure-option=LDFLAGS="  "
> --configure-option=CPPFLAGS="   " --gcc-options="-Wall
> -Werror=unused-but-set-variable -Wno-error=inline   " --with-gcc="gcc"
> --with-ld="ld.gold" --with-ar="ar" --with-alex="/usr/bin/alex"
> --with-happy="/usr/bin/happy"
>
> Configuring ghc-prim-0.6.1...
>
> ghc-cabal: '/home/simonpj/code/HEAD/inplace/bin/ghc-stage1' exited with an
>
> error:
>
> Failed to read "target arch" value ""
>
>
>
> libraries/ghc-prim/ghc.mk:4: recipe for target
> 'libraries/ghc-prim/dist-install/package-data.mk' failed
>
> make[1]: *** [libraries/ghc-prim/dist-install/package-data.mk] Error 1
>
> Makefile:123: recipe for target 'all' failed
>
> make: *** [all] Error 2
>
> simonpj@MSRC-3645512:~/code/HEAD$
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Marge bot UX improvement suggestion

[Phyx]

Speaking of bots.. What's a Moe bot? Haha.

Sent from my Mobile

On Thu, Apr 25, 2019, 16:06 Ben Gamari  wrote:

> Ömer Sinan Ağacan  writes:
>
> > I think it'd be useful if Marge listed commits that it's trying to merge
> in the
> > description of the MR so that we could get a list of commits in the
> emails.
> > Currently the emails just say "Marge Bot Barch MR - DO NOT TOUCH" with
> an empty
> > body.
> >
> I'm generally reluctant to invest any more effort into Marge. She's in
> general barely holding together and hopefully within a month we'll be
> able to move to GitLab's native merge train solution.
>
> Cheers,
>
> - Ben
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Validation Failures on aarch64

[Phyx]

Just a wild guess, but considering the time it's taken to run through the
testsuite you're running it on a reasonable AArch64 machine.
You may be hitting the weak memory ordering bugs
https://gitlab.haskell.org/ghc/ghc/issues/15449 , though I haven't followed
the ticket very closely...

On Mon, Apr 8, 2019 at 8:55 PM Travis Whitaker 
wrote:

> Hello GHC devs,
>
> When attempting to validate a patch on aarch64, it seems there are a large
> number of validation failures:
>
> SUMMARY for test run started at Mon Apr  8 07:19:05 2019 UTC
>  0:15:35 spent to go through
> 6890 total tests, which gave rise to
>17169 test cases, of which
>10018 were skipped
>   41 had missing libraries
> 3151 expected passes
>  150 expected failures
>   11 caused framework failures
>0 caused framework warnings
>0 unexpected passes
> 3798 unexpected failures
>0 unexpected stat failures
>
> The failures seem consistent on recent-ish master, specifically the
> neighborhood of 6113d0d4540af7853c71e9f42a41c3b0bab386fd. Is this to be
> expected?
>
> Thanks,
>
> Travis Whitaker
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC label conventions

[Phyx]

Hi Ben,


> I consider Linux/x86-64 to be the default; e.g. if there a ticket isn't
> labelled with one of the operating system labels then it should be
> assumed that either the issue is OS-independent or it's Linux. This is a
> compromise but given that we need to assign labels manually, it didn't
> seem like manually labelling all Linux tickets was worth the effort.
> Also, once !653 lands issue authors will be prompted for their operating
> system in the issue description.
>

That's fair, I had thought this might be the case, but have you considered
that you also
lose the ability to filter using an inclusion filter then? To get list of
x86 only Linux you would have
exclude all competing tags then (if possible).

But this also introduce an ambiguity, there's no way to say for instance
that an issue effects Linux and
Windows for instance as you would just have the Windows tag.


> > 2. The ARM one is that Arm (e.g. pre AArch32) or just all Arm
> > architectures? (I see no AArch64) either.
>
> ARM refers to all ARM architectures. I'll update the description to be
> more clear on this.
>
> > Is this because we didn't have a label for them in trac?
> >
> Pretty much.
>
> > 3. Slightly unrelated, on Phabricator I used to have a customized side
> bar
> > with helpful links such as this one. Can I do the same on GitLab?
> >
> I'm afraid the answer may be "no". I have this page in my browser's
> bookmarks list. I suppose we could add it to _sidebar [1], which is
> displayed in the sidebar on the right side of the screen when viewing
> the wiki; however this seems like a bit of a slippery slope as it isn't
> really a top-level page.
>

Ah ok, was just checking, I can do a local plugin instead.

Thanks,
Tamar


> Cheers,
>
> - Ben
>
> [1] https://gitlab.haskell.org/ghc/ghc/wikis/_sidebar
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GitHub Mirror is broken

[Phyx]

Hi Ben,

I think the mirror is stuck again. Hasn't updated in 8 days.

Cheers,
Tamar

On Tue, Feb 19, 2019 at 6:30 AM Ben Gamari  wrote:

> Artem Pelenitsyn  writes:
>
> > Hello devs,
> >
> > This is just to let you know that the latestes commit on GitHub ghc/ghc
> > repo dates back to 22th of January. Personally, I find GitHub mirror
> quite
> > useful for ocasional searches over the code base. Therefore, I'd
> > appreciated repairing the mirror.
> >
> Fixed. It seems like the mirroring service got stuck.
>
> Cheers,
>
>  - Ben
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Proposal: Don't read environment files by default

[Phyx]

> I am quite confused as to how people are using `ghci` without loading the
environment files, at least in the context of cabal v2 (aka "new cabal").
 When you run `ghci` on its own, unless you load an environment file, you
would only have access to globally installed packages, which is almost
never what you want.   What is the workflow that this proposal optimizes?

That's precisely how I use ghci. In quite a few of my use-cases all I care
or want are the boot libraries and nothing else. I can appreciate that I
may not be the common case here but I definitely use it this way.

I'm agnostic on whether this change or not of reading the environment files
by default. But I am very much against the cabal ghci or cabal ghc
interfaces. It's one of the things I loathe the most about stack. I don't
care what the compiler does by default on his own but I don't want to add
another layer to the onion.

I find this very hard to debug. Perhaps cabal should just ask you what you
want the first time and that'll be your default.

Tamar

Sent from my Mobile

On Thu, Mar 28, 2019, 18:10 Iavor Diatchki  wrote:

> I am quite confused as to how people are using `ghci` without loading the
> environment files, at least in the context of cabal v2 (aka "new cabal").
>  When you run `ghci` on its own, unless you load an environment file, you
> would only have access to globally installed packages, which is almost
> never what you want.   What is the workflow that this proposal optimizes?
>
> The default behavior should be what's commonly useful and, in my
> experience, when I run GHCi in the context of a project, I pretty much
> always want it to load the environment associated with the project.   It is
> incredibly useful when you are working on a project where there are
> multiple broken modules (e.g., during refactoring), and you want to fix
> them one at a time, in the order that makes sense to you.
>
> -Iavor
>
>
> On Thu, Mar 28, 2019 at 10:02 AM Bardur Arantsson 
> wrote:
>
>> On 28/03/2019 14.58, Oleg Grenrus wrote:
>> > There is. Add
>> >
>> > write-ghc-environment-files: never
>> >
>> > to your ~/.cabal/config assuming you have cabal-insall-2.4.1.0 or later.
>> >
>>
>> That doesn't really work if you actually want to be able to use both
>> ways of working, does it? That same thing applies to
>>
>>   export GHC_ENVIRONMENT=-
>>
>> which someone else posted, but at least that can be done by tooling
>> before invoking ghc. It's odd to have to change a global setting to
>> avoid this. (However, thanks for the hints -- I'll be setting that
>> GHC_ENVIRONMENT from now on.)
>>
>> +1 for changing the default.
>>
>> It seems really weird to force other tooling to opt out when this could
>> easily be solved by just having
>>
>>cabal ghci
>>cabal ghc
>>
>> commands which set up the environment properly and tell users to use
>> that if they want to use cabal's environment files. FWIW, I also see
>> e.g. ghc as low-level tooling akin to the plain 'gcc' command whereas
>> e.g. cabal or stack are more like cmake + make/ninja, i.e. it's not
>> something users should really be running unless they know what they're
>> doing *and* it should be as tooling-friendly as possible.
>>
>> Regards,
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows release quality

[Phyx]

Hi Andreas,

GHC 8.6.4 not supporting profiling libs was the first thing mentioned in
the release email

 - A regression resulting in segmentation faults on Windows introduced
   by the fix for #16071 backported in 8.6.3. This fix has been reverted,
   meaning that 8.6.4 is once again susceptible to #16071. #16071 will
   be fixed in GHC 8.8.1.

It was also stated that it would be back in 8.8.1.  At this point there was
no way to get profiling libs
on 8.6.x without a major backport of linker changes from master.  The
choice was made to revert the
change and release 8.6.4 without profiling libraries because of a stack
allocation bug that was dormant for
years but completely killed the 32 bit distribution. That said the
changelog linked to the wrong issue, the second
two should have been #15934 but that's not hard to figure out by looking at
the ticket.

This was not an oversight, both the choice to release GHC 8.6.4 without
profiling libs (which really to the average user is not mission critical)
and the fact to release 8.6.4 at all were thought out decisions. It could
have been communicated better yes.

That 8.6.3 wasn't removed I don't know. I pulled it from chocolatey at
least.

8.6 is more production ready than 8.4 was, it just doesn't support
profiling libs for a while till 8.8 yes.

Tamar.

On Tue, Mar 19, 2019 at 5:57 PM Andreas Klebinger 
wrote:

> Hello Devs,
>
> After running into #16408 today I realized there is as of yet no released
> bindist
> of the 8.6 series which I would consider stable for windows.
>
> GHC 8.6.1 and 8.6.2 had a series of critical bugs which applied to
> multiple platforms: https://gitlab.haskell.org/ghc/ghc/issues/16408
> GHC 8.6.3 loops forever if compiling certain code using TH on windows.
> This affects some very popular hackage packages: (#16057)
> 
> GHC 8.6.4 (marked stable) currently ships without profiling libraries,
> making profiling impossible.
>
> Being stuck with 8.4 is one thing, and if properly communicated not too
> bad.
> But it requires work to even find out about these (major) issues and to
> discover that 8.6 is NOT production ready for windows.
>
> We offered the broken 8.6.3 as stable for weeks without any indication
> that it was broken.
> We still serve GHC 8.6.4 as stable without any hint about the missing
> profiling libraries.
>
> I can't offer solutions in this case but I feel like something about the
> release management has to change if .
> Having to check the GHC bugtracker to find out if the current stable
> release is actually stable is just not sustainable.
>
>
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Discussion: Hadrian's defaults

[Phyx]

Agreed, I was also an opponent of -c as the default, and as you pointed out
it only works for the case where the default is used. But even if the
defaults are used it is still harmful to do it automatically as the user's
environment could have changed resulting in different configure output if
you automatically rerun it.

Not only isn't there a sane UX for this, there isn't even a safe way to do
this.

Sent from my Mobile

On Fri, Mar 15, 2019, 16:51 Alp Mestanogullari  wrote:

> I have to admit I sympathize with Moritz's view. Since `-c` only
> "subsumes" the case where we call 'configure' with no extra env var or
> argument, and in the absence of a generic way to pass options to
> 'configure' when using -c, I'd quite like to keep -c as a "cherry on top",
> for users who just need to boot and configure with the default arguments
> and for whom hadrian provides a way to save a few keystrokes. Just like
> Moritz, I'm not even sure it would make all that much sense to provide a
> way to pass configure arguments through hadrian, I have a hard time seeing
> a better UX than just running configure with the extra arguments directly
> and _then_ calling hadrian to start the build.
> On 15/03/2019 14:35, Moritz Angermann wrote:
>
> Hi Arnaud,
>
>
> On Mar 15, 2019, at 8:32 PM, Spiwack, Arnaud  
>  wrote:
>
> On Thu, Mar 14, 2019 at 7:20 PM Herbert Valerio Riedel  
>  wrote:
> I don't have the ticket number at my fingertips but it should be fairly easy 
> to find.
>
> I'm afraid it doesn't appear to be. Could you share your arguments in this 
> thread?
>
> This was the last one that lead to the current `-c` state:
> - https://github.com/snowleopard/hadrian/issues/457
> There is also
> - https://github.com/snowleopard/hadrian/issues/655
>
> if you look through the issues on snowleopard/hadrian and sort by comment 
> frequency
> you'll likely find quite a lot of further discussion about not making 
> configure and
> boot the default.
>
>
>
> On Fri, Mar 15, 2019 at 3:10 AM Moritz Angermann  
>  wrote:
> It's magically conflating two different phases with `-c`. The configure phase 
> and
> the build phase. Making this the default means it's always magic. I don't 
> like magic!
>
> Unfortunately, I really don't understand what you are saying. What's magic 
> about combining the phases?
>
>
> We have two phases:
>
> Phase 1: autoconf
>
>   This phase is essentially a code-generation phase, where specific templates 
> are
>   instantiated to configure time value.  Which again can be split into two 
> specific
>   subproblems:
>
>   - Generation of the configure script from the configure.ac and aclocal.m4 
> files
> using autoconf.
>   - Generating code using the configure script by computing configure time 
> calues
> and filling those into the `.in` files producing the files that lack the 
> `.in`
> extension.
>
> Phase 2: building
>
>   This has been traditionally the job of make, and this is what hadrian should
>   replace.
>
>
> By subsuming the configure phase (by invoking ./configure) from hadrian we 
> loose
> the phase distinction and if the `-c` flag is optional, users will *not even 
> see*
> a flag that indicates that the system will run `./configure` for them. This 
> is the
> magic I'm referring to and to which I strongly object.  If we can retire 
> autoconf
> and do the whole configuration in hadrian, that story may change.  But as 
> long as
> we are using an autoconf based configuration we should *not* run that 
> magically.
> The `-c` flag is at least there to show that hadrian is explicilty instructed 
> to
> run configure.
>
> ./configure supports its own set of flags, if hadrian subsumes those, we'd 
> need
> some generic way of passing flags to ./configure, at which point I have to ask
> why do we do this in the first place and try to call ./configure from within 
> hadrian?
>
> Unless you want to reconfigure ghc, or hack on it's autoconf part, you are 
> likely
> going to run the following only:
>
> ./boot --hadrian
> ./configure 
> ./hadrian/build.sh -j ...
> ./hadrian/build.sh -j ...
> ./hadrian/build.sh -j ...
> ./hadrian/build.sh -j ...
> ...
>
> the configure step is required, and should be explicit. That is where you 
> configure
> your ghc build. Set host/build/target values, and other configure flags that
> influence how you want your ghc to be configure. Hadrian is there to build 
> that
> configuration.  Mixing both may be convenient but hides the fact that there 
> is a
> ./configure step.  I consider this hiding to be magic which is meant to 
> benefit the
> user but hides what's really going on.  And again I don't like magic.
>
> Cheers,
>  Moritz
>
> PS: we also don't hide the `./configure` step in the usual `./configure 
>  && make -j`
> instructions when building other software, even though you could surely 
> hack that into
> your Makefile if you so wanted to.  Why start with ghc now?
>
>
> ___
> ghc-devs mailing 
> 

Re: Trac to GitLab migration underway

[Phyx]

Hi all,

Question, how do I view only Windows issues? I see there is a windows tag
but nothing is linked to it. And tickets contain sporadically the "Trac
Metadata" field.

So I'm not sure how to get the same overview I used on trac.

Kind regards,
Tamar

On Tue, Mar 12, 2019, 18:42 Richard Eisenberg  wrote:

>
> > On Mar 11, 2019, at 8:13 PM, Ben Gamari  wrote:
> >
> > Richard Eisenberg  writes:
> > For instance, consider the case of
> > #16347 [1]. You will note that below the "Related issues" section there
> > is a list of related merge requests (strangely formatted completely
> > differently).
>
> Aha. That's exactly what I wanted. If that note is added to an issue when
> an MR mentions the issue number, I'm quite satisfied.
>
> >
> > To be honest, how this list is formed is a bit of a mystery to me. The
> > fact that !525 is included is not surprises: !525 mentions #16347 in
> > its description. However, !436 is a bit less obvious since it does not
> > mention #16347 at all. My hypothesis is that it is transitively included
> > via #16344, which does mention #16437 and is related to it.
>
> Transitive closures have a habit of sometimes getting large. But we'll
> tackle that problem when/if it comes up.
> > Indeed, as I mentioned earlier this may be a bit of a hard thing to
> > accomplish in the near-term. One measure we could take to ensure that
> > tests aren't forgotten is to include a "field" in the default merge
> > request description which contributors would be asked to fill in with
> > the names of the tests that cover their change.
>
> That's not a bad starting point. But it will hit only new contributors,
> right? Or will I be filling out such a form too (it's been a few weeks
> since my last patch submission, I'm afraid.)
>
> Thanks!
> Richard
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [Haskell-cafe] Final steps in GHC's Trac-to-GitLab migration

[Phyx]

Can they be sent from a different email address than the main.
Gitlab+margebot are quite. Ahum, noisy.. and filtering based on message
content has a potential for false positives.

Kind regards,
Tamar

On Wed, Mar 6, 2019, 12:33 Matthew Pickering 
wrote:

> I think gitlab can be configured so notifications are sent for new issues
> and comments on issues which should achieve the same thing as the mailing
> list did?
>
>
>
> On Wed, Mar 6, 2019 at 12:00 PM Sylvain Henry  wrote:
>
>> I use it to track tickets and I would also like to see it continued.
>>
>> Sylvain
>>
>> On 06/03/2019 12:33, Ara Adkins wrote:
>> > Personally I would like to see it continued, but it may not be worth
>> the work if I’m in a minority here.
>> >
>> > A potential stopgap would be to ‘watch’ the GHC project on our gitlab
>> instance, but I can’t see any way to decide to get emails for notifications
>> rather than having to check in at GitLab all the time.
>> >
>> > _ara
>> >
>> >> On 6 Mar 2019, at 11:21, Ben Gamari  wrote:
>> >>
>> >>
>> >>
>> >>> On March 6, 2019 6:11:49 AM EST, Ara Adkins  wrote:
>> >>> Super excited for this! Thank you to everyone whose put in so much
>> hard
>> >>> work to get it done!
>> >>>
>> >>> One question: what is happening with the trac tickets mailing list? I
>> >>> imagine it’ll be going away, but for those of us that use it to keep
>> >>> track of things is there a recommended alternative?
>> >>>
>> >> The ghc-commits list will continue to work.
>> >>
>> >> The ghc-tickets list is a good question. I suspect that under gitlab
>> there will be less need for this list but we may still want to continue
>> maintaining it regardless for continuity's sake. Thoughts?
>> >>
>> >> Cheers,
>> >>
>> >> - Ben
>> >>
>> >>
>> >>
>> >>> Best,
>> >>> _ara
>> >>>
>>  On 6 Mar 2019, at 01:21, Ben Gamari  wrote:
>> 
>>  Hi everyone,
>> 
>>  Over the past few weeks we have been hard at work sorting out the
>>  last batch of issues in GHC's Trac-to-GitLab import [1]. At this
>> >>> point I
>>  believe we have sorted out the issues which are necessary to perform
>> >>> the
>>  final migration:
>> 
>>  * We are missing only two tickets (#1436 and #2074 which will require
>> >>> a
>>    bit of manual intervention to import due to extremely large
>>    description lengths)
>> 
>>  * A variety of markup issues have been resolved
>> 
>>  * More metadata is now preserved via labels. We may choose to
>>    reorganize or eliminate some of these labels in time but it's
>> >>> easier
>>    to remove metadata after import than it is to reintroduce it. The
>>    logic which maps Trac metadata to GitLab labels can be found here
>> >>> [2]
>>  * We now generate a Wiki table of contents [3] which is significantly
>>    more readable than GitLab's default page list. This will be updated
>>    by a cron job until underlying GitLab pages list becomes more
>>    readable.
>> 
>>  * We now generate redirects for Trac ticket and Wiki links (although
>>    this isn't visible in the staging instance)
>> 
>>  * Milestones are now properly closed when closed in Trac
>> 
>>  * Mapping between Trac and GitLab usernames is now a bit more robust
>> 
>>  As in previous test imports, we would appreciate it if you could have
>> >>> a
>>  look over the import and let us know of any problems your encounter.
>> 
>>  If no serious issues are identified with the staging site we plan to
>>  proceed with the migration this coming weekend. The current migration
>>  plan is to perform the final import on gitlab.haskell.org on
>> >>> Saturday, 9
>>  March 2019.
>> 
>>  This will involve both gitlab.haskell.org and ghc.haskell.org being
>> >>> down
>>  for likely the entirety of the day Saturday and likely some of Sunday
>>  (EST time zone). Read-only access will be available to
>>  gitlab.staging.haskell.org for ticket lookup while the import is
>>  underway.
>> 
>>  After the import we will wait at least a week or so before we begin
>> >>> the
>>  process of decommissioning Trac, which will be kept in read-only mode
>>  for the duration.
>> 
>>  Do let me know if the 9 March timing is problematic.
>> 
>>  Cheers,
>> 
>>  - Ben
>> 
>> 
>>  [1] https://gitlab.staging.haskell.org/ghc/ghc
>>  [2]
>> >>>
>> https://github.com/bgamari/trac-to-remarkup/blob/master/TicketImport.hs#L227
>>  [3] https://gitlab.staging.haskell.org/ghc/ghc/wikis/index
>>  ___
>>  Haskell-Cafe mailing list
>>  To (un)subscribe, modify options or view archives go to:
>>  http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
>>  Only members subscribed via the mailman list are allowed to post.
>> >> --
>> >> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>> > 

Re: [ANNOUNCE] GHC 8.6.4 is now available

[Phyx]

Thanks! To be fair I'm not really sure how popular it is. I was wondering
since once I upload the chocolatey packages it's immutable.

If there's anything I can help with let me know.

Cheers,
Tamar

On Tue, Mar 5, 2019, 21:47 Ben Gamari  wrote:

> Phyx  writes:
>
> > Hi Ben,
> >
> > Thanks for the release! I was wondering, any reason for the no i386
> Windows?
> > Should I just create the package without it or is it coming?
> >
> This is one configuration I haven't yet had a chance to CI-ify. I was
> hoping it would be missed but since you mention it I've gone ahead and
> put up an attempt at fixing this (!495 [1]). It will likely take a few
> iterations to get right but I'll try to get it sorted by the end of the
> week.
>
> Cheers,
>
> - Ben
>
>
> [1] https://gitlab.haskell.org/ghc/ghc/merge_requests/495
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 8.6.4 is now available

[Phyx]

Hi Ben,

Thanks for the release! I was wondering, any reason for the no i386 Windows?
Should I just create the package without it or is it coming?

Thanks,
Tamar

On Tue, Mar 5, 2019 at 8:53 PM Ben Gamari  wrote:

>
> Hello everyone,
>
> The GHC team is very happy to announce the availability of GHC 8.6.4, a
> bugfix release in the GHC 8.6 series. The source distribution, binary
> distributions, and documentation for this release are available at
>
> https://downloads.haskell.org/~ghc/8.6.4
>
> The 8.6.4 release fixes several regressions present in 8.6.3 including:
>
>  - A regression resulting in segmentation faults on Windows introduced
>by the fix for #16071 backported in 8.6.3. This fix has been reverted,
>meaning that 8.6.4 is once again susceptible to #16071. #16071 will
>be fixed in GHC 8.8.1.
>
>  - A bug resulting in incorrect locking on Darwin, potentially resulting
>in hangs at shutdown (#16150)
>
>  - A few bugs in the profiled runtime resulting in the potential for
>memory unsafety has been fixed (#15508).
>
>  - The `process` and `transformers` libraries shipped now properly
>reflect released Hackage releases (#16199)
>
>  - A bug where TH name resolution would break in a plugin context has
>been fixed (#16104)
>
>  - Compilers that do not support TemplateHaskell no longer advertise
>such support in `--supported-languages` (#16331)
>
> As a few of these issues are rather serious users are strongly
> encouraged to upgrade. See Trac [1] for a full list of issues resolved
> in this release.
>
> Note that this release ships with one significant but long-standing bug
> (#14251): Calls to functions taking both Float# and Double# may result
> in incorrect code generation when compiled using the LLVM code generator.
> This is not a new issue, it has existed as long as the LLVM code
> generator has existed; however, changes in code generation in 8.6 made
> it more likely that user code using only lifted types will trigger it.
>
> Note also that this is the first release cut from our (yet again)
> revamped continuous integration infrastructure. While we have done a
> great deal of checking to ensure that the build configuration reflects
> that of previous releases, do let us know if something looks off.
>
> Happy compiling!
>
> Cheers,
>
> - Ben
>
> [1]
> https://ghc.haskell.org/trac/ghc/query?status=closed=8.6.4=id=summary=status=type=priority=milestone=component=priority
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Scaling back CI (for now)?

[Phyx]

That aside, the CIs don't seem stable at all. Frequent timeouts even before
they start. I have been trying to merge 3 changes for a while now and
everytime one of them times out and I have to restart the timed out ones.
Then there are merge conflicts and I have to start over.

This is "bot wackamole" :)

On Sun, Feb 3, 2019, 13:56 Matthew Pickering 
wrote:

> It has been established today that Marge is failing to run in batch
> mode for some reason which means it takes at least as long as CI takes
> to complete for each commit to be merged. The rate is about 4
> commits/day with the current configuration.
>
> On Sat, Feb 2, 2019 at 7:57 PM Sebastian Graf  wrote:
> >
> > Hi,
> >
> > Am Sa., 2. Feb. 2019 um 16:09 Uhr schrieb Matthew Pickering <
> matthewtpicker...@gmail.com>:
> >>
> >>
> >> All the other flavours should be run once the commit reaches master.
> >>
> >> Thoughts?
> >
> >
> > That's even better than my idea of only running them as nightlies. In
> favor!
> >
> >>
> >> Cheers,
> >>
> >> Matt
> >> ___
> >> ghc-devs mailing list
> >> ghc-devs@haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: WIP branches

[Phyx]

The solution I use to this branch overload is changing my fetch refspecs to
list explicitly the branches I want.

https://git-scm.com/book/en/v2/Git-Internals-The-Refspec

It's not ideal but it gets the job done. I wish git allowed you to exclude
branches instead, as I could just exclude /wip/* then.

Tamar

On Tue, Feb 5, 2019, 19:15 Sylvain Henry  wrote:

> > What is the advantage of having ghc-wip instead of having all devs just
> have their own forks?
>
> I am all for each dev having its own fork. The ghc-wip repo would be just
> for devs having an SVN workflow (i.e. several people working with commit
> rights on the same branch/fork). If no-one uses this workflow or if Gitlab
> allows fine tuning of permissions on user forks, we may omit the ghc-wip
> repo altogether.
>
> Regards,
> Sylvain
>
> PS: you didn't send your answer to the list, only to me
>
> On 05/02/2019 19:44, Richard Eisenberg wrote:
> > I agree that movement in this direction would be good (though I don't
> feel the pain from the current mode -- it just seems suboptimal). What is
> the advantage of having ghc-wip instead of having all devs just have their
> own forks?
> >
> > Thanks,
> > Richard
> >
> >> On Feb 5, 2019, at 11:36 AM, Sylvain Henry  wrote:
> >>
> >> Hi,
> >>
> >> Every time we fetch the main GHC repository, we get *a lot* of "wip/*"
> branches. That's a lot of noise, making the bash completion of "git
> checkout" pretty useless for instance:
> >>
> >>> git checkout 
> >> zsh: do you wish to see all 945 possibilities (329 lines)?
> >>
> >> Unless I'm missing something, they seem to be used to:
> >> 1) get the CI run on personal branches (e.g. wip/USER/whatever)
> >> 2) share code between different people (SVN like)
> >> 3) archival of not worth merging but still worth keeping code (cf
> https://ghc.haskell.org/trac/ghc/wiki/ActiveBranches)
> >>
> >> Now that we have switched to Gitlab, can we keep the main repository
> clean of those branches?
> >> 1) The CI is run on user forks and on merge requests in Gitlab so we
> don't need this anymore
> >> 2 and 3) Can we have a Gitlab project ("ghc-wip" or something) that
> isn't protected and dedicated to this? The main project could be protected
> globally instead of per-branch so that only Ben and Marge could create
> release branches, merge, etc. Devs using wip branches would only have to
> add "ghc-wip" as an additional remote repo.
> >>
> >> Any opinion on this?
> >>
> >> Thanks,
> >> Sylvain
> >>
> >> ___
> >> ghc-devs mailing list
> >> ghc-devs@haskell.org
> >> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Thoughts on the Contributing page

[Phyx]

>
> Indeed. I hope it's not difficult to fix this, but I'm not sure where to
> start. Any suggestions are very welcome.
>
>
It is unfortunately, because it's not a bug. Odds are both of you are using
MinTTY which people blindly recommend to use.
The problem is MinTTY is not designed to run native Windows applications,
it is designed to run ported POSIX applications. e.g. things linked against
the
msys2/cygwin runtime. Because of this it implements a very intrusive hack
in order to be able to handle signals the way they would be under a posix
application,
which simply isn't compatible with Windows signals see
https://github.com/mintty/mintty/issues/56

Because your build is triggered by a batch file, when you press ctrl+c
mintty will terminate the parent process indiscriminately if it's a script,
e.g. the batch process is killed but
none of the children are.  This is why ghci has ghcii.sh as a workaround.
e.g. why pressing ctrl+c terminate ghci instead of the computation as we
can't.

this is why you should not use MinTTY with a native Windows application.
MinTTY also introduces a number of character encoding problems for native
Windows applications, see
https://github.com/mintty/mintty/wiki/Tips#inputoutput-interaction-with-alien-programs.
In short, the issues you're seeing are by design.

Now you have two options:

1) Don't use mintty, instead use something like ConEmu
https://conemu.github.io/ with bash.exe from msys2 as the terminal
host.This will work for Native application but have
issues with interactive posix applications like msys2 gdb (mingw64 gdb
should in theory work fine though, but it's slightly more limited) and
i.e.. tmux won't work. (the problems are thus flipped).

2) Use WinPTY https://github.com/rprichard/winpty to wrap native
application calls (you can just install it via pacman), which works and you
can keep using MinTTY, but this does have a slight performance overhead
as the reason it works is that it spawns a hidden buffer and scrapes output
from it and send events to it to check the native behavior for Windows
applications.

TL;DR; It's not a simple issue, the application can only deal with one
signal processing method at a time.

Cheers,
Tamar


> Cheers,
> Andrey
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] You should try Hadrian

[Phyx]

Hi Andrey

On Sun, Jan 27, 2019 at 10:49 PM Andrey Mokhov <
andrey.mok...@newcastle.ac.uk> wrote:

> Hi Tamar,
>
>
>
> Here is the relevant bullet point from the README:
>
>
>
> > On Windows, if you do not want to install MSYS, you can
>
> > use the Stack-based build script (Stack provides a managed
>
> > MSYS environment), as described in these instructions.
>
> > If you don't mind installing MSYS yourself or already have it,
>
> > you can use the Cabal-based build script.
>

Yes, I was referring to the "My first build" heading which had a call to
build.bat, but it seems my branch was just old and the file was updated 11
days ago to use cabal instead of stack.
Now the rest of the file also makes sense. Apologies for that, I thought I
had updated


>
> This claim is based on my experience. Installing the MSYS environment has
> never worked out smoothly for me. Doing this via Stack was indeed more
> robust (especially, when struggling with building GHC on Windows CI!). Has
> this been different in your experience?
>
>
>

Yes, stack does nothing special than just un-tar the binary distribution of
msys2. The problem is that this binary distribution is not kept up to date
unless things break. By that point they may have gotten so out of date that
the distribution simply can't even be upgraded. e.g. A while ago they used
a distribution that's so old it couldn't deal with pacman's invalidating
old certificates, which means you couldn't use it to update ca-certificates.

It also can't handle when msys upstream changes core dependencies. One such
update is a change in march that introduced a cyclic dependency between catgets
libcatgets and some packages. Or when they change the package layout as
they did removing the old shell scripts and making Mingw32.exe and
Mingw64.exe. I can name many more. The fact is the msys2 installers are set
up to work around these updates, or you must work around them when
initializing the environment to fix these.

And I see no evidence based on past issues that stack actually keeps their
msys2 installs up to date. So I don't want to go into the business of
managing stack msys2 issues for ghc builds.

> I'm just confused when it was decided to switch the defaults,
>
> > and why, without any consultation.
>
>
>
> I’m not sure what you mean. Could you clarify? The file `doc/windows.md`
> is 3 years old and hasn’t changed much since creation. The default build
> script `build.bat` currently uses Cabal:
>
>
>
> ```
>
> rem By default on Windows we build Hadrian using Cabal
>
> hadrian/build.cabal.bat %*
>
> ```
>

Yes.. I'm pretty sure when I looked at it before today it was pointing to
build.stack.bat, but that seems to be a two week old tree. So my fault
there.

Sorry, should have checked on gitlab!

Regards,
Tamar


>
>
> P.S.: I’ve just noticed that `doc/windows.md` hasn’t been updated when
> moving to GitLab, and created this MR to fix this:
>
>
>
> https://gitlab.haskell.org/ghc/ghc/merge_requests/239
>
>
>
> Please jump into the comments there if you’d like me to fix/clarify
> anything.
>
>
>
> Thanks for reaching out!
>
>
>
> Cheers,
>
> Andrey
>
>
>
> *From:* Phyx [mailto:loneti...@gmail.com]
> *Sent:* 27 January 2019 21:11
> *To:* Andrey Mokhov ; Ben Gamari <
> b...@well-typed.com>
> *Cc:* GHC developers 
> *Subject:* Re: [ANNOUNCE] You should try Hadrian
>
>
>
> Hi Andrey,
>
>
>
> I'm looking at
> https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/README.md and
> https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/windows.md
>
> wondering why the default instructions for Windows are using stack, this
> isn't currently the case.
>
>
>
> In order for ./boot and configure to work already you need to be in an
> msys2 environment. So having stack install its own, un-updated msys2 is not
> a workflow I would recommend.
>
>
>
> There's a dubious claim there that using stack is "more robust", what is
> this claim based on?
>
> I'm just confused when it was decided to switch the defaults, and why,
> without any consultation.
>
>
>
> Regards,
>
> Tamar
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] You should try Hadrian

[Phyx]

Hi Andrey,

I'm looking at
https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/README.md and
https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/windows.md
wondering why the default instructions for Windows are using stack, this
isn't currently the case.

In order for ./boot and configure to work already you need to be in an
msys2 environment. So having stack install its own, un-updated msys2 is not
a workflow I would recommend.

There's a dubious claim there that using stack is "more robust", what is
this claim based on?
I'm just confused when it was decided to switch the defaults, and why,
without any consultation.

Regards,
Tamar

On Fri, Jan 25, 2019 at 2:27 AM Andrey Mokhov 
wrote:

> Dear GHC developers,
>
>
>
> Summary: You should try to use Hadrian as the GHC build system, because it
> will (hopefully!) become the default around GHC 8.8.
>
>
>
> What is Hadrian and how can I try it?
>
> =
>
>
>
> Hadrian is a new build system for GHC written in Haskell. It lives in the
> directory “hadrian” in the GHC tree, and we have been actively developing
> it in the past year to reach feature and correctness parity with the
> existing Make-based build system. While we haven't quite reached this goal
> (more on this below), Hadrian is already working well and we run Hadrian
> jobs alongside the Make ones in our CI pipelines since the recent move to
> GitLab.
>
>
>
> At this point, we would like to encourage everyone to try using Hadrian
> for their usual GHC development tasks. Hadrian's documentation resides in
> GHC's source tree, and below are the documents you will be most interested
> in:
>
>
>
> ·https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/README.md:
> The root of Hadrian's documentation. It explains the basics and points to
> more specific documents where appropriate.
>
>
>
> ·
> https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/make.md: A
> cheatsheet-style document for GHC developers used to the Make build system
> (that is, most/all of you), showing equivalent Make/Hadrian commands for
> many tasks.
>
>
>
> ·
> https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/user-settings.md:
> A description of the “user settings” mechanism in Hadrian, which is where
> you can customise the build flavour, choose the packages to build, add
> file/package/platform-specific command line flags, etc.
>
>
>
> ·
> https://gitlab.haskell.org/ghc/ghc/blob/master/hadrian/doc/testsuite.md:
> A description of the “test” rule and all the options it supports.
>
>
>
> The documentation can surely be improved, so please do not hesitate to
> send us feedback and suggestions here, or even better on Trac: make sure
> you choose the component "Build System (Hadrian)" when creating a new
> ticket.
>
>
>
> You need Hadrian
>
> 
>
>
>
> Hadrian is new, requires time to learn, and still has rough edges, but it
> has been developed to make your lives better. Here are a few advantages of
> Hadrian over the Make-based build system:
>
>
>
> 1) Hadrian is more reliable.
>
>
>
> Hadrian can capture build dependencies more accurately, which means you
> rarely (if ever) need to do a clean rebuild.
>
>
>
> 2) Hadrian is faster.
>
>
>
> Hadrian is faster for two reasons: (i) more accurate build dependencies,
> (ii) tracking of file contents instead of file modification times. Both
> allow you to avoid a lot of unnecessary rebuilds. Building Hadrian itself
> may take a while but needs to be done only once.
>
>
>
> 3) Hadrian is easier to understand and modify.
>
>
>
> You no longer need to deal with Make's global namespace of mutable string
> variables. Hadrian is written in the language you love; it has modules,
> types and pure functions.
>
>
>
>
>
> If you come across a situation where Hadrian is worse than the Make build
> system in any of the above aspects, this is a bug and you should report it.
>
>
>
> Helping Hadrian
>
> ===
>
>
>
> The best way to help is to try Hadrian, and let us know how it goes, what
> doesn't work, what's missing for you, what you think should be easier, and
> so on. Below is a list of known issues that we are in the process of fixing
> or that we will be tackling soon:
>
>
>
> ·Stage 2 GHC should be dynamically linked most of the time, but
> it never is, currently. See https://ghc.haskell.org/trac/ghc/ticket/15837
>
> ·There are about a dozen of failing tests in the GHC testsuite,
> some related to #15837.
>
> ·Binary distributions haven't been thoroughly tested on many
> platforms (only some Linux flavours). There will definitely be some issues
> here. For example, the binary distribution rule currently fails on Windows:
> https://ghc.haskell.org/trac/ghc/ticket/16073.
>
> ·There is no “validate” rule yet, only “test”, but we have all
> the pieces to make this happen and it has a very high priority.
>
> ·There are issues with building cross compilers: see
> 

Re: Using same stdout/stderr files for multiple tests?

[Phyx]

Actually, I had a few minutes to spare on the train so
https://gitlab.haskell.org/ghc/ghc/merge_requests/226 :)

Cheers,
Tamar

On Sun, Jan 27, 2019 at 4:36 PM Phyx  wrote:

> Hi Omer,
>
> If you frequently require this it would be a trivial thing to add, it's a
> small modification to find_expected_file,
> I can probably find some time next week to do this for you if you want.
>
> >
> > I don't believe there is a way to do this. I would likely make
> > test_debug.stdout a symlink to test.stdout.
>
> Please don't, the symlinks won't persist on Windows where we'll end up
> with file copies instead, which goes horribly wrong once you modify one of
> them.
>
> Regards,
> Tamar
>
> On Wed, Jan 23, 2019 at 4:44 AM Ömer Sinan Ağacan 
> wrote:
>
>> These days when I need something like this I just use a make rule, which
>> is not
>> great as you have to run multiple tests in one make rule and you can't
>> run only
>> one with the TESTS parameter or skip some of the tests.
>>
>> Another problem is I can't have multiple tests that use single source
>> file,
>> again I have to use a make rule.
>>
>> Ömer
>>
>> Ben Gamari , 23 Oca 2019 Çar, 02:44 tarihinde şunu
>> yazdı:
>> >
>> > Ömer Sinan Ağacan  writes:
>> >
>> > > I have a test that I want to run with different compile and runtime
>> parameters.
>> > > I managed to reuse the source file across different tests by adding a
>> > > extra_files(['source.hs']) to the tests, but I don't know how to do
>> the same for
>> > > stdout/stderr files. Any ideas?
>> > >
>> > > In more details, I have
>> > >
>> > > test.hs
>> > > test.stdout
>> > >
>> > > and two tests
>> > >
>> > > test('test',
>> > >  [extra_run_opts('...')],
>> > >  compile_and_run,
>> > >  [])
>> > >
>> > > test('test_debug',
>> > >  [extra_run_opts('...'),
>> > >   extra_hc_opts('-debug'),
>> > >   extra_files(['test.hs'])],
>> > >  compile_and_run,
>> > >  [])
>> > >
>> > > The first test works fine, but the second test fails because I don't
>> know how to
>> > > tell it to use test.stdout as the stdout file and it looks for
>> > > test_debug.stdout.
>> > >
>> > This is probably no longer relevant but:
>> >
>> > I don't believe there is a way to do this. I would likely make
>> > test_debug.stdout a symlink to test.stdout.
>> >
>> > Cheers,
>> >
>> > - Ben
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Using same stdout/stderr files for multiple tests?

[Phyx]

Hi Omer,

If you frequently require this it would be a trivial thing to add, it's a
small modification to find_expected_file,
I can probably find some time next week to do this for you if you want.

>
> I don't believe there is a way to do this. I would likely make
> test_debug.stdout a symlink to test.stdout.

Please don't, the symlinks won't persist on Windows where we'll end up with
file copies instead, which goes horribly wrong once you modify one of them.

Regards,
Tamar

On Wed, Jan 23, 2019 at 4:44 AM Ömer Sinan Ağacan 
wrote:

> These days when I need something like this I just use a make rule, which
> is not
> great as you have to run multiple tests in one make rule and you can't run
> only
> one with the TESTS parameter or skip some of the tests.
>
> Another problem is I can't have multiple tests that use single source file,
> again I have to use a make rule.
>
> Ömer
>
> Ben Gamari , 23 Oca 2019 Çar, 02:44 tarihinde şunu
> yazdı:
> >
> > Ömer Sinan Ağacan  writes:
> >
> > > I have a test that I want to run with different compile and runtime
> parameters.
> > > I managed to reuse the source file across different tests by adding a
> > > extra_files(['source.hs']) to the tests, but I don't know how to do
> the same for
> > > stdout/stderr files. Any ideas?
> > >
> > > In more details, I have
> > >
> > > test.hs
> > > test.stdout
> > >
> > > and two tests
> > >
> > > test('test',
> > >  [extra_run_opts('...')],
> > >  compile_and_run,
> > >  [])
> > >
> > > test('test_debug',
> > >  [extra_run_opts('...'),
> > >   extra_hc_opts('-debug'),
> > >   extra_files(['test.hs'])],
> > >  compile_and_run,
> > >  [])
> > >
> > > The first test works fine, but the second test fails because I don't
> know how to
> > > tell it to use test.stdout as the stdout file and it looks for
> > > test_debug.stdout.
> > >
> > This is probably no longer relevant but:
> >
> > I don't believe there is a way to do this. I would likely make
> > test_debug.stdout a symlink to test.stdout.
> >
> > Cheers,
> >
> > - Ben
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [GitLab] Introducing Marge-bot

[Phyx]

> I also don't think one should be allowed to approve their own
> PR. If it is trivial enough to justify a self-accept then someone else
> should also be able to trivially accept it.

I disagree whole heartedly, as someone who's had to wait weeks for trivial
patches to get reviews no thanks.
We should have a  formal definition of what is allowed to get committed as
trivial much like a lot of open source
projects do and go from there.

I prefer a practical workflow, not just one that works for areas of the
compiler where you have many people working,
It's a very frustrating experience otherwise.

Tamar.

On Wed, Jan 23, 2019 at 9:29 AM Matthew Pickering <
matthewtpicker...@gmail.com> wrote:

> It seems that in order for marge-bot to work best we need to
> tighten up our policy towards merging so that it is only Marge
> who performs the merges. I think Marge gets confused if people
> push to master under her feet which means rebasing again and duplicating
> work.
>
> Can we disable the "Merge when pipeline succeeds button" in order to
> facilitate this?
>
> I also don't think one should be allowed to approve their own
> PR. If it is trivial enough to justify a self-accept then someone else
> should also be able to trivially accept it.
>
> Therefore I believe the correct workflow is:
>
> 1. Make a MR and assign it to someone if you want their specific review.
> 2. When the MR has been approved, the approver assigns the MR to marge.
> 3. Marge then performs the merge in her own time.
>
> Cheers,
>
> Matt
>
> On Tue, Jan 22, 2019 at 8:31 PM Ben Gamari  wrote:
> >
> > Hi everyone,
> >
> > As you might have noticed there is a new face on GitLab: Meet
> > @marge-bot.
> >
> > Marge will be helping us with the pain of merging merge requests:
> > Currently the typical workflow to merge an accepted MR involves the
> > following:
> >
> >  1. Rebase the MR on top of the current `master` branch
> >  2. Click on the "Merge when pipeline succeeds" button
> >  3. Wait.
> >  4. If another MR is merged before yours, return to step (1)
> >
> > Given the volume of patches that we have, this process gets tiresome
> > quite quickly. Upstream knows [1] about this issue and is actively
> > working towards a solution which will likely be ready in a few months.
> >
> > In the meantime, Marge automates this currently-manual process. With
> > Marge merging a patch involves just two steps:
> >
> >  1. Ensure that the MR has at least one approval. This should happen in
> > the course of normal review but ping @bgamari, @alpmestan, @osa1, or
> > @tdammers if this was forgotten.
> >
> >  2. Use the "Assignee" field in the sidebar on the right side of the MR
> > to assign it to @marge-bot.
> >
> > Once Marge notices your MR she will dutifully watch over it, rebasing it
> > as necessary until it is merged. If something goes awry, she will leave
> > a (hopefully) helpful message and assign the MR back to you.
> >
> > So far Marge has been working out reasonably well and seems to be an
> > improvement over the status quo. However, she still has some quirks so
> > let me know if you think she is behaving erratically or otherwise have
> > questions.
> >
> > Cheers,
> >
> > - Ben
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: MR does not merge

[Phyx]

The problem is with the different platforms that need to be tested. I could
see this getting stuck in an infinite retest and rebase loop because you
have to wait on different builds to finish not just one.

So I think this is a harder problem to solve than we're making it out to
be. The system needs some concept of fairness and prevention of this retry
cycle. And you can't not retest the patch after a rebase as the
functionality of the already merged patch may be conflicting.

I imagine the bit cannot serialize all testing on all platforms otherwise
the slowest platform becomes the problem.

On Wed, Jan 16, 2019, 15:20 Evan Laforge  wrote:

> At work we use "marge bot", https://github.com/smarkets/marge-bot
> which is an automatic bot that will do the rebase and resubmit thing.
> I think it's pretty essential, because otherwise any intervening merge
> means you have to babysit the merge button.  It's also more efficient
> if you have expensive CI, because it serializes the runs.
>
> BTW, I've extended marge with a "try rebase, then try merge" strategy,
> which is useful if people merge from head, and a "merge CI run" which
> is useful if you have an expensive CI you want to run only on merge,
> not on every single branch push.
>
> On Wed, Jan 16, 2019 at 10:56 PM Matthew Pickering
>  wrote:
> >
> > There is problem with the interaction between "merge when validated"
> > and "fast forward merge only" option.
> >
> > If anyone commits to master between clicking the button and validation
> > finishing then the merge will fail as the patch needs to be rebased
> > before it can be merged.
> >
> > I'm not sure what the plan to deal with this is.
> >
> > On Wed, Jan 16, 2019 at 2:49 PM Simon Peyton Jones via ghc-devs
> >  wrote:
> > >
> > > Ben
> > >
> > > Six days ago I submitted this MR
> > >
> > > https://gitlab.haskell.org/ghc/ghc/merge_requests/109
> > >
> > > Just tiny refactorings.  I said “merge when validated”
> > >
> > > But six days later, it still appears not to have merged.  What’s up?
> I was expecting it to merge in a matter of an hour or two.
> > >
> > > Thanks
> > >
> > > Simon
> > >
> > > ___
> > > ghc-devs mailing list
> > > ghc-devs@haskell.org
> > > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Blocking a task indefinitely in the RTS

[Phyx]

I'm trying to model an Asynchronous I/O call/interface against a
synchronous Haskell call.
I want the behavior of an unsafe blocking foreign call, without blocking in
the foreign call itself.



On Tue, Jan 8, 2019 at 6:24 PM Carter Schonwald 
wrote:

> What’s the underlying problem you’re trying to model?
>
> On Tue, Jan 8, 2019 at 3:56 AM Phyx  wrote:
>
>> > Oh, I see :(  I guess it's not that easy of a fix then.  Perhaps the
>> RTS could use a new intrinsic for blocking on foreign state
>>
>> Yeah, that's what I was/am currently working on, "IOPort" has much of the
>> same property of MVar but doesn't have this deadlock guarantee and only
>> supports a single put/take at a time.
>> But debugging CMM is... not fun :( so I was wondering if I was just
>> missing something with the existing mechanisms.
>>
>>
>> On Tue, Jan 8, 2019 at 8:23 AM Phil Ruffwind  wrote:
>>
>>> > I did try removing this check to see, but it really didn't like that.
>>> It
>>> > caused GC to be triggered over and over again as the RTS tried
>>> desperately
>>> > to find something to do, doesn't seem to consider "do nothing" as a
>>> valid
>>> > state.
>>>
>>> Oh, I see :(  I guess it's not that easy of a fix then.  Perhaps the RTS
>>> could use a new intrinsic for blocking on foreign state.
>>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Blocking a task indefinitely in the RTS

[Phyx]

> Oh, I see :(  I guess it's not that easy of a fix then.  Perhaps the RTS
could use a new intrinsic for blocking on foreign state

Yeah, that's what I was/am currently working on, "IOPort" has much of the
same property of MVar but doesn't have this deadlock guarantee and only
supports a single put/take at a time.
But debugging CMM is... not fun :( so I was wondering if I was just missing
something with the existing mechanisms.


On Tue, Jan 8, 2019 at 8:23 AM Phil Ruffwind  wrote:

> > I did try removing this check to see, but it really didn't like that. It
> > caused GC to be triggered over and over again as the RTS tried
> desperately
> > to find something to do, doesn't seem to consider "do nothing" as a valid
> > state.
>
> Oh, I see :(  I guess it's not that easy of a fix then.  Perhaps the RTS
> could use a new intrinsic for blocking on foreign state.
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Blocking a task indefinitely in the RTS

[Phyx]

> Maybe that should be considered a false positive (bug) for the deadlock
checker?  Just because the Haskell runtime has a single thread, that
doesn't imply the whole program is necessarily single-threaded (in the
presence of foreign things).  I'd think this is a legitimate use case for
MVars.

Perhaps, I can see it both ways really :(. The papers and docs for MVars
put a strong emphasis on this deadlock guarantee (though I've really only
skimmed the paper), taking foreign calls into consideration does somewhat
weaken this of course. I do agree with you that this wasn't the behavior I
expected from MVars and STMs, particularly because they are such low level
building blocks, for instance QSem is constructed with MVar, the docs make
no mention of this but I suspect QSem and the rest all don't do what you'd
expect on the non-threaded RTS.

I did try removing this check to see, but it really didn't like that. It
caused GC to be triggered over and over again as the RTS tried desperately
to find something to do, doesn't seem to consider "do nothing" as a valid
state.

On Tue, Jan 8, 2019 at 6:59 AM Phil Ruffwind  wrote:

> Okay, I skimmed rts/Schedule.c and now see the problem you mentioned :(
>
> > On the non-threaded runtime the timeslice case doesn't apply and you only
> > have one capability, it will force a GC to try to revive some tasks, and
> if
> > at the end of
> > this the tasks are still blocked it will release one in order to attempt
> to
> > proceed. In short, as far as I can tell I don't think it considers
> > reach-ability at all, not for MVars.
>
> Maybe that should be considered a false positive (bug) for the deadlock
> checker?  Just because the Haskell runtime has a single thread, that
> doesn't imply the whole program is necessarily single-threaded (in the
> presence of foreign things).  I'd think this is a legitimate use case for
> MVars.
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Blocking a task indefinitely in the RTS

[Phyx]

I have simply copy pasted the code you provided. Note that my actual code
code doesn't pass anything to a foreign interface. It stores everything in
a Haskell mutable object.
The RTS on completion of actions simply schedules a task which inspects
this objects and wakes up as many blocked tasks as possible.

> Strange, how could the scheduler assume a deadlock if the MVar could be
called from a closure that is still alive?  Can you show the code that
you're testing?

Because as far as I can tell, it doesn't care. When it comes to MVars and
STMs the scheduler assumes a deadlock when all tasks on all capabilities
are blocked.
For the threaded runtime it has an early exit condition from this code in
the case where there has been any activity in a complete timeslice or when
you're blocked on
specific calls such as I/O.

On the non-threaded runtime the timeslice case doesn't apply and you only
have one capability, it will force a GC to try to revive some tasks, and if
at the end of
this the tasks are still blocked it will release one in order to attempt to
proceed. In short, as far as I can tell I don't think it considers
reach-ability at all, not for MVars.

This is why things e.g. doing a takeMVar will also process pending puts
etc, so that if you actually enter a blocked state, you know you had no
other choice.

On Mon, Jan 7, 2019 at 11:24 PM Phil Ruffwind  wrote:

> Strange, how could the scheduler assume a deadlock if the MVar could be
> called from a closure that is still alive?  Can you show the code that
> you're testing?
>
> On Mon, Jan 7, 2019, at 14:09, Phyx wrote:
> > Hi Phil,
> >
> > Thanks for the reply, however that just gives me a forced deadlock
> removal
> > as before.
> >
> > new bound thread (1)
> > cap 0: schedule()
> > cap 0: running thread 1 (ThreadRunGHC)
> > cap 0: thread 1 stopped (blocked on an MVar)
> > thread1 @ 03205388 is blocked on an MVar @
> > 032040c8 (TSO_DIRTY)
> > deadlocked, forcing major GC...
> > all threads:
> > threads on capability 0:
> > other threads:
> > thread1 @ 03205388 is blocked on an MVar @
> > 032040c8 (TSO_DIRTY)
> > cap 0: starting GC
> >
> > I don't believe any solution involving MVars will work for the
> non-threaded
> > RTS. Though I'd love to be wrong here...
> >
> > Regards,
> > Tamar
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Blocking a task indefinitely in the RTS

[Phyx]

Hi John,

I can, but the only reason that would work is because a blocking foreign
call is except from the
deadlock detection code.  I can't block on the Handle itself as all the I/O
calls are non-blocking for performance reasons.

Instead what I want to do is just pause the current task, which means I
don't have to allocate OS level locks for each task
and have a mutable state for each task, which is essentially what an MVar
does.  My other concern is that using OS primitives
would hit performance a bit, which increases as the load increases. So I'd
much rather do it purely in the Haskell world.

Kind regards,
Tamar

On Mon, Jan 7, 2019 at 5:44 AM John Lato  wrote:

> Can you use an os-level structure? E.g. block on a file descriptor,
> socket, or something like that?
>
> On Sun, Jan 6, 2019, 10:37 Phyx 
>> Hi All,
>>
>> I'm looking for a way to block a task indefinitely until it is woken up
>> by an external event in both the threaded and non-threaded RTS and returns
>> a value that was stored/passed. MVar works great for the threaded RTS, but
>> for the non-threaded there's a bunch of deadlock detection in the scheduler
>> that would forcibly free the lock and resume the task with an opaque
>> exception. This means that MVar and anything derived from it is not usable.
>>
>> STMs are more expensive but also have the same deadlock code. So again no
>> go. The reason it looks like a deadlock to the RTS is that the "Wake-up"
>> call in the non-threaded rts will come from C code running inside the RTS.
>> The RTS essentially just sees all tasks blocked on it's main capability and
>> (usually rightly so) assumes a deadlock occurred.
>>
>> You have other states like BlockedOnForeign etc but those are not usable
>> as a primitive. Previous iterations of I/O managers have each invented
>> primitives for this such as asyncRead#, but they are not general and can't
>> be re-used, and requires a different solution for threaded and non-threaded.
>>
>> I have started making a new primitive IOPort for this, based on the MVar
>> code, but this is not trivial... (currently I'm getting a segfault
>> *somewhere* in the primitive's cmm code). The reason is that the semantics
>> are decidedly different from what MVars guarantee. I should also mention
>> that this is meant to be internal to base (i.e no exported).
>>
>> So before I continue down this path and some painful debugging..., does
>> anyone know of a way to block a task, unblock it later and pass a value
>> back? It does not need to support anything complicated such as multiple
>> take/put requests etc.
>>
>> Cheers,
>> Tamar
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Blocking a task indefinitely in the RTS

[Phyx]

Hi Phil,

Thanks for the reply, however that just gives me a forced deadlock removal
as before.

new bound thread (1)
cap 0: schedule()
cap 0: running thread 1 (ThreadRunGHC)
cap 0: thread 1 stopped (blocked on an MVar)
thread1 @ 03205388 is blocked on an MVar @
032040c8 (TSO_DIRTY)
deadlocked, forcing major GC...
all threads:
threads on capability 0:
other threads:
thread1 @ 03205388 is blocked on an MVar @
032040c8 (TSO_DIRTY)
cap 0: starting GC

I don't believe any solution involving MVars will work for the non-threaded
RTS. Though I'd love to be wrong here...

Regards,
Tamar

On Sun, Jan 6, 2019 at 9:38 PM Phil Ruffwind  wrote:

> What if you wrap the MVar in a foreign closure?
>
> import Control.Concurrent.MVar (newEmptyMVar, putMVar, takeMVar)
> import Control.Exception (bracket)
> import Foreign.Ptr (FunPtr, freeHaskellFunPtr)
>
> foreign import ccall "wrapper" wrapAwaken :: IO () -> IO (FunPtr (IO
> ()))
>
> main = do
>   mvar <- newEmptyMVar
>   bracket (wrapAwaken (putMVar mvar ())) freeHaskellFunPtr $ \ awaken
> -> do
> -- giveToExternalCode awaken
> takeMVar mvar
>
> On Sun, Jan 6, 2019, at 10:37, Phyx wrote:
> > Hi All,
> >
> > I'm looking for a way to block a task indefinitely until it is woken up
> by
> > an external event in both the threaded and non-threaded RTS and returns a
> > value that was stored/passed. MVar works great for the threaded RTS, but
> > for the non-threaded there's a bunch of deadlock detection in the
> scheduler
> > that would forcibly free the lock and resume the task with an opaque
> > exception. This means that MVar and anything derived from it is not
> usable.
> >
> > STMs are more expensive but also have the same deadlock code. So again no
> > go. The reason it looks like a deadlock to the RTS is that the "Wake-up"
> > call in the non-threaded rts will come from C code running inside the
> RTS.
> > The RTS essentially just sees all tasks blocked on it's main capability
> and
> > (usually rightly so) assumes a deadlock occurred.
> >
> > You have other states like BlockedOnForeign etc but those are not usable
> as
> > a primitive. Previous iterations of I/O managers have each invented
> > primitives for this such as asyncRead#, but they are not general and
> can't
> > be re-used, and requires a different solution for threaded and
> non-threaded.
> >
> > I have started making a new primitive IOPort for this, based on the MVar
> > code, but this is not trivial... (currently I'm getting a segfault
> > *somewhere* in the primitive's cmm code). The reason is that the
> semantics
> > are decidedly different from what MVars guarantee. I should also mention
> > that this is meant to be internal to base (i.e no exported).
> >
> > So before I continue down this path and some painful debugging..., does
> > anyone know of a way to block a task, unblock it later and pass a value
> > back? It does not need to support anything complicated such as multiple
> > take/put requests etc.
> >
> > Cheers,
> > Tamar
> > ___
> > ghc-devs mailing list
> > ghc-devs@haskell.org
> > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Blocking a task indefinitely in the RTS

[Phyx]

Hi All,

I'm looking for a way to block a task indefinitely until it is woken up by
an external event in both the threaded and non-threaded RTS and returns a
value that was stored/passed. MVar works great for the threaded RTS, but
for the non-threaded there's a bunch of deadlock detection in the scheduler
that would forcibly free the lock and resume the task with an opaque
exception. This means that MVar and anything derived from it is not usable.

STMs are more expensive but also have the same deadlock code. So again no
go. The reason it looks like a deadlock to the RTS is that the "Wake-up"
call in the non-threaded rts will come from C code running inside the RTS.
The RTS essentially just sees all tasks blocked on it's main capability and
(usually rightly so) assumes a deadlock occurred.

You have other states like BlockedOnForeign etc but those are not usable as
a primitive. Previous iterations of I/O managers have each invented
primitives for this such as asyncRead#, but they are not general and can't
be re-used, and requires a different solution for threaded and non-threaded.

I have started making a new primitive IOPort for this, based on the MVar
code, but this is not trivial... (currently I'm getting a segfault
*somewhere* in the primitive's cmm code). The reason is that the semantics
are decidedly different from what MVars guarantee. I should also mention
that this is meant to be internal to base (i.e no exported).

So before I continue down this path and some painful debugging..., does
anyone know of a way to block a task, unblock it later and pass a value
back? It does not need to support anything complicated such as multiple
take/put requests etc.

Cheers,
Tamar
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows test failures

[Phyx]

forgot to copy in ghc-devs.

On Fri, Dec 7, 2018 at 12:05 AM Phyx  wrote:

> Ah great,
>
> Normally an msys2 .profile will contain the following line
>
> # Set user-defined locale
> export LANG=$(locale -uU)
>
> which would attach ".UTF-8" to the user's current locale.
> I'm guessing when you run it from emacs the profile isn't loaded (probably
> because bash isn't being called with --login) or something in bashrc or
> profile is overwriting this.
>
> Setting this should bring the test failures down more.
>
> Which leaves only these tests
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/plugins/plugin-recomp-pure.run  plugin-recomp-pure [bad
> exit code] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/plugins/plugin-recomp-impure.runplugin-recomp-impure
> [bad exit code] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/plugins/plugin-recomp-flags.run plugin-recomp-flags [bad
> exit code] (normal)
>
> as the remaining unexplained ones.
>
> Do these still fail for you?
>
> Cheers,
> Tamar
>
>
> On Thu, Dec 6, 2018 at 11:50 PM Simon Peyton Jones 
> wrote:
>
>> Aha!   Yes, in libraries/base/tests, I find that make TEST=T3307 fails;
>> but succeeds with
>>
>>
>>
>> export LANG=en_GB.UTF-8
>>
>>
>>
>> Progress!
>>
>>
>>
>> Simon
>>
>>
>>
>> *From:* Phyx 
>> *Sent:* 06 December 2018 23:43
>> *To:* Simon Peyton Jones 
>> *Cc:* ghc-devs@haskell.org Devs 
>> *Subject:* Re: Windows test failures
>>
>>
>>
>> Hi Simon,
>>
>>
>>
>> > Does that help at all?
>>
>> >
>>
>> > I can try your “export LANG=en_GB.UTF-8” … shall I do that? Can I test
>> its efficacy by running one test rather than 6,000 of them?  In which case,
>> which one?
>>
>>
>>
>> Yes, "en_GB" doesn't seem to be a unicode locale, one test you can try to
>> check is T3307,so make TEST="T3307" -C testsuite/tests/
>>
>>
>>
>> I tried using your "en_GB" locale and the test failed for me too then.
>>
>>
>>
>> Kind regards,
>>
>> Tamar
>>
>>
>>
>> On Thu, Dec 6, 2018 at 2:17 PM Simon Peyton Jones 
>> wrote:
>>
>> Hi Tamar
>>
>>
>>
>> Thanks for working on this.
>>
>>
>>
>> if it's an msys2 shell, what does "locale" return?
>>
>>
>>
>> It’s a shell running inside emacs.  Here’s what locale returns
>>
>> /c/tmp$ locale
>>
>> LANG=en_GB
>>
>> LC_CTYPE="en_GB"
>>
>> LC_NUMERIC="en_GB"
>>
>> LC_TIME="en_GB"
>>
>> LC_COLLATE="en_GB"
>>
>> LC_MONETARY="en_GB"
>>
>> LC_MESSAGES="en_GB"
>>
>> LC_ALL=
>>
>>
>>
>> Does that help at all?
>>
>>
>>
>> I can try your “export LANG=en_GB.UTF-8” … shall I do that? Can I test
>> its efficacy by running one test rather than 6,000 of them?  In which case,
>> which one?
>>
>>
>> Thanks
>>
>>
>>
>> Simon
>>
>>
>>
>>
>>
>> *From:* Phyx 
>> *Sent:* 02 December 2018 20:43
>> *To:* Simon Peyton Jones 
>> *Cc:* ghc-devs@haskell.org Devs 
>> *Subject:* Re: Windows test failures
>>
>>
>>
>> Hi Simon,
>>
>>
>>
>> That's a bit better (still need to figure out why the recent threading
>> issues, but one problem at a time :) )
>>
>>
>>
>> From that list T10672_x64 is one I'm looking at already, seems to have
>> something to do with the libstdc++ destructors.
>>
>> Plugins 09 and 10 are the other two I know about, but haven't had time to
>> look at them yet. Frankly I know too little about plugins to make an
>> accurate determination here, but the input files are empty
>>
>> yet it expects output, so I don't know what it's supposed to do here.  If
>> someone who knows more about plugins can chime in that would save some time.
>>
>>
>>
>> The segfaulting plugin I haven't triaged yet. Now the remaining failures
>> aside from T14452 that Roland is taking care of, seem to have to do with
>> your locale in your console.  You seem to be running the
>>
>> tests in a console that has latin-1 locale? So some unicode characters
>> fail enco

Re: Windows test failures

[Phyx]

Hi Roland,

Thanks for looking into thiis,

> To fix the issue on Windows, the compiler and the plugin should use the
same buffer for stdout.

I'm not convinced that they're not. I'm guessing the answer lies in why
your messages have a different
ordering, but I don't know why off the top of my head.

if the plugins use the stdout caf then it should be using the same
CharBuffer.

You can verify this by doing something like

cbuf <- readIORef haCharBuffer stdout
summaryBuffer cbuf
putStrLn $ "buffer: " ++ show cbuf

at the places you check the buffer mode.

> However I don't know whether this is possible / difficult / easy?
> What's your opinion?

On Tue, Dec 4, 2018 at 11:52 AM Roland Senn  wrote:

> Hi Tamar,
>
> WINDOWS
> ===
> On Windows I did the following changes before running the 'plugin09' test:
> 1.) In the compiler (HscMain.hs), just before calling the plugin function
> 'parsedPlugin', I set BufferMode of the file stdout to LineBuffering.
> 2.) At the same place, I write a message to stdout with the text "COMPILER
> About to call plugin parse" and the result of the buffer-mode query.
> 3.) In the plugin, in the function parsedPlugin, I query and print (to
> stdout) the buffer mode.
> 4.) I added the heading PLUGIN to the normal parse message issued by the
> parsedPlugin function
> 5.) In the compiler (HscMain.hs) just after returning from the plugin, I
> print the line "COMPILER Returning from plugin parse" to stdout.
> 6.) In the  plugin function interfaceLoadPlugin' that is called much
> later, I flush stdout, and add the heading "PLUGIN".
>
> This gives the following interesting result:
>
>  COMPILER About to call plugin parse: LineBuffering
>  COMPILER Returning from plugin parse
>  PLUGIN Buffermode: BlockBuffering Nothing
>  PLUGIN parsePlugin(a,b)
>  PLUGIN interfacePlugin: Prelude
>  ...
>
> The output lines do not appear in the sequence they were produced!!
> The plugin doesn't see/inherit the BlockBuffer mode (LineBuffering) set by
> the compiler!!
>
> This is a strong indication, that *there are two different buffers for
> stdout*. One in the compiler and another one in the plugin.
> At the end of the processing, the buffer in the compiler is automatically
> flushed, however the buffer in the plugin never gets flushed!
>
> LINUX
> =
> I did a similar test in Linux, however, here I set the buffer mode to
> 'Blockmode Nothing' and I didn't do a manual flush in the plugin. I got the
> following result:
>
>  COMPILER About to call plugin parse: Buffering mode: BlockBuffering
> Nothing
>  PLUGIN Buffering: BlockBuffering Nothing
>  PLUGIN parsePlugin(a,b)
>  COMPILER Returning from plugin parse
>  PLUGIN interfacePlugin: Prelude
>  ...
>
> Here the lines are in the same order as they were produced.
> The setting of the Buffering mode is inherited by the plugin.
>
> I think, on Linux the compiler and the plugin share the same buffer.
>
> To fix the issue on Windows, the compiler and the plugin should use the
> same buffer for stdout.
> However I don't know whether this is possible / difficult / easy?
> What's your opinion?
>
> Many thanks and kind regards
>Roland
>
> Here are my changes for Windows in code:
>
> Change in HscMain the line "import System.IO (fixIO)" to "import System.IO
> "
>
> Last lines of function HscMain.hs:hscParse'
>
> -- apply parse transformation of plugins
> let applyPluginAction p opts
>   = parsedResultAction p opts mod_summary
> liftIO $ hSetBuffering stdout LineBuffering
> mode <- liftIO $ hGetBuffering stdout
> liftIO $ putStrLn ("COMPILER About to call plugin parse: " ++
> show mode)
> rsxresult <- withPlugins dflags applyPluginAction res
> liftIO $ putStrLn "COMPILER Returning from plugin parse"
> return rsxresult
>
> New code for function SourcePlugin.hs:parsedPlugin
>
> parsedPlugin opts _ pm
>   = do
>mode <- liftIO $ hGetBuffering stdout
>liftIO $ putStrLn $ "PLUGIN Buffermode: " ++ show mode
>liftIO $ putStrLn $ "PLUGIN parsePlugin(" ++ intercalate "," opts
> ++ ")"
>return pm
>
> New code for function SourcePlugin.hs:interfaceLoadPlugin'
>
> interfaceLoadPlugin' :: [CommandLineOption] -> ModIface -> IfM lcl ModIface
> interfaceLoadPlugin' _ iface
>   = do liftIO $ putStrLn $ "PLUGIN interfacePlugin: "
>   ++ (showSDocUnsafe $ ppr $ mi_module iface)
>liftIO $ hFlush stdout
>return iface
>
>
>
> Am Die

Re: Windows test failures

[Phyx]

Hi Simon,

> Does that help at all?

>

> I can try your “export LANG=en_GB.UTF-8” … shall I do that? Can I test
its efficacy by running one test rather than 6,000 of them?  In which case,
which one?


Yes, "en_GB" doesn't seem to be a unicode locale, one test you can try to
check is T3307,so make TEST="T3307" -C testsuite/tests/


I tried using your "en_GB" locale and the test failed for me too then.


Kind regards,

Tamar

On Thu, Dec 6, 2018 at 2:17 PM Simon Peyton Jones 
wrote:

> Hi Tamar
>
>
>
> Thanks for working on this.
>
>
>
> if it's an msys2 shell, what does "locale" return?
>
>
>
> It’s a shell running inside emacs.  Here’s what locale returns
>
> /c/tmp$ locale
>
> LANG=en_GB
>
> LC_CTYPE="en_GB"
>
> LC_NUMERIC="en_GB"
>
> LC_TIME="en_GB"
>
> LC_COLLATE="en_GB"
>
> LC_MONETARY="en_GB"
>
> LC_MESSAGES="en_GB"
>
> LC_ALL=
>
>
>
> Does that help at all?
>
>
>
> I can try your “export LANG=en_GB.UTF-8” … shall I do that? Can I test
> its efficacy by running one test rather than 6,000 of them?  In which case,
> which one?
>
>
> Thanks
>
>
>
> Simon
>
>
>
>
>
> *From:* Phyx 
> *Sent:* 02 December 2018 20:43
> *To:* Simon Peyton Jones 
> *Cc:* ghc-devs@haskell.org Devs 
> *Subject:* Re: Windows test failures
>
>
>
> Hi Simon,
>
>
>
> That's a bit better (still need to figure out why the recent threading
> issues, but one problem at a time :) )
>
>
>
> From that list T10672_x64 is one I'm looking at already, seems to have
> something to do with the libstdc++ destructors.
>
> Plugins 09 and 10 are the other two I know about, but haven't had time to
> look at them yet. Frankly I know too little about plugins to make an
> accurate determination here, but the input files are empty
>
> yet it expects output, so I don't know what it's supposed to do here.  If
> someone who knows more about plugins can chime in that would save some time.
>
>
>
> The segfaulting plugin I haven't triaged yet. Now the remaining failures
> aside from T14452 that Roland is taking care of, seem to have to do with
> your locale in your console.  You seem to be running the
>
> tests in a console that has latin-1 locale? So some unicode characters
> fail encoding/decoding.
>
>
>
> If it's a Windows shell you can change it to utf-8 using "chcp 65001", if
> it's an msys2 shell, what does "locale" return?
>
>
>
> For reference mine is
>
>
>
> $ locale
> LANG=en_GB.UTF-8
> LC_CTYPE="en_GB.UTF-8"
> LC_NUMERIC="en_GB.UTF-8"
> LC_TIME="en_GB.UTF-8"
> LC_COLLATE="en_GB.UTF-8"
> LC_MONETARY="en_GB.UTF-8"
> LC_MESSAGES="en_GB.UTF-8"
> LC_ALL=
>
>
>
> If it does say latin1 you can change it with
>
>
>
> export LANG=en_GB.UTF-8
>
>
>
> This should fix more of the tests.
>
>
>
> The reason I don't mark the remaining tests as expect fail yet is because
> I haven't had the time to triage them, so I don't know their severity and
>
> last time there were a few nasty issues hidden in them.
>
>
>
> Unfortunately I won't have time to look at them till next weekend.
>
>
>
> Thanks,
>
> Tamar
>
>
>
> On Fri, Nov 30, 2018 at 9:49 PM Simon Peyton Jones 
> wrote:
>
> At the end of the first test run it would have given a list of tests that
> failed and a line saying TEST=" List of tests..."
>
>
>
> Copy that line and at the root of the checkout do
>
>
>
> make TEST=" List of tests..."  test -C testsuite/tests
>
>
>
> (that's uppercase C). This will run everything using one thread. :)
>
>
>
> OK, done.  Results below.
>
>
>
> Simon
>
>
>
>
>
> /c/code/HEAD$ make TEST="T10420 T10672_x64 T13385 T14452 T15815 T3307
> T3319 T4006 TH_scopedTvs environment001 plugin-recomp-change
> plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins07
> plugins09 plugins10 plugins11 plugins13 plugins14 print017"  test -C
> testsuite/tests
>
> make: Entering directory '/c/code/HEAD/testsuite/tests'
>
> PYTHON="python3" "python3" ../driver/runtests.py  -e
> "ghc_compiler_always_flags='-dcore-lint -dcmm-lint -no-user-package-db
> -rtsopts  -fno-warn-missed-specialisations -fshow-warning-groups
> -fdiagnostics-color=never -fno-diagnostics-show-caret -Werror=compat
> -dno-debug-output'" -e config.compiler_debugged=False -e
> ghc_with_native_codegen=True -e config.have_van

Re: Windows test failures

[Phyx]

Hi Roland,

Thanks for looking into these.

> I looked into the testcases 'plugins09', 'plugins10' and 'plugins11' and
found the following: GHC-Windows uses BufferMode 'BlockBuffering Nothing',
however, GHC-Linux uses 'LineBuffering'.

Ah, yes, this isn't technically a Linux vs Windows thing, GHC will always
default to LineBuffering for terminals and BlockBuffering for anything
else. The issue is that msys2 terminals have std handles that are backed by
files instead of pipes. This is an artifact of how they try to emulate
posix shells, See
https://github.com/msys2/msys2/wiki/Porting#standard-streams-in-mintty.
This means that when GHC runs inside an msys2 program such as bash it'll
always default to BlockBuffering.

> I don't know anything about the "Why" and "Where" in the GHC IO module on
Windows, so I'm unable to come up with a patch.

The above said the handles should be getting flushed when the finalizers
are run, but these aren't 100% guaranteed so if the tests rely on this then
your solution (to force buffer mode to LineBuffering) sounds like perfectly
adequate.  Could you put a patch up with that?

My new I/O manager takes a different approach to determine the buffer mode,
but I still have some kinks to work out before posting it.

> PS: I can't say anything about the tests 'plugin-recomp-pure' and
'plugin-recomp-impure' as these tests run successfully on my (slow) Windows
box.

These don't fail for me or harbormaster either, so if Simon is able to
consistently reproduce these then I'll have to ask him for a core dump so I
can take a look.

Thanks again,
I appreciate the help!

Regards,
Tamar

On Mon, Dec 3, 2018 at 3:34 PM Roland Senn  wrote:

> Hi Tamar,
>
> I looked into the testcases 'plugins09', 'plugins10' and 'plugins11' and
> found the following: GHC-Windows uses BufferMode 'BlockBuffering Nothing',
> however, GHC-Linux uses 'LineBuffering'.
>
> If I add an 'import System.IO' and the line 'liftIO $ hSetBuffering stdout
> LineBuffer' as first line in the do block of the function
> '.../testuite/tests/plugins/simple-plugin/Simple/SourcePlugin.hs:parsedPlugin'
> then all 3 tests pass successfully on Windows!
>
> I don't know anything about the "Why" and "Where" in the GHC IO module on
> Windows, so I'm unable to come up with a patch.
>
> Regards
>Roland
>
> PS: I can't say anything about the tests 'plugin-recomp-pure' and
> 'plugin-recomp-impure' as these tests run successfully on my (slow) Windows
> box.
>
> Am Sonntag, den 02.12.2018, 20:42 + schrieb Phyx:
>
> Hi Simon,
>
> That's a bit better (still need to figure out why the recent threading
> issues, but one problem at a time :) )
>
> From that list T10672_x64 is one I'm looking at already, seems to have
> something to do with the libstdc++ destructors.
> Plugins 09 and 10 are the other two I know about, but haven't had time to
> look at them yet. Frankly I know too little about plugins to make an
> accurate determination here, but the input files are empty
> yet it expects output, so I don't know what it's supposed to do here.  If
> someone who knows more about plugins can chime in that would save some time.
>
> The segfaulting plugin I haven't triaged yet. Now the remaining failures
> aside from T14452 that Roland is taking care of, seem to have to do with
> your locale in your console.  You seem to be running the
> tests in a console that has latin-1 locale? So some unicode characters
> fail encoding/decoding.
>
> If it's a Windows shell you can change it to utf-8 using "chcp 65001", if
> it's an msys2 shell, what does "locale" return?
>
> For reference mine is
>
> $ locale
> LANG=en_GB.UTF-8
> LC_CTYPE="en_GB.UTF-8"
> LC_NUMERIC="en_GB.UTF-8"
> LC_TIME="en_GB.UTF-8"
> LC_COLLATE="en_GB.UTF-8"
> LC_MONETARY="en_GB.UTF-8"
> LC_MESSAGES="en_GB.UTF-8"
> LC_ALL=
>
> If it does say latin1 you can change it with
>
> export LANG=en_GB.UTF-8
>
> This should fix more of the tests.
>
> The reason I don't mark the remaining tests as expect fail yet is because
> I haven't had the time to triage them, so I don't know their severity and
> last time there were a few nasty issues hidden in them.
>
> Unfortunately I won't have time to look at them till next weekend.
>
> Thanks,
> Tamar
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows test failures

[Phyx]

de/HEAD/testsuite
>
> cd "plugins/plugin-recomp-pure.run" && $MAKE -s --no-print-directory
> plugin-recomp-pure
>
> Timeout happened...killed process "cd "plugins/plugin-recomp-pure.run" &&
> $MAKE -s --no-print-directory plugin-recomp-pure  "...
>
>
>
> Wrong exit code for plugin-recomp-pure()(expected 0 , actual 99 )
>
> Stdout ( plugin-recomp-pure ):
>
> [1 of 1] Compiling Main ( plugin-recomp-test.hs,
> plugin-recomp-test.o )
>
> Stderr ( plugin-recomp-pure ):
>
> Simple Plugin Passes Queried
>
> Got options:
>
> Simple Plugin Pass Run
>
> *** unexpected failure for plugin-recomp-pure(normal)
>
> => plugin-recomp-impure(normal) 20 of 21 [0, 8, 1]
>
> cd "plugins/plugin-recomp-impure.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugin-recomp-impure.run" && $MAKE -s --no-print-directory
> plugin-recomp-impure
>
> Wrong exit code for plugin-recomp-impure()(expected 0 , actual 2 )
>
> Stdout ( plugin-recomp-impure ):
>
> [1 of 1] Compiling Main ( plugin-recomp-test.hs,
> plugin-recomp-test.o )
>
> Linking plugin-recomp-test.exe ...
>
> [1 of 1] Compiling Main ( plugin-recomp-test.hs,
> plugin-recomp-test.o ) [Plugin forced recompilation]
>
> Stderr ( plugin-recomp-impure ):
>
> Simple Plugin Passes Queried
>
> Got options:
>
> Simple Plugin Pass Run
>
> Simple Plugin Passes Queried
>
> Access violation in generated code when reading 0xa824
>
>
>
> Attempting to reconstruct a stack trace...
>
>
>
>Frame Code address
>
> * 0x49adab0 0x1eb49ec C:\code\HEAD\inplace\bin\ghc-stage2.exe+0x1ab49ec
>
>
>
> make[1]: *** [Makefile:99: plugin-recomp-impure] Error 11
>
> *** unexpected failure for plugin-recomp-impure(normal)
>
> => plugin-recomp-flags(normal) 21 of 21 [0, 9, 1]
>
> cd "plugins/plugin-recomp-flags.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugin-recomp-flags.run" && $MAKE -s --no-print-directory
> plugin-recomp-flags
>
> Traceback (most recent call last):
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 824, in
> test_common_work
>
> do_test(name, way, func, args, files)
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 910, in do_test
>
> result = func(*[name,way] + args)
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 993, in run_command
>
> return simple_run( name, '', override_options(cmd), '' )
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 1338, in simple_run
>
> dump_stderr(name)
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 1501, in
> dump_stderr
>
> print(str)
>
> UnicodeEncodeError: 'latin-1' codec can't encode characters in position
> 24-25: ordinal not in range(256)
>
>
>
> Unexpected results from:
>
> TEST="T10672_x64 T14452 T3307 T4006 environment001 plugin-recomp-impure
> plugin-recomp-pure plugins09 plugins10 plugins11"
>
>
>
> SUMMARY for test run started at Fri Nov 30 21:07:17 2018 GMTST
>
> 0:26:45 spent to go through
>
>   21 total tests, which gave rise to
>
>   36 test cases, of which
>
>   11 were skipped
>
>
>
>0 had missing libraries
>
>   15 expected passes
>
>0 expected failures
>
>
>
>1 caused framework failures
>
>0 caused framework warnings
>
>0 unexpected passes
>
>9 unexpected failures
>
>0 unexpected stat failures
>
>
>
> Unexpected failures:
>
>driver/T14452.run   T14452 [bad stdout] (normal)
>
>rts/T10672/T10672_x64.run   T10672_x64 [bad exit code]
> (normal)
>
>libraries/base/tests/T4006.run  T4006 [bad stdout] (normal)
>
>libraries/base/tests/IO/environment001.run  environment001 [bad stdout]
> (normal)
>
>plugins/plugins09.run   plugins09 [bad stdout]
> (normal)
>
>plugins/plugins10.run   plugins10 [bad stdout]
> (normal)
>
>plugins/plugins11.run   plugins11 [bad exit code]
> (normal)
>
>plugins/plugin-recomp-pure.run  plugin-recomp-pure [bad
> exit code] (normal)
>
>plugins/plugin-recomp-impure.runplugin-recomp-impure [bad
> exit code] (normal)
>

Re: Windows test failures

[Phyx]

y plugins11
>
> Timeout happened...killed process "cd "plugins/plugins11.run" && $MAKE -s
> --no-print-directory plugins11  "...
>
>
>
> Wrong exit code for plugins11()(expected 0 , actual 99 )
>
> *** unexpected failure for plugins11(normal)
>
> => plugins13(normal) 16 of 21 [0, 7, 1]
>
> cd "plugins/plugins13.run" && $MAKE -s --no-print-directory -C
> simple-plugin package.plugins13 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugins13.run" && $MAKE -s --no-print-directory plugins13
>
> => plugins14(normal) 17 of 21 [0, 7, 1]
>
> cd "plugins/plugins14.run" && $MAKE -s --no-print-directory -C
> simple-plugin package.plugins14 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugins14.run" && $MAKE -s --no-print-directory plugins14
>
> => T10420(normal) 18 of 21 [0, 7, 1]
>
> cd "plugins/T10420.run" && $MAKE -s --no-print-directory -C
> rule-defining-plugin package.T10420 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/T10420.run" && $MAKE -s --no-print-directory T10420
>
> => plugin-recomp-pure(normal) 19 of 21 [0, 7, 1]
>
> cd "plugins/plugin-recomp-pure.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugin-recomp-pure.run" && $MAKE -s --no-print-directory
> plugin-recomp-pure
>
> Timeout happened...killed process "cd "plugins/plugin-recomp-pure.run" &&
> $MAKE -s --no-print-directory plugin-recomp-pure  "...
>
>
>
> Wrong exit code for plugin-recomp-pure()(expected 0 , actual 99 )
>
> Stdout ( plugin-recomp-pure ):
>
> [1 of 1] Compiling Main ( plugin-recomp-test.hs,
> plugin-recomp-test.o )
>
> Stderr ( plugin-recomp-pure ):
>
> Simple Plugin Passes Queried
>
> Got options:
>
> Simple Plugin Pass Run
>
> *** unexpected failure for plugin-recomp-pure(normal)
>
> => plugin-recomp-impure(normal) 20 of 21 [0, 8, 1]
>
> cd "plugins/plugin-recomp-impure.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugin-recomp-impure.run" && $MAKE -s --no-print-directory
> plugin-recomp-impure
>
> Wrong exit code for plugin-recomp-impure()(expected 0 , actual 2 )
>
> Stdout ( plugin-recomp-impure ):
>
> [1 of 1] Compiling Main ( plugin-recomp-test.hs,
> plugin-recomp-test.o )
>
> Linking plugin-recomp-test.exe ...
>
> [1 of 1] Compiling Main ( plugin-recomp-test.hs,
> plugin-recomp-test.o ) [Plugin forced recompilation]
>
> Stderr ( plugin-recomp-impure ):
>
> Simple Plugin Passes Queried
>
> Got options:
>
> Simple Plugin Pass Run
>
> Simple Plugin Passes Queried
>
> Access violation in generated code when reading 0xa824
>
>
>
> Attempting to reconstruct a stack trace...
>
>
>
>Frame Code address
>
> * 0x49adab0 0x1eb49ec C:\code\HEAD\inplace\bin\ghc-stage2.exe+0x1ab49ec
>
>
>
> make[1]: *** [Makefile:99: plugin-recomp-impure] Error 11
>
> *** unexpected failure for plugin-recomp-impure(normal)
>
> => plugin-recomp-flags(normal) 21 of 21 [0, 9, 1]
>
> cd "plugins/plugin-recomp-flags.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite
>
> cd "plugins/plugin-recomp-flags.run" && $MAKE -s --no-print-directory
> plugin-recomp-flags
>
> Traceback (most recent call last):
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 824, in
> test_common_work
>
> do_test(name, way, func, args, files)
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 910, in do_test
>
> result = func(*[name,way] + args)
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 993, in run_command
>
> return simple_run( name, '', override_options(cmd), '' )
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 1338, in simple_run
>
> dump_stderr(name)
>
>   File "/c/code/HEAD/testsuite/driver/testlib.py", line 1501, in
> dump_stderr
>
> print(str)
>
> UnicodeEncodeError: 'latin-1' codec can't encode characters in position
> 24-25: ordinal not in range(256)
>
>
>
> Unexpected results from:
>
> TEST="T10672_x64 T14452 T3307 T4006 environment001 plugin-recomp-impure
> plugin-recomp-pure plugins09 plugins10 plugins11"
>
>
>
> SUMMARY for test run started at Fri Nov 30 21:07:17 2

Re: Windows test failures

[Phyx]

Hi Simon,

Could you try rerunning those failing tests with make TEST? No threads? I
have been trying to clean up the testsuite and there should be only 3
failing tests, two plugin tests with bad stdout and T10672_x64 which I am
looking into.

I have however noticed that the testsuite has become increasingly unstable
under threads for some reason.

I also haven't run trunk yet since last week so they could be new..

But rerunning the fails with no threads should give a better idea.

Regards,
Tamar

On Fri, Nov 30, 2018, 07:46 Simon Peyton Jones via ghc-devs <
ghc-devs@haskell.org> wrote:

> Ben, Phyx, and other CI folk
>
> ‘sh validate –fast’ still gives a lot of failures on Window.  The output
> is below. I would be so good to get this to zero!
>
> (another minor thing is that it would be great to eliminate the long path
> at the beginning – this is platform independent – I have to delete manually
> many times each day.
>
> Thanks
>
> Simon
>
> Unexpected passes:
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/rts/T9405.run  T9405 [unexpected] (normal)
>
>
>
> Unexpected failures:
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/driver/T14452.run   T14452 [bad stdout]
> (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci/linking/ghcilink001.runghcilink001 [bad exit
> code] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci/prog010/ghci.prog010.run   ghci.prog010 [bad exit
> code] (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci/scripts/ghci050.runghci050 [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci/should_run/T14963a.run T14963a [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci.debugger/scripts/print012.run  print012 [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci.debugger/scripts/print016.run  print016 [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci.debugger/scripts/break019.run  break019 [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci.debugger/scripts/break028.run  break028 [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad exit
> code] (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci.debugger/scripts/T2740.run T2740 [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/indexed-types/should_fail/T7102a.runT7102a [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/plugins/plugin-recomp-change.runplugin-recomp-change
> [bad exit code] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/rts/T10672/T10672_x64.run   T10672_x64 [bad exit
> code] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/th/TH_spliceGuard.run   TH_spliceGuard [exit
> code non-0] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/th/TH_overlaps.run  TH_overlaps [exit code
> non-0] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/th/T5886.runT5886 [exit code non-0]
> (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/th/T10697_decided_3.run T10697_decided_3 [exit
> code non-0] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/th/T11629.run   T11629 [exit code non-0]
> (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/libraries/base/tests/T4006.run  T4006 [bad stdout]
> (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/libraries/base/tests/IO/environment001.run  environment001 [bad
> stdout] (normal)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/ghci/should_run/T15633b.run T15633b [bad exit code]
> (ghci)
>
>/c/Users/simonpj/AppData/Local/Temp/ghctest-i30xogs3/test
> spaces/plugins/plugins07.run   plugins07 [bad

Re: split_marker crash

[Phyx]

On Mon, Nov 19, 2018 at 8:53 PM Simon Peyton Jones 
wrote:

> I do indeed have SplitObs=YES in my validate.mk.
>
> In the past at least, that made binaries quite a bit smaller.
>
> Happy to change it to SplitSections=YES if that is more kosher these days?
>

Yes, SplitObjs is no longer supported, will also cause an extremely slow
compilation depending on your optimization level and ways.
Removing the line should let the compiler default to the best choice, in
this case it should go to SplitSections on its
own when compiling stage2.

Regards,
Tamar


>
> Simon
>
>
>
> *From:* Andreas Klebinger 
> *Sent:* 19 November 2018 20:38
> *To:* Ben Gamari 
> *Cc:* Phyx ; Simon Peyton Jones <
> simo...@microsoft.com>; ghc-devs@haskell.org
> *Subject:* Re: split_marker crash
>
>
>
> I might already have found the specific cause.
>
> It seems with split sections we generate a dummy CmmProc, which has as
> entry label
> > error (split_sections_entry)
>
> My patch likely introduces strictness on that field where it was lazy
> before.
> If this is the cause I expect to have a patch up in a hour or two and will
> merge
> it after it validates.
>
> Cheers,
> Andreas
>
> Ben Gamari schrieb:
>
> Andreas Klebinger   
> writes:
>
>
>
> Hello,
>
>
>
> I'm fine with reverting for now. But could you give me a way to
>
> reproduce this error?
>
>
>
> I've not seen crashes on either linux and windows in various configs.
>
>
>
> I suspect that Simon is building with SplitObjects enabled. To be
>
> honest, I would really like to remove this feature; SplitSections is
>
> better in nearly every regard. However, we have been stalled since
>
> SplitSections doesn't quite work yet on Windows (or, IIRC, the toolchain
>
> is prohibitively slow when it's used). I believe Tamar was working on
>
> fixing this.
>
>
>
> Cheers,
>
>
>
> - Ben
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: split_marker crash

[Phyx]

On Mon, Nov 19, 2018 at 8:33 PM Ben Gamari  wrote:

> Andreas Klebinger  writes:
>
> > Hello,
> >
> > I'm fine with reverting for now. But could you give me a way to
> > reproduce this error?
> >
> > I've not seen crashes on either linux and windows in various configs.
> >
> I suspect that Simon is building with SplitObjects enabled. To be
> honest, I would really like to remove this feature; SplitSections is
> better in nearly every regard. However, we have been stalled since
> SplitSections doesn't quite work yet on Windows (or, IIRC, the toolchain
> is prohibitively slow when it's used). I believe Tamar was working on
> fixing this.
>

SplitObjects has been off in favor of SplitSections on Windows since Aug.
https://github.com/ghc/ghc/commit/23774c98f1368b41515cbd5223b87ea6dbf644e1

SplitObjects is not usable on Windows anymore so it was disabled
https://ghc.haskell.org/trac/ghc/ticket/15051


> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 8.6.1 released

[Phyx]

Hi All,

I've made a ticket for this but it seems it hasn't gotten any attention at
all. As it stands now the 8.6.1 tarballs for Windows are a bit broken.
Because of a mistake I've made during the mapping of the ACLs from fopen to
CreateFile it's accidentally asking for WRITE attributes rights when
opening a read-only file.
Note that this is slightly different from WRITE permissions on the file
itself. So reading a read-only file works fine, as long as you're the owner
or have sufficient right to modify the metadata.
This is why the CI did not catch it.  The CI cannot create a file which
it's not an owner off, or for which it doesn't have permissions to remove
the file (it would get stuck).

The only way to catch this is to run GHC from a privileged location such as
how chocolatey installs it or how Haskell Platform would. Essentially this
means no GHC on chocolatey or HP
can run without you being an admin or the owner of the location it was
installed to, and the same applies to any binaries produced by this GHC.
This probably will prevent HP builds for it.

The ticket is here https://ghc.haskell.org/trac/ghc/ticket/15667 and the
patch has been sitting at https://phabricator.haskell.org/D5177

I'll modify my chocolatey packages to actually run the GHC after installing
it as a post install step. This should catch such errors in the future
during betas.

Thanks,
Tamar

On Mon, Sep 24, 2018 at 2:37 PM Ben Gamari  wrote:

>
>
> On September 24, 2018 2:09:13 AM CDT, Jens Petersen 
> wrote:
> >I have built 8.6.1 for Fedora 27, 28, 29, Rawhide, and EPEL7 in:
> >
> >https://copr.fedorainfracloud.org/coprs/petersen/ghc-8.6.1/
> >
> >The repo also includes latest cabal-install.
> >
> Thanks Jens! This is a very helpful service.
>
> Cheers,
>
> - Ben
>
>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows testsuite failures

[Phyx]

Hi Simon,

Sure I'll do that. I'll create the patches later tonight.

Kind regards,
Tamar

On Mon, Sep 24, 2018, 13:35 Simon Peyton Jones 
wrote:

> Tamar
>
>
>
> Thank you, that’s great!
>
>
>
> For the ones that need more work, can we mark them as expect-broken, so
> that they don’t pollute the testsuite output?
>
>
> Simon
>
>
>
>
>
> *From:* loneti...@gmail.com 
> *Sent:* 24 September 2018 07:13
>
>
> *To:* Simon Peyton Jones 
> *Subject:* RE: Windows testsuite failures
>
>
>
>
>
> Hi Simon,
>
>
>
> I created some patches to fix the majority of these
>
>
>
> https://phabricator.haskell.org/D5174
>
> https://phabricator.haskell.org/D5175
>
> https://phabricator.haskell.org/D5176
>
>
>
> The remaining ones I’ve either pinged the patches that caused the issues
> or created tickets
>
> For them because they’re actual bugs that require a bit more time to find
> the cause and fix.
>
>
>
> https://ghc.haskell.org/trac/ghc/ticket/15668
>
> https://ghc.haskell.org/trac/ghc/ticket/15669
>
> https://ghc.haskell.org/trac/ghc/ticket/15670
>
> https://ghc.haskell.org/trac/ghc/ticket/15671
>
>
>
> these should bring down the amount of failing tests to about 5.
>
>
>
> Kind Regards,
>
> Tamar
>
>
>
> *From: *Simon Peyton Jones 
> *Sent: *Thursday, September 20, 2018 12:05
> *To: *Phyx 
> *Subject: *RE: Windows testsuite failures
>
>
>
> Thanks Tamar.  I’ll look forward to hearing back
>
>
>
> S
>
>
>
> *From:* Phyx 
> *Sent:* 20 September 2018 12:02
> *To:* Simon Peyton Jones 
> *Cc:* ghc-devs@haskell.org
> *Subject:* Re: Windows testsuite failures
>
>
>
> Hi Simon,
>
>
>
> Thanks for the email. I haven't been building head much as I'm working on
> top of some older commits. From a quick look it seems like the plugin ones
> are probably testisms, the plugins aren't found so likely a missing path
> entry somewhere.
>
>
>
> The linker ones are are weird, I'll need to take a closer look at those,
> likely culprit is my recent patch, I had been testing in the 32 bit build
> and didn't notice these.
>
>
>
> There are a few worrying segfault that shouldn't be there on some random
> tests so I'll take a closer look at those too.
>
>
>
> And the stat changes need to be updated.
>
>
>
> The framework failures I don't see on harbormaster, so think they are
> again a threading artifact. Need to figure out a more effective way to
> debug these to find a permanent fix. The ones that harbormaster does see
> are encoding related. touch is failing on non-ascii names.
>
>
>
> I will take a look this weekend.
>
>
>
> Kind regards,
>
> Tamar
>
>
>
> On Thu, Sep 20, 2018, 11:33 Simon Peyton Jones 
> wrote:
>
> Hi Tamar
>
> The list of testsuite failure on Windows has grown quite long – see
> below.  Most seem to concern plugins or linking.
>
> Do you know what is going on here?  If they can’t be fixed, can we mark
> them as expect_broken on Windows, so that it’s easier (when developing) to
> know when I’ve introduced a regression.  Currently I have to do a manual
> diff against a rather long list.
>
> Thanks!
>
> Simon
>
>
>
> *SUMMARY for test run started at Thu Sep 20 00:13:20 2018 GMTST*
>
> *1:03:11 spent to go through*
>
> *6530 total tests, which gave rise to*
>
> *   18728 test cases, of which*
>
> *   12206 were skipped*
>
>
>
> *  33 had missing libraries*
>
> *6278 expected passes*
>
> * 173 expected failures*
>
>
>
> *   9 caused framework failures*
>
> *   1 caused framework warnings*
>
> *   0 unexpected passes*
>
> *  31 unexpected failures*
>
> *   7 unexpected stat failures*
>
>
>
> *Unexpected failures:*
>
> *   ghci/linking/dyn/T10955dyn.run  T10955dyn [bad exit code]
> (normal)*
>
> *   ghci/linking/dyn/T10955.run T10955 [bad stderr] (ghci)*
>
> *   ghci/linking/dyn/T11072gcc.run  T11072gcc [bad exit code]
> (normal)*
>
> *   numeric/should_run/FloatFnInverses.run  FloatFnInverses [bad stdout]
> (normal)*
>
> *   plugins/T11244.run  T11244 [bad stderr] (normal)*
>
> *   plugins/plugin-recomp-change.runplugin-recomp-change [bad exit
> code] (normal)*
>
> *   rts/T7040_ghci.run  T7040_ghci [bad stdout] (ghci)*
>
> *   rts/linker_unload.run   linker_unload [bad exit code]
> (normal)*
>
> *   rts/linker_error1.run   linker_error1 [bad exit 

Re: constant pooling in GHC

[Phyx]

I simply mean that the same constant is not emitted more than once to
memory. I.e. If a constant string is used twice, there is only one value in
.rodata for it.

On Fri, Sep 21, 2018, 01:20 Carter Schonwald 
wrote:

> what precisely do you mean by constant pooling? i can guess several
> meanings, but what do you mean specifically?
>
> On Sat, Sep 15, 2018 at 10:33 AM Phyx  wrote:
>
>> Hi All,
>>
>> I'm hoping someone here can save me some time.  I'm working on something
>> that relies on constants in the same translation unit being pooled before
>> assembly.
>>
>> I've noticed GHC does some const pooling at -O1 , but this doesn't seem
>> to be very consistent, which leads me to believe this is a byproduct of
>> another optimization rather than an explicit thing.
>>
>> I couldn't find anything obvious in the sources, so does GHC
>> intentionally do constant pooling already? Or do I need a new pass to
>> guarantee it.
>>
>> Thanks,
>> Tamar
>>
> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows testsuite failures

[Phyx]

Hi Simon,

Thanks for the email. I haven't been building head much as I'm working on
top of some older commits. From a quick look it seems like the plugin ones
are probably testisms, the plugins aren't found so likely a missing path
entry somewhere.

The linker ones are are weird, I'll need to take a closer look at those,
likely culprit is my recent patch, I had been testing in the 32 bit build
and didn't notice these.

There are a few worrying segfault that shouldn't be there on some random
tests so I'll take a closer look at those too.

And the stat changes need to be updated.

The framework failures I don't see on harbormaster, so think they are again
a threading artifact. Need to figure out a more effective way to debug
these to find a permanent fix. The ones that harbormaster does see are
encoding related. touch is failing on non-ascii names.

I will take a look this weekend.

Kind regards,
Tamar

On Thu, Sep 20, 2018, 11:33 Simon Peyton Jones 
wrote:

> Hi Tamar
>
> The list of testsuite failure on Windows has grown quite long – see
> below.  Most seem to concern plugins or linking.
>
> Do you know what is going on here?  If they can’t be fixed, can we mark
> them as expect_broken on Windows, so that it’s easier (when developing) to
> know when I’ve introduced a regression.  Currently I have to do a manual
> diff against a rather long list.
>
> Thanks!
>
> Simon
>
>
>
> *SUMMARY for test run started at Thu Sep 20 00:13:20 2018 GMTST*
>
> *1:03:11 spent to go through*
>
> *6530 total tests, which gave rise to*
>
> *   18728 test cases, of which*
>
> *   12206 were skipped*
>
>
>
> *  33 had missing libraries*
>
> *6278 expected passes*
>
> * 173 expected failures*
>
>
>
> *   9 caused framework failures*
>
> *   1 caused framework warnings*
>
> *   0 unexpected passes*
>
> *  31 unexpected failures*
>
> *   7 unexpected stat failures*
>
>
>
> *Unexpected failures:*
>
> *   ghci/linking/dyn/T10955dyn.run  T10955dyn [bad exit code]
> (normal)*
>
> *   ghci/linking/dyn/T10955.run T10955 [bad stderr] (ghci)*
>
> *   ghci/linking/dyn/T11072gcc.run  T11072gcc [bad exit code]
> (normal)*
>
> *   numeric/should_run/FloatFnInverses.run  FloatFnInverses [bad stdout]
> (normal)*
>
> *   plugins/T11244.run  T11244 [bad stderr] (normal)*
>
> *   plugins/plugin-recomp-change.runplugin-recomp-change [bad exit
> code] (normal)*
>
> *   rts/T7040_ghci.run  T7040_ghci [bad stdout] (ghci)*
>
> *   rts/linker_unload.run   linker_unload [bad exit code]
> (normal)*
>
> *   rts/linker_error1.run   linker_error1 [bad exit code]
> (normal)*
>
> *   rts/linker_error2.run   linker_error2 [bad exit code]
> (normal)*
>
> *   rts/T12771/T12771.run   T12771 [bad exit code]
> (normal)*
>
> *   rts/T13082/T13082_good.run  T13082_good [bad exit code]
> (normal)*
>
> *   rts/T14611/T14611.run   T14611 [bad exit code]
> (normal)*
>
> *   simplCore/should_compile/T7702.run  T7702 [exit code non-0]
> (normal)*
>
> *   rts/T10672/T10672_x64.run   T10672_x64 [bad exit code]
> (normal)*
>
> *   libraries/Win32/tests/T4452.run T4452 [bad exit code] (normal)*
>
> *   plugins/plugins01.run   plugins01 [bad exit code]
> (normal)*
>
> *   plugins/plugins07.run   plugins07 [bad exit code]
> (normal)*
>
> *   plugins/plugins09.run   plugins09 [bad exit code]
> (normal)*
>
> *   plugins/plugins11.run   plugins11 [bad exit code]
> (normal)*
>
> *   plugins/plugins12.run   plugins12 [bad exit code]
> (normal)*
>
> *   plugins/plugins13.run   plugins13 [bad exit code]
> (normal)*
>
> *   plugins/plugins14.run   plugins14 [bad exit code]
> (normal)*
>
> *   plugins/plugins15.run   plugins15 [bad exit code]
> (normal)*
>
> *   plugins/T10420.run  T10420 [bad exit code]
> (normal)*
>
> *   plugins/T10294.run  T10294 [bad exit code]
> (normal)*
>
> *   plugins/T10294a.run T10294a [bad exit code]
> (normal)*
>
> *   plugins/T12567a.run T12567a [bad exit code]
> (normal)*
>
> *   plugins/plugin-recomp-pure.run  plugin-recomp-pure [bad exit
> code] (normal)*
>
> *   plugins/plugin-recomp-impure.runplugin-recomp-impure [bad exit
> code] (normal)*
>
> *   plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit
> code] (normal)*
>
>
>
> *Unexpected stat failures:*
>
> *   perf/compiler/T9872d.run T9872d [stat not good enough]
> (normal)*
>
> *   perf/compiler/T12425.run T12425 [stat not good enough]
> (optasm)*
>
> *   perf/compiler/T12234.run T12234 [stat not good enough]
> (optasm)*
>
> *   perf/compiler/T12150.run T12150 [stat not good enough]
> 

constant pooling in GHC

[Phyx]

Hi All,

I'm hoping someone here can save me some time.  I'm working on something
that relies on constants in the same translation unit being pooled before
assembly.

I've noticed GHC does some const pooling at -O1 , but this doesn't seem to
be very consistent, which leads me to believe this is a byproduct of
another optimization rather than an explicit thing.

I couldn't find anything obvious in the sources, so does GHC intentionally
do constant pooling already? Or do I need a new pass to guarantee it.

Thanks,
Tamar
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Haskell DLL (using FFI) causes memory leak.

[Phyx]

Oh and also, neither of those tickets you linked to should prevent you from
using a newer GHC to make shared libraries.

the Rts.h headers are nothing but convenience functions and you can just
create the prototypes you need in your own headers.

On Mon, Aug 27, 2018 at 5:46 PM Phyx  wrote:

> Hi,
>
> You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't
> designed to be unload-able, which is why you can't call hs_init again after
> you stop.
>
> The leak happens only when you do a foreign export because foreign exports
> create C wrappers to initializers for each function you export.
>
> https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668
>
> These are marked as static initializers, and as such the CRT will
> initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH
> time the destructors would be run, but we don't have destructors for these
> initializers, so whatever they did will always persist.
>
> Your use case is fairly uncommon, but file a bug report against the leaks
> anyway. Why do you need to keep loading and unloading the dll?
>
> Note that FreeLibrary like dlclose on unix does not guarantee that the
> library is freed, it just decrements the reference count. and when it
> reaches 0 it will *eventually* unload it. Notice that if you use
> UnmapViewOfFile instead of FreeLibrary the leak doesn't happen as that
> *actually* immediately unmaps the image.  But this will get you in all
> sorts of trouble later on when terminating (you'll likely segfault at that
> point).
>
> Things will get worse when you actually do call hs_init, because hs_init
> will call initLinker which will load libraries of it's own. Unloading the
> top level
> does not decrease the reference counts of the dependencies, and we do no
> extra work to ensure this.
>
> If you really need to load/reload libraries, then I fear your only choice
> is some kind of process isolation and communicating back using share
> memory/pipes
> or some sort of IPC.
>
> Regards,
> Tamar
>
> On Mon, Aug 27, 2018 at 4:33 PM Ольга Филиппская 
> wrote:
>
>> It's true that one must initialize the runtime when calling Haskell from
>> C/C++. We do this in the real project and just a pair of calls
>> hs_init/hs_exit causes memory to be leaked even faster. But I tried to
>> construct the minimal example to describe the bug, so I reduced all the
>> extra-code.
>>
>> пн, 27 авг. 2018 г. в 18:00, Vanessa McHale :
>>
>>> I don't know about C++, but I do know that when calling Haskell from C
>>> code one must initialize the runtime and then exit it once finished. I also
>>> know that one must only initialize the runtime once over the course of the
>>> program's run. As far as I can tell, this does not occur in your example.
>>>
>>> On 08/27/2018 09:44 AM, Ольга Филиппская wrote:
>>>
>>> Hello.
>>>
>>> *Summary: *memory is consumed without releasing when a Haskell DLL
>>> (that uses FFI) is loaded and unloaded in an endless loop.
>>>
>>> *Details*: I'm working on a C++ project relying on a DLL that uses
>>> Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer
>>> versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due
>>> to some bugs: ticket #1
>>> <https://ghc.haskell.org/trac/ghc/ticket/14472#no2>, ticker #2
>>> <https://ghc.haskell.org/trac/ghc/ticket/14784#no1>). The C++ project
>>> was built with MSVC compiler 2015.
>>>
>>> We noticed that memory is not released when the library is unloaded.
>>> I've constructed a baseline example to reproduce the bug. It has three
>>> files: the first one is *HaskellSources.hs*(see attachments). It
>>> contains one foreign export function foo. This foreign export function is
>>> necessary to reproduce the bug. The second file is *CWrapper.cpp*. I
>>> intentionally didn't include any Haskell function export because they make
>>> no difference for this example. The last file is the code of the main
>>> program (see *main.cpp*) that loads the library and unloads it at once
>>> in an endless loop.
>>>
>>> There are two cases: whether HaskellExports.o is included into the
>>> library or not.
>>> 1. HaskellExports.o is not included into the library (see attached build
>>> script* build_no_hs.sh*). Everything works just fine (i.e. amount of
>>> memory consumed by the app is the same before DLL loading and right after
>>> unloading.)
>>> 2. HaskellExports.o is included into the library (

Re: Haskell DLL (using FFI) causes memory leak.

[Phyx]

Hi,

You're likely just hitting a memory leak, Haskell DLLs and the RTS aren't
designed to be unload-able, which is why you can't call hs_init again after
you stop.

The leak happens only when you do a foreign export because foreign exports
create C wrappers to initializers for each function you export.
https://github.com/ghc/ghc/blob/21f0f56164f50844c2150c62f950983b2376f8b6/compiler/deSugar/DsForeign.hs#L668

These are marked as static initializers, and as such the CRT will
initialize them on DLL_PROCESS_ATTACH time. At DLL_PROCESS_DETACH
time the destructors would be run, but we don't have destructors for these
initializers, so whatever they did will always persist.

Your use case is fairly uncommon, but file a bug report against the leaks
anyway. Why do you need to keep loading and unloading the dll?

Note that FreeLibrary like dlclose on unix does not guarantee that the
library is freed, it just decrements the reference count. and when it
reaches 0 it will *eventually* unload it. Notice that if you use
UnmapViewOfFile instead of FreeLibrary the leak doesn't happen as that
*actually* immediately unmaps the image.  But this will get you in all
sorts of trouble later on when terminating (you'll likely segfault at that
point).

Things will get worse when you actually do call hs_init, because hs_init
will call initLinker which will load libraries of it's own. Unloading the
top level
does not decrease the reference counts of the dependencies, and we do no
extra work to ensure this.

If you really need to load/reload libraries, then I fear your only choice
is some kind of process isolation and communicating back using share
memory/pipes
or some sort of IPC.

Regards,
Tamar

On Mon, Aug 27, 2018 at 4:33 PM Ольга Филиппская 
wrote:

> It's true that one must initialize the runtime when calling Haskell from
> C/C++. We do this in the real project and just a pair of calls
> hs_init/hs_exit causes memory to be leaked even faster. But I tried to
> construct the minimal example to describe the bug, so I reduced all the
> extra-code.
>
> пн, 27 авг. 2018 г. в 18:00, Vanessa McHale :
>
>> I don't know about C++, but I do know that when calling Haskell from C
>> code one must initialize the runtime and then exit it once finished. I also
>> know that one must only initialize the runtime once over the course of the
>> program's run. As far as I can tell, this does not occur in your example.
>>
>> On 08/27/2018 09:44 AM, Ольга Филиппская wrote:
>>
>> Hello.
>>
>> *Summary: *memory is consumed without releasing when a Haskell DLL (that
>> uses FFI) is loaded and unloaded in an endless loop.
>>
>> *Details*: I'm working on a C++ project relying on a DLL that uses
>> Haskell code. The DLL was built with GHC 8.0.1 x64, OS is Windows 7. (Newer
>> versions of GHC - 8.2.1 and later - do not allow you to build such DLLs due
>> to some bugs: ticket #1
>> , ticker #2
>> ). The C++ project
>> was built with MSVC compiler 2015.
>>
>> We noticed that memory is not released when the library is unloaded. I've
>> constructed a baseline example to reproduce the bug. It has three files:
>> the first one is *HaskellSources.hs*(see attachments). It contains one
>> foreign export function foo. This foreign export function is necessary to
>> reproduce the bug. The second file is *CWrapper.cpp*. I intentionally
>> didn't include any Haskell function export because they make no difference
>> for this example. The last file is the code of the main program (see
>> *main.cpp*) that loads the library and unloads it at once in an endless
>> loop.
>>
>> There are two cases: whether HaskellExports.o is included into the
>> library or not.
>> 1. HaskellExports.o is not included into the library (see attached build
>> script* build_no_hs.sh*). Everything works just fine (i.e. amount of
>> memory consumed by the app is the same before DLL loading and right after
>> unloading.)
>> 2. HaskellExports.o is included into the library (see *build_w_hs.sh*).
>> Memory is not released after unloading the DLL, after repeated load/unload
>> cycles memory consumption keeps growing until finally the application
>> crashes.
>>
>> Is this a known problem? Is it tracked in your bugtracker? (Or maybe it
>> is not a problem at all and I'm doing it wrong).
>>
>> Thank you for your time.
>>
>> --
>> * Olga Philippskaya .*
>>
>>
>> ___
>> ghc-devs mailing 
>> listghc-devs@haskell.orghttp://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
> --
> *Филиппская Ольга.*
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>

Re: [ANNOUNCE] GHC 8.6.1-beta1 available

[Phyx]

Hi Ben,

I forgot to update the changelog at the time I think, It should probably be
stated that this release fixes the 32 bit Windows segfaults.

Cheers,
Tamar

On Sat, Aug 11, 2018 at 3:31 AM, Ben Gamari  wrote:

>
> Hello everyone,
>
> The GHC development team is very pleased to announce the first beta
> leading up to GHC 8.6.1 release. The usual release artifacts are
> available from
>
> https://downloads.haskell.org/~ghc/8.6.1-beta1
>
> This beta fixes most of the bugs reported in the first two alphas and
> brings all of the core libraries up to their final release versions.
>
> The 8.6 release fixes over 300 bugs from the 8.4 series and introduces a
> number of exciting features. These most notably include:
>
>  * Significantly better handling of macOS linker command size limits,
>avoiding linker errors while linking large projects
>
>  * A new deriving mechanism, `deriving via`, providing a convenient way
>for users to extend Haskell's typeclass deriving mechanism
>
>  * Quantified constraints, allowing forall quantification in contexts
>
>  * An early version of the GHCi `:doc` command
>
>  * The `ghc-heap-view` package, allowing introspection into the
>structure of GHC's heap
>
>  * Valid hole fit hints, helping the user to find terms to fill typed
>holes in their programs
>
>  * The BlockArguments extension, allowing the `$` operator to be omitted
>in some unambiguous contexts
>
>  * The next phase of the MonadFail proposal, enabling
>-XMonadFailDesugaring by default
>
> A full list of the changes in this release can be found in the
> release notes:
>
> https://downloads.haskell.org/~ghc/8.6.1-beta1/docs/html/
> users_guide/8.6.1-notes.html
>
> This will very likely be the last release before the final 8.6.1 so do
> give it a thorough testing and, as always, report any issues you
> encounter. Thanks for your help!
>
> Cheers,
>
> - Ben
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Harbormaster: Build failure on OS/X build. How to fix it as a non Mac user

[Phyx]

Hi,

They are stats failures. Not good enough, which means you have a regression
in memory usage. Look at the full test log
https://phabricator.haskell.org/harbormaster/build/48354/1/?l=100

They also failed on windows, but currently harbormaster doesn't fail when
tests fail for windows. Clicking the build log will show you they failed on
windows too.

Regards,
Tamar

On Fri, Jun 29, 2018, 16:35 Roland Senn  wrote:

> Hi all,
>
> Phabricator informed me, that the builds on Harbourmaster for my
> changes failed.
>
> Looking at the log at https://phabricator.haskell.org/B21368 I can see,
> that the builds on Linux and Windows succeeded, however the tests on
> OS/X failed.
>
> Looking at the failed tests at https://phabricator.haskell.org/harborma
> ster/build/48354/
>  it tells me
> that the 2 tests TEST="T5631 T6048"
> failed.
>
> Now, I don't own a machine with OS/X. How can I look, what really
> failed, and how can I try to fix it?
>
> Many thanks
>Roland
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Strace

[Phyx]

0]*
>
> *cd "./plugins13.run" && $MAKE -s --no-print-directory -C simple-plugin
> package.plugins13 TOP=/c/code/HEAD/testsuite*
>
> *cd "./plugins13.run" && $MAKE -s --no-print-directory plugins13  *
>
> *=> plugins14(normal) 14 of 25 [0, 2, 0]*
>
> *cd "./plugins14.run" && $MAKE -s --no-print-directory -C simple-plugin
> package.plugins14 TOP=/c/code/HEAD/testsuite*
>
> *cd "./plugins14.run" && $MAKE -s --no-print-directory plugins14  *
>
> *=> plugins15(normal) 15 of 25 [0, 2, 0]*
>
> *cd "./plugins15.run" && $MAKE -s --no-print-directory -C simple-plugin
> package.plugins15 TOP=/c/code/HEAD/testsuite*
>
> *cd "./plugins15.run" && $MAKE -s --no-print-directory plugins15  *
>
> *=> T10420(normal) 16 of 25 [0, 2, 0]*
>
> *cd "./T10420.run" && $MAKE -s --no-print-directory -C
> rule-defining-plugin package.T10420 TOP=/c/code/HEAD/testsuite*
>
> *cd "./T10420.run" && $MAKE -s --no-print-directory T10420  *
>
> *=> T10294(normal) 17 of 25 [0, 2, 0]*
>
> *cd "./T10294.run" && $MAKE -s --no-print-directory -C annotation-plugin
> package.T10294 TOP=/c/code/HEAD/testsuite*
>
> *cd "./T10294.run" && $MAKE -s --no-print-directory T10294  *
>
> *=> T10294a(normal) 18 of 25 [0, 2, 0]*
>
> *cd "./T10294a.run" && $MAKE -s --no-print-directory -C annotation-plugin
> package.T10294a TOP=/c/code/HEAD/testsuite*
>
> *cd "./T10294a.run" && $MAKE -s --no-print-directory T10294a  *
>
> *=> frontend01(normal) 19 of 25 [0, 2, 0]*
>
> *cd "./frontend01.run" && $MAKE -s --no-print-directory frontend01  *
>
> *=> T11244(normal) 20 of 25 [0, 2, 0]*
>
> *cd "./T11244.run" && $MAKE -s --no-print-directory -C
> rule-defining-plugin package.T11244 TOP=/c/code/HEAD/testsuite*
>
> *cd "./T11244.run" && $MAKE -s --no-print-directory T11244  *
>
> *Actual stderr output differs from expected:*
>
> *diff -uw "./T11244.run/T11244.stderr.normalised"
> "./T11244.run/T11244.run.stderr.normalised"*
>
>  unexpected failure for T11244(normal)*
>
> *=> T12567a(normal) 21 of 25 [0, 3, 0]*
>
> *cd "./T12567a.run" && $MAKE -s --no-print-directory -C simple-plugin
> package.T12567a TOP=/c/code/HEAD/testsuite*
>
> *cd "./T12567a.run" && $MAKE -s --no-print-directory T12567a  *
>
> *=> T14335(normal) 22 of 25 [0, 3, 0]*
>
> *cd "./T14335.run" && $MAKE -s --no-print-directory -C simple-plugin
> package.plugins01 TOP=/c/code/HEAD/testsuite*
>
> *cd "./T14335.run" &&  "/c/code/HEAD/inplace/bin/ghc-stage2.exe" -c
> T14335.hs -dcore-lint -dcmm-lint -no-user-package-db -rtsopts
> -fno-warn-missed-specialisations -fshow-warning-groups
> -fdiagnostics-color=never -fno-diagnostics-show-caret -dno-debug-output
>  -package-db simple-plugin/pkg.plugins01/local.package.conf -fplugin
> Simple.Plugin-fexternal-interpreter -package simple-plugin -static*
>
> *=> plugin-recomp-pure(normal) 23 of 25 [0, 3, 0]*
>
> *cd "./plugin-recomp-pure.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite*
>
> *cd "./plugin-recomp-pure.run" && $MAKE -s --no-print-directory
> plugin-recomp-pure  *
>
> *=> plugin-recomp-impure(normal) 24 of 25 [0, 3, 0]*
>
> *cd "./plugin-recomp-impure.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite*
>
> *cd "./plugin-recomp-impure.run" && $MAKE -s --no-print-directory
> plugin-recomp-impure  *
>
> *=> plugin-recomp-flags(normal) 25 of 25 [0, 3, 0]*
>
> *cd "./plugin-recomp-flags.run" && $MAKE -s --no-print-directory -C
> plugin-recomp package.plugins01 TOP=/c/code/HEAD/testsuite*
>
> *cd "./plugin-recomp-flags.run" && $MAKE -s --no-print-directory
> plugin-recomp-flags  *
>
>
>
> *Unexpected results from:*
>
> *TEST="T11244 plugins09 plugins11"*
>
>
>
> *SUMMARY for test run started at Mon Jun 18 11:24:57 2018 GMTST*
>
> *0:11:18 spent to go through*
>
> *  25 total tests, which gave rise to*
>
> *  35 test cases, of which*
>
> *  11 were skipped*
>
>
>
> *   0 had missing libraries*
>
> *  19 expected passes*
>
> *   2 expected failures*
>
>
>
> *   0 caused framew

Re: Strace

[Phyx]

Hi Simon,

On Wed, Jun 13, 2018 at 5:24 PM, Simon Peyton Jones 
wrote:

> OK – so maybe the root cause is a framework failure – and indeed for the
> last few weeks I’ve seen
>
> Framework failures:
>
>plugins/plugins07.run  plugins07 [normal] (pre_cmd failed: 2)
>
>plugins/T10420.run T10420 [normal] (pre_cmd failed: 2)
>
>plugins/T11244.run T11244 [normal] (pre_cmd failed: 2)
>
>
>
> I have just learned to live with these failures, because I knew you were
> working on making things better.  But it sounds as if they are still taking
> place.
>

The commit I made should have reduced the amount of failing tests to 0.
framework failures are always quite unusual.


>
> So:
>
>- Yes, please make it not happen by default
>
> I've removed the code, if you update it should be gone. It was there and
on by default because I was trying to debug failures on Harbormaster, I
realized a switch isn't very useful as I won't be able to toggle it for
Harbormaster anyway.


>
>-
>- If you don’t get these framework failures, can we work together to
>resolve them?
>
> These don't happen for me nor on Harbormaster, try picking a test, e.g T10420


run only that test to make sure it's not a threading issue:

make TEST=T10420 test -C testsuite/tests

If it still gives a framework error then do at the top level

make VERBOSE=3 TEST=T10420 test -C testsuite/tests

once it runs, the output should contain the command it ran as a pre_cmd,
and the stdout and
stderr from the pre_cmd output. Could you then send the error?

if it doesn't show any of this, try

make CLEANP=0 VERBOSE=3 TEST= T10420 test -C testsuite/tests --trace

and copy and paste the pre_cmd command, which should just replay the action
it did.


Cheers,
Tamar


>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* Phyx 
> *Sent:* 13 June 2018 17:19
> *To:* Simon Peyton Jones 
> *Cc:* ghc-devs@haskell.org
> *Subject:* Re: Strace
>
>
>
> Hi Simon,
>
>
>
> The strace is only supposed to run when the normal test pre_cmd fails.
>
> If it's running that often it means your tests are all failing during
> pre_cmd with a framework failure
>
> https://git.haskell.org/ghc.git/blobdiff/4778cba1dbb6adf495930322d7f9e9
> db0af60d8f..60fb2b2160aa16194b74262f4df8fad5af171b0f:/testsuite/driver/
> testlib.py
>
>
>
> But maybe I shouldn't turn this on my default. I'll pramaterize it when I
> get home.
>
>
>
> Tamar.
>
>
>
> On Wed, Jun 13, 2018, 17:09 Simon Peyton Jones 
> wrote:
>
> Tamar
>
> I’m getting *megabytes* of output from ‘sh validate’ on windows.  It
> looks like this
>
>   629  151745 [main] sh 2880 fhandler_base::fhaccess: returning 0
>
>   291  152036 [main] sh 2880 faccessat: returning 0
>
> 7757  159793 [main] sh 2880 fhandler_base_overlapped::wait_overlapped:
> wfres 0, wores 1, bytes 7
>
> 179457 1608947 [main] make 11484 fhandler_base_overlapped::wait_overlapped:
> wfres 0, wores 1, bytes 7
>
>99  159892 [main] sh 2880 fhandler_base_overlapped::wait_overlapped:
> normal write, 7 bytes ispipe() 1
>
>   180 1609127 [main] make 11484 fhandler_base_overlapped::wait_overlapped:
> normal read, 7 bytes ispipe() 1
>
>   139  160031 [main] sh 2880 write: 7 = write(1, 0x6000396A0, 7)
>
>   142 1609269 [main] make 11484 fhandler_base::read: returning 7, binary
> mode
>
>   139 1609408 [main] make 11484 read: 7 = read(5, 0x60005B4B0, 7)
>
>   136 1609544 [main] make 11484 read: read(5, 0x60005B4B7, 193) blocking
>
> 4693  164724 [main] sh 2880 set_signal_mask: setmask 0, newmask 8,
> mask_bits 0
>
> but with hundreds of thousands of lines.  (I have not counted)
>
> I believe that it may be the result of this line, earlier in the log
>
> cd "/c/Users/simonpj/AppData/Local/Temp/ghctest-8fa9s6rk/test
> spaces/./plugins/plugins07.run" && *strace* $MAKE -s --no-print-directory
> -C rule-defining-plugin package.plugins07 TOP=/c/code/HEAD/testsuite#
>
> Note the strace.
>
> That in turn was added in your commit
>
> commit 60fb2b2160aa16194b74262f4df8fad5af171b0f
>
> Author: Tamar Christina 
>
> Date:   Mon May 28 19:34:11 2018 +0100
>
>
>
> Clean up Windows testsuite failures
>
>
>
> Summary:
>
> Another round and attempt at getting these down to 0.
>
> Could you perhaps have made a mistake here?  Currently validate is
> unusable.
>
> Thanks!
>
> Simon
>
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Strace

[Phyx]

Hi Simon,

The strace is only supposed to run when the normal test pre_cmd fails.
If it's running that often it means your tests are all failing during
pre_cmd with a framework failure
https://git.haskell.org/ghc.git/blobdiff/4778cba1dbb6adf495930322d7f9e9db0af60d8f..60fb2b2160aa16194b74262f4df8fad5af171b0f:/testsuite/driver/testlib.py

But maybe I shouldn't turn this on my default. I'll pramaterize it when I
get home.

Tamar.

On Wed, Jun 13, 2018, 17:09 Simon Peyton Jones 
wrote:

> Tamar
>
> I’m getting *megabytes* of output from ‘sh validate’ on windows.  It
> looks like this
>
>   629  151745 [main] sh 2880 fhandler_base::fhaccess: returning 0
>
>   291  152036 [main] sh 2880 faccessat: returning 0
>
> 7757  159793 [main] sh 2880 fhandler_base_overlapped::wait_overlapped:
> wfres 0, wores 1, bytes 7
>
> 179457 1608947 [main] make 11484
> fhandler_base_overlapped::wait_overlapped: wfres 0, wores 1, bytes 7
>
>99  159892 [main] sh 2880 fhandler_base_overlapped::wait_overlapped:
> normal write, 7 bytes ispipe() 1
>
>   180 1609127 [main] make 11484 fhandler_base_overlapped::wait_overlapped:
> normal read, 7 bytes ispipe() 1
>
>   139  160031 [main] sh 2880 write: 7 = write(1, 0x6000396A0, 7)
>
>   142 1609269 [main] make 11484 fhandler_base::read: returning 7, binary
> mode
>
>   139 1609408 [main] make 11484 read: 7 = read(5, 0x60005B4B0, 7)
>
>   136 1609544 [main] make 11484 read: read(5, 0x60005B4B7, 193) blocking
>
> 4693  164724 [main] sh 2880 set_signal_mask: setmask 0, newmask 8,
> mask_bits 0
>
> but with hundreds of thousands of lines.  (I have not counted)
>
> I believe that it may be the result of this line, earlier in the log
>
> cd "/c/Users/simonpj/AppData/Local/Temp/ghctest-8fa9s6rk/test
> spaces/./plugins/plugins07.run" && *strace* $MAKE -s --no-print-directory
> -C rule-defining-plugin package.plugins07 TOP=/c/code/HEAD/testsuite#
>
> Note the strace.
>
> That in turn was added in your commit
>
> commit 60fb2b2160aa16194b74262f4df8fad5af171b0f
>
> Author: Tamar Christina 
>
> Date:   Mon May 28 19:34:11 2018 +0100
>
>
>
> Clean up Windows testsuite failures
>
>
>
> Summary:
>
> Another round and attempt at getting these down to 0.
>
> Could you perhaps have made a mistake here?  Currently validate is
> unusable.
>
> Thanks!
>
> Simon
>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Why do we prevent static archives from being loaded when DYNAMIC_GHC_PROGRAMS=YES?

[Phyx]

You could work around the dlopen issue as long as the static library is
compiled with -fPIC by using  --whole-archive (assuming you permit dangling
references which will need to be resolved later) and making a shared
library out of the static code. But you'd have to create one shared library
per static library and preserve the order so you don't end up with symbol
collisions.

And you'd likely not want to do it this on every relink. But i think the -
fPIC is a much greater hurdle. Very few of the static libraries a user may
want to use would have this likely.

I think it'll end up being quite a messy situation..


On Thu, Jun 7, 2018, 22:01 Simon Marlow  wrote:

> There's a technical restriction. The static code would be compiled with
> the small memory model, so it would have 32-bit relocations for external
> references, assuming that those references would resolve to something in
> the low 2GB of the address space. But we would be trying to link it against
> shared libraries which could be loaded anywhere in the address space.
>
> If the static code was compiled with -fPIC then it might be possible, but
> there's also the restriction that we wouldn't be able to dlopen() a shared
> library that depends on the statically linked code, because the system
> linker can't see the symbols that the RTS linker has loaded. GHC doesn't
> currently know about this restriction, so it would probably go ahead and
> try, and things would break.
>
> Cheers
> Simon
>
>
> On 29 May 2018 at 04:05, Moritz Angermann  wrote:
>
>> Dear friends,
>>
>> when we build GHC with DYNAMIC_GHC_PROGRAMS=YES, we essentially prevent
>> ghc/ghci
>> from using archives (.a).  Is there a technical reason behind this?  The
>> only
>> only reasoning so far I've came across was: insist on using
>> dynamic/shared objects,
>> because the user said so when building GHC.
>>
>> In that case, we don't however prevent GHC from building archive (static)
>> only
>> libraries.  And as a consequence when we later try to build another
>> archive of
>> a different library, that depends via TH on the former library, GHC will
>> bail
>> and complain that we don't have the relevant dynamic/shared object.  Of
>> course we
>> don't we explicitly didn't build it.  But the linker code we have in GHC
>> is
>> perfectly capable of loading archives.  So why don't we want to fall back
>> to
>> archives?
>>
>> Similarly, as @deech asked on twitter[1], why we prevent GHCi from
>> loading static
>> libraries?
>>
>> I'd like to understand the technical reason/rational for this behavior.
>> Can
>> someone help me out here?  If there is no fundamental reason for this
>> behavior,
>> I'd like to go ahead and try to lift it.
>>
>> Thank you!
>>
>> Cheers,
>>  Moritz
>>
>> ---
>> [1]: https://twitter.com/deech/status/1001182709555908608
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Trac email

[Phyx]

No that was Gershom whom fixed it before. Perhaps he knows what's up?

Tamar

On Fri, Jun 1, 2018, 03:17 Simon Peyton Jones  wrote:

> Alas I am still getting no Trac mail whatsoever, despite Tamar’s work
> below.  Can anyone help?  It’s quite disabling.
>
>
>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* Phyx 
> *Sent:* 31 May 2018 22:43
> *To:* Artem Pelenitsyn 
> *Cc:* Simon Peyton Jones ; ghc-devs@haskell.org
> *Subject:* Re: Trac email
>
>
>
> The mail servers were backed up with spam apparently. No emails were
> delivered or received to any of the mailing lists
> https://www.reddit.com/r/haskell/comments/8mza0y/whats_up_with_mailhaskellorg_dark_since_sunday/
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.reddit.com%2Fr%2Fhaskell%2Fcomments%2F8mza0y%2Fwhats_up_with_mailhaskellorg_dark_since_sunday%2F=02%7C01%7Csimonpj%40microsoft.com%7Cc9289ade89fc485594c908d5c73f856c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636633997996261627=9bI3I5H21eIXKEVj9yGh8wgEYcestQTIkCMswR6SqKM%3D=0>
>
>
>
> Cheers,
>
> Tamar
>
> On Thu, May 31, 2018, 15:26 Artem Pelenitsyn 
> wrote:
>
> Hello,
>
>
>
> It stopped working for me from like Sunday and wasn't working for about
> two days. Then, on Tuesday, it silently started working again. Although, I
> haven't checked that I get all the emails.
>
>
>
> --
>
> Best, Artem
>
>
>
> On Thu, 31 May 2018, 21:13 Simon Peyton Jones via ghc-devs, <
> ghc-devs@haskell.org> wrote:
>
> Devs
>
> Trac has entirely stopped sending me email, so I have no idea what’s
> happening on the GHC front.  Could someone unglue it?
>
> Thanks!
>
> Simon
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs=02%7C01%7Csimonpj%40microsoft.com%7Cc9289ade89fc485594c908d5c73f856c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636633997996271635=fO5%2BEqQ9nm608%2FQ0CyyLzGSwNhakfW%2BDxmf%2FixIrjMQ%3D=0>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.haskell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc-devs=02%7C01%7Csimonpj%40microsoft.com%7Cc9289ade89fc485594c908d5c73f856c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C1%7C636633997996281643=tZML5j%2BBun2YFcBLMdJ14go7XFZOQZT28P5BAySActU%3D=0>
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: -fghci-leak-check apparently causes many tests to fail

[Phyx]

I don't know what -fghci-leak-check does at all, but if they are to be
expected we shouldn't accept the changes. Instead change the default
options in the testsuite to pass -fno-ghci-leak-check (I assume that
exists)

On Thu, May 31, 2018, 06:49 Ryan Scott  wrote:

> I recently ran the testsuite and experienced a very large number of
> testsuite failures, all of which seem to involve the new -fghci-leak-check
> flag. Here is the list of tests that fail:
>
> Unexpected failures:
>ghci/prog001/prog001.runprog001 [bad stdout] (ghci)
>ghci/prog002/prog002.runprog002 [bad stdout] (ghci)
>ghci/prog003/prog003.runprog003 [bad stdout] (ghci)
>ghci/prog010/ghci.prog010.run   ghci.prog010 [bad stdout] (ghci)
>ghci/prog013/prog013.runprog013 [bad stdout] (ghci)
>ghci/prog012/prog012.runprog012 [bad stdout] (ghci)
>ghci/prog009/ghci.prog009.run   ghci.prog009 [bad stdout] (ghci)
>ghci/scripts/ghci025.runghci025 [bad stdout] (ghci)
>ghci/scripts/ghci038.runghci038 [bad stdout] (ghci)
>ghci/scripts/ghci057.runghci057 [bad stdout] (ghci)
>ghci/scripts/T2182ghci.run  T2182ghci [bad stdout] (ghci)
>ghci/scripts/ghci058.runghci058 [bad stdout] (ghci)
>ghci/scripts/T6106.run  T6106 [bad stdout] (ghci)
>ghci/scripts/T8353.run  T8353 [bad stdout] (ghci)
>ghci/scripts/T9293.run  T9293 [bad stdout] (ghci)
>ghci/scripts/T10989.run T10989 [bad stdout] (ghci)
>ghci/should_run/T13825-ghci.run T13825-ghci [bad stdout] (ghci)
>ghci.debugger/scripts/print007.run  print007 [bad stdout] (ghci)
>ghci.debugger/scripts/break009.run  break009 [bad stdout] (ghci)
>ghci.debugger/scripts/break008.run  break008 [bad stdout] (ghci)
>ghci.debugger/scripts/break026.run  break026 [bad stdout] (ghci)
>perf/space_leaks/T4029.run  T4029 [bad stdout] (ghci)
>
> And the full failing test output can be found here [1]. (I won't post it
> inline, since it's quite large).
>
> Are these changes expected? I'm not at all familiar with
> -fghci-leak-check, so I don't know if we should accept the new output or
> not.
>
> Ryan S.
> -
> [1]
> https://gist.githubusercontent.com/RyanGlScott/f920737287049b82947e1c47cdbc2b94/raw/4fe68d47cc78675424e09cf451be556c6f430d08/gistfile1.txt
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Trac email

[Phyx]

The mail servers were backed up with spam apparently. No emails were
delivered or received to any of the mailing lists
https://www.reddit.com/r/haskell/comments/8mza0y/whats_up_with_mailhaskellorg_dark_since_sunday/

Cheers,
Tamar

On Thu, May 31, 2018, 15:26 Artem Pelenitsyn 
wrote:

> Hello,
>
> It stopped working for me from like Sunday and wasn't working for about
> two days. Then, on Tuesday, it silently started working again. Although, I
> haven't checked that I get all the emails.
>
> --
> Best, Artem
>
>
> On Thu, 31 May 2018, 21:13 Simon Peyton Jones via ghc-devs, <
> ghc-devs@haskell.org> wrote:
>
>> Devs
>>
>> Trac has entirely stopped sending me email, so I have no idea what’s
>> happening on the GHC front.  Could someone unglue it?
>>
>> Thanks!
>>
>> Simon
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: End of Windows Vista support in GHC-8.6?

[Phyx]

Hi Simon,

Whatever happened to this? The wiki was updated but I don't see a commit
actually removing vista support.

Did you end up not doing this anymore?

Thanks,
Tamar

On Mon, Mar 5, 2018 at 7:21 PM, Simon Jakobi <simon.jak...@googlemail.com>
wrote:

> Thanks everyone!
>
> I have updated https://ghc.haskell.org/trac/ghc/wiki/Platforms/Windows
> accordingly.
>
> Cheers,
> Simon
>
> 2018-03-05 18:29 GMT+01:00 Phyx <loneti...@gmail.com>:
>
>>
>>
>> On Mon, Mar 5, 2018, 17:23 Ben Gamari <b...@well-typed.com> wrote:
>>
>>> Simon Jakobi via ghc-devs <ghc-devs@haskell.org> writes:
>>>
>>> > Hi!
>>> >
>>> > Given that Vista’s EOL was in April 2017
>>> > <https://support.microsoft.com/en-us/help/22882/windows-vist
>>> a-end-of-support>
>>> > i assume that there’s no intention to keep supporting it in GHC-8.6!?
>>> >
>>> > I’m asking because I intend to use a function
>>> > <https://msdn.microsoft.com/en-us/library/windows/desktop/dd
>>> 405488.aspx>
>>> > that requires Windows 7 or newer for #13362
>>> > <https://ghc.haskell.org/trac/ghc/ticket/13362>.
>>> >
>>> Given that it's EOL'd, dropping Vista sounds reasonable to me.
>>>
>>> Tamar, any objection?
>>>
>>
>> No objections, however do make sure to test both 32 and 64 bit builds of
>> ghc when you use the API, it's new enough and rare enough that it may not
>> be implemented in both mingw-64 tool chains (we've had similar issues
>> before).
>>
>> Thanks,
>> Tamar
>>
>>
>>> Cheers,
>>>
>>> - Ben
>>>
>>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows: cannot create here-doc

[Phyx]

Hi Simon,

This one is very strange, from the error and the fact that it continues it
looks like whatever TEMP and TMP
are pointing to are not always available or some locking issue is going on.

I would try running ProcMon
https://docs.microsoft.com/en-us/sysinternals/downloads/procmon and
filtering paths to
whatever TEMP and TMP are set to.

In the dialog that opens when you run it, select in the first box path,
select the "starts with" condition then enter the Windows version of the
path in the third one.
Make sure the final box is set to "Include" and press add. Then try again.
Once it fails, stop capturing (File -> click on "capture events" to
unselect).

This should contain the accesses to things in your TEMP and why it failed.

(If TEMP contains a unix path you can get a windows one with e.g  cygpath
-w $TEMP)

Once we know more can figure out what's going on.

Thanks,
Tamar

On Wed, Apr 4, 2018 at 10:17 PM, Simon Peyton Jones 
wrote:

> Tamar
>
> I think your suggestion of adding
>
> # Set user-defined locale
> export LANG=$(locale -uU)
>
> worked.  No more perl messages.
>
> Still having various troubles…
>
> Here’s the current one (log below).  I’m getting lots of
>
>./configure: line 2534: cannot create temp file for here-document:
> Device or resource busy
>
> And finally
>
> checking whether the C compiler works... no
>
> sed: can't read conftest.c: No such file or directory
>
> configure: error: in `/c/code/HEAD':
>
> configure: error: C compiler cannot create executables
>
> I have environment variables TEMP and TMP set.
>
> Oddly, re-running configure gets further – see second log below.
>
> All deeply strange.
>
> Thanks
>
> Simon
>
>
>
> *Log 1: first run of ‘sh validate –fast’*
>
> Creating libraries/array/ghc.mk
>
> Creating libraries/base/ghc.mk
>
> Creating libraries/binary/ghc.mk
>
> Creating libraries/bytestring/ghc.mk
>
> Creating libraries/Cabal/Cabal/ghc.mk
>
> Creating libraries/containers/ghc.mk
>
> Creating libraries/deepseq/ghc.mk
>
> Creating libraries/directory/ghc.mk
>
> Creating libraries/dph/dph-base/ghc.mk
>
> Creating libraries/dph/dph-prim-interface/ghc.mk
>
> Creating libraries/dph/dph-prim-seq/ghc.mk
>
> Creating libraries/dph/dph-prim-par/ghc.mk
>
> Creating libraries/dph/dph-lifted-base/ghc.mk
>
> Creating libraries/dph/dph-lifted-boxed/ghc.mk
>
> Creating libraries/dph/dph-lifted-copy/ghc.mk
>
> Creating libraries/dph/dph-lifted-vseg/ghc.mk
>
> Creating libraries/filepath/ghc.mk
>
> Creating libraries/ghc-boot/ghc.mk
>
> Creating libraries/ghc-boot-th/ghc.mk
>
> Creating libraries/ghc-compact/ghc.mk
>
> Creating libraries/ghc-prim/ghc.mk
>
> Creating libraries/ghci/ghc.mk
>
> Creating libraries/haskeline/ghc.mk
>
> Creating libraries/hpc/ghc.mk
>
> Creating libraries/integer-gmp/ghc.mk
>
> Creating libraries/integer-simple/ghc.mk
>
> Creating libraries/mtl/ghc.mk
>
> Creating libraries/parallel/ghc.mk
>
> Creating libraries/parsec/ghc.mk
>
> Creating libraries/pretty/ghc.mk
>
> Creating libraries/primitive/ghc.mk
>
> Creating libraries/process/ghc.mk
>
> Creating libraries/random/ghc.mk
>
> Creating libraries/stm/ghc.mk
>
> Creating libraries/template-haskell/ghc.mk
>
> Creating libraries/terminfo/ghc.mk
>
> Creating libraries/text/ghc.mk
>
> Creating libraries/time/ghc.mk
>
> Creating libraries/transformers/ghc.mk
>
> Creating libraries/unix/ghc.mk
>
> Creating libraries/vector/ghc.mk
>
> Creating libraries/Win32/ghc.mk
>
> Creating libraries/xhtml/ghc.mk
>
> Booting .
>
> Booting libraries/base/
>
> Booting libraries/directory/
>
> Booting libraries/ghc-prim/
>
> Booting libraries/integer-gmp/
>
> Booting libraries/process/
>
> Booting libraries/terminfo/
>
> Booting libraries/time/
>
> Booting libraries/unix/
>
> *./configure: line 2534: cannot create temp file for here-document: Device
> or resource busy*
>
> checking for gfind... no
>
> checking for find... /usr/bin/find
>
> checking for sort... /usr/bin/sort
>
> checking for GHC version date... inferred 8.5.20180402
>
> checking for GHC Git commit id... inferred 6ae53e4f4af1a156a875e05a2c12ef
> eaa2e7d902
>
> checking for ghc... /c/fp/HP-8.2.2/bin/ghc
>
> checking version of ghc... 8.2.2
>
> GHC path canonicalised to: c:/fp/HP-8.2.2/bin/ghc
>
> checking build system type... x86_64-pc-mingw64
>
> checking host system type... x86_64-pc-mingw64
>
> checking target system type... x86_64-pc-mingw64
>
> Build platform inferred as: x86_64-unknown-mingw32
>
> Host platform inferred as: x86_64-unknown-mingw32
>
> Target platform inferred as: x86_64-unknown-mingw32
>
> GHC build  : x86_64-unknown-mingw32
>
> GHC host   : x86_64-unknown-mingw32
>
> GHC target : x86_64-unknown-mingw32
>
> LLVM target: x86_64-unknown-windows
>
> checking for path to top of build tree... C:/code/HEAD
>
> configure: Checking for Windows toolchain tarballs...
>
> configure: Extracting Windows toolchain from archives (may take a while)...
>
> configure: In-tree MingW-w64 tree created
>
> checking for 

Re: Perl locale

[Phyx]

Hi Simon,

That's weird, "ENG" isn't a valid locale as far as I know.

The locale should be getting set by your .profile by the line

# Set user-defined locale
export LANG=$(locale -uU)

In my case that returns

$ locale -uU
en_GB.UTF-8

I would check to see if that line is in your .profile (it should be by
default) or in case you're not login into bash (using --login), in your
.bashrc. That should fix the warning.

However it's weird that it's using c:/msys64/usr/bin/perl to build GHC. As
far as I know, the only usage of perl in GHC is for SplitObjs
in which case we use a very old perl distribution that should have been
downloaded by configure into inplace/perl/

and configure should have picked that one up. It's hardcoded somewhat to do
so. What does configure report that it found?

in any case, setting the locale as above should fix the warning.

Cheers,
Tamar

On Wed, Apr 4, 2018 at 2:57 PM, Simon Peyton Jones 
wrote:

> Hi Tamar
>
> One other thing happened to me when building GHC, very early.  See below.
> I think the perl concerned is c:/msys64/usr/bin/perl.
>
> Any ideas?  It seems non-fatal
>
> Thanks
>
> Simon
>
> perl: warning: Setting locale failed.
>
> perl: warning: Please check that your locale settings:
>
>LC_ALL = (unset),
>
>LANG = "ENG"
>
> are supported and installed on your system.
>
> perl: warning: Falling back to the standard locale ("C").
>
> perl: warning: Setting locale failed.
>
> perl: warning: Please check that your locale settings:
>
>LC_ALL = (unset),
>
>LANG = "ENG"
>
> are supported and installed on your system.
>
> perl: warning: Falling back to the standard locale ("C").
>
> perl: warning: Setting locale failed.
>
> perl: warning: Please check that your locale settings:
>
>LC_ALL = (unset),
>
>LANG = "ENG"
>
> are supported and installed on your system.
>
> perl: warning: Falling back to the standard locale ("C").
>
> perl: warning: Setting locale failed.
>
> perl: warning: Please check that your locale settings:
>
>LC_ALL = (unset),
>
>LANG = "ENG"
>
> are supported and installed on your system.
>
> perl: warning: Falling back to the standard locale ("C").
>
> perl: warning: Setting locale failed.
>
> perl: warning: Please check that your locale settings:
>
>LC_ALL = (unset),
>
>LANG = "ENG"
>
> are supported and installed on your system.
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: More windows

[Phyx]

Hi Simon,

Hmm I'm not sure about replacing sh with bash. I think bash has some
Non-POSIX extensions that may affect the behavior of valid posix scripts.

Is bash --login slow as well? How about once sh or bash starts, are
commands still slow then?

I assume your computer is domain joined and you may be hitting a very long
standing issue with certain domain joined machines
https://github.com/Alexpux/MSYS2-packages/issues/138#issuecomment-70813762

The solution seems to be to cache the user info locally instead of it
having to query the domain controller everytime. See solution 2 here
https://gist.github.com/k-takata/9b8d143f0f3fef5abdab for instructions

Does that help the problem?

I believe you had a similar problem last time setting up a new machine. At
that time magit was also slow.

Kind regards,
Tamar

On Mon, Apr 2, 2018, 23:23 Simon Peyton Jones  wrote:

> Tamar
>
> I’ve noticed that “sh” (which is invoked at lot by make etc) takes AGES to
> start up.  At least I think it’s ‘sh’ that is causing the delay.
>
> I think it’s c:/msys64/usr/bin/sh.exe
>
> From searching the web (eg
> https://www2.cs.duke.edu/csl/docs/unix_course/intro-60.html)  it seems
> likely that it executes c:/msys64/etc/profile first.
>
> And If I put an ‘echo’ at the start and end of that file, they do seem to
> take place with a significant gap between them.
>
> I have not started sprinkling more echos, but does that ring any bells?
>
> Can I replace ‘sh’ with c:/msys64/usr/bin/bash.exe, which seems to be
> faster?   (My evnt variable SHELL already points to bash.exe. )  And if so,
> how would I do that? An environment variable.  Physically copy bash.exe to
> sh.exe?  Or what?
>
> Thanks
>
> Simon
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows

[Phyx]

On Mon, Mar 26, 2018 at 11:08 AM, Simon Peyton Jones via ghc-devs <
ghc-devs@haskell.org> wrote:

> One other thing. The instructions at
>
> https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows
>
> say to set PATH thus
>
>
>
> export PATH=/mingw64/bin:\$PATH
>
>
>
> But
>
>- in the installation tree I got from msys, *there IS no
>c:/mingw64/bin*.
>- I have a root directory c:/msys64.
>
>
These are correct, the instructions are written from the point of view of a
msys2 environment and not the Windows one.
So /mingw64 refers to an msys2 path not a windows path. In windows the /
would indeed be interpreted to %HOMEDRIVE%/mingw64
but in msys2 this refers to what the msy2 filesystem has been mounted to.
This is usually C:\msys64.

Tamar@Yelow ~/ghc2> cygpath -w /mingw64/bin/
E:\msys64\mingw64\bin\

the msys2 tool cygpath converts between the two usually on demand. but you
can query it for paths.
What the instructions intended was for you to modify your msys2
environment, not your Windows one, as generally putting
Cygwin derived programs on your global windows path is not advisable.


>
>-
>- Most binaries (eg *bash*, *find*) seem to be in c:/mingw64/usr/bin
>
> Yes, any binary that is a posix port, e.g. depends on newlib crt instead
of the Microsoft runtime goes into the usual /usr/bin folder. This would
include core-utils,
make etc.


>
>-
>- But there is also c:/msys64/mingw64/bin, which (perhaps as a result
>of the pacman step) has lots more stuf like *ar*, *cc*, etc.
>
> This folder is meant to contain native ports, e.g. programs that have been
"ported" to work on Windows using a standard windows runtime. These have no
dependencies on
newlib/Cygwin. This is the distinctions between these two folders.


>-
>
>
>
> All very confusing.  For now I have put both c:/msys64/usr/bin and
> c:/msys64/mingw64/bin in my path.   That seems to work, but is clearly not
> what the instructions say.
>

The instruction could be somewhat clearer here, but they are intended to
only affect your msys2 installation, and never your global Windows system.


>
>
> Thanks
>
>
>
> Simon
>
>
>
> *From:* ghc-devs  *On Behalf Of *Shao, Cheng
> *Sent:* 26 March 2018 10:59
> *To:* ghc-devs@haskell.org
> *Subject:* Re: Windows
>
>
>
> Hi Simon,
>
>
>
> If the build environment is managed by an MSYS2 installation, then the
> MinGW64 shell startup script automatically sets up "MSYSTEM" for you. It
> can be launched like "C:\msys64\msys2_shell.cmd -mingw64 -mintty".
>
>
>
> On Mon, Mar 26, 2018 at 5:46 PM, Simon Peyton Jones via ghc-devs <
> ghc-devs@haskell.org> wrote:
>
> Making it part of the error message would be v helpful.
>
> I have added a section to "Troubleshooting" on
> https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation/Windows
>
> But it should really be part of the instructions higher up to sa
> export MSYSTEM=MINGW64
>
> Might someone do that?  I wasn't quite sure where
>
> Simon
>
>
> |  -Original Message-
> |  From: ghc-devs  On Behalf Of Ben Gamari
> |  Sent: 24 March 2018 16:42
> |  To: Gabor Greif ; loneti...@gmail.com
> |  Cc: ghc-devs@haskell.org
> |  Subject: Re: Windows
> |
> |  Gabor Greif  writes:
> |
> |  > Just an idea...
> |  >
> |  > could this hint be part of the `configure` error message?
> |  >
> |  Indeed. See D4526.
> |
> |  Cheers,
> |
> |  - Ben
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: End of Windows Vista support in GHC-8.6?

[Phyx]

On Mon, Mar 5, 2018, 17:23 Ben Gamari  wrote:

> Simon Jakobi via ghc-devs  writes:
>
> > Hi!
> >
> > Given that Vista’s EOL was in April 2017
> > <
> https://support.microsoft.com/en-us/help/22882/windows-vista-end-of-support
> >
> > i assume that there’s no intention to keep supporting it in GHC-8.6!?
> >
> > I’m asking because I intend to use a function
> > 
> > that requires Windows 7 or newer for #13362
> > .
> >
> Given that it's EOL'd, dropping Vista sounds reasonable to me.
>
> Tamar, any objection?
>

No objections, however do make sure to test both 32 and 64 bit builds of
ghc when you use the API, it's new enough and rare enough that it may not
be implemented in both mingw-64 tool chains (we've had similar issues
before).

Thanks,
Tamar


> Cheers,
>
> - Ben
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Can't push to staging area?

[Phyx]

This is a local git configuration issue. Your pack scripts (git-upload-pack
etc) are on a path that requires sudo or your sudoers configuration is
wrong. See
https://stackoverflow.com/questions/24059597/phabricator-git-ssh-clone-fails-with-password-required-error

On Thu, Jan 4, 2018, 01:06 Brandon Allbery  wrote:

> "sudo: a password is required"
>
> Can't even tell if that's local (beware e.g. Ubuntu defaults) or remote
> (which would be a configuration problem on the remote end, not to mention
> seeming like a bad idea).
>
> On Wed, Jan 3, 2018 at 6:36 PM, Bartosz Nitka  wrote:
>
>> I'm trying to update a diff and I run into this:
>>
>> $ arc diff
>> Linting...
>>  LINT OKAY  No lint problems.
>> Running unit tests...
>> No unit test engine is configured for this project.
>>  PUSH STAGING  Pushing changes to staging area...
>> sudo: a password is required
>> fatal: Could not read from remote repository.
>>
>> Please make sure you have the correct access rights
>> and the repository exists.
>>  STAGING FAILED  Unable to push changes to the staging area.
>> Usage Exception: Failed to push changes to staging area. Correct the
>> issue, or use --skip-staging to skip this step.
>>
>>
>> I believe that it worked for me before with my setup, and I seem to be
>> in compliance with
>>
>> https://ghc.haskell.org/trac/ghc/wiki/Phabricator#Startingoff:Fixingabugsubmittingareview
>>
>> Any ideas?
>>
>> Thanks,
>> Bartosz
>> ___
>> ghc-devs mailing list
>> ghc-devs@haskell.org
>> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>>
>
>
>
> --
> brandon s allbery kf8nh   sine nomine
> associates
> allber...@gmail.com
> ballb...@sinenomine.net
> unix, openafs, kerberos, infrastructure, xmonad
> http://sinenomine.net
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: ghc-prim 0.5.1.1 not on Hackage

[Phyx]

Hmm I think this one belongs on the ghc trac.

The version doesn't seem to exist in master either
https://github.com/ghc/ghc/commits/master/libraries/ghc-prim/ghc-prim.cabal
and only exists in the 8.2 branch.

The commit and bug report it references #14171 doesn't seem to have any
code changes to ghc-prim (that I can see). So I'm not sure why this was
bumped.

So this version is a bit of a mystery..

On Tue, Dec 19, 2017, 20:47 Michael Snoyman  wrote:

> The Stackage issue tracker just received an issue about ghc-prim 0.5.1.1.
>
> https://github.com/fpco/stackage/issues/3115
>
> This version of the package has not been uploaded to Hackage, so:
>
> * The Stackage snapshots refer to 0.5.1.1
> * The stackage.org documentation shows version 0.5.1.1
> * `stack unpack` and `cabal unpack` both fail on ghc-prim-0.5.1.1
>
> Also, it appears that the GHC 8.2.2 release notes still refer to ghc-prim
> 0.5.1.0.
>
> I'm happy to file an issue for this, but last time this popped up it was
> unclear if the issue should be filed on GHC Trac or the Hackage Trustees
> issue tracker.
>
> Michael
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [GHC] #14537: Do not link to msvcrt.dll

[Phyx]

Add a -v3 to ghc to see what's happening.

On Wed, Dec 13, 2017, 07:23 GHC  wrote:

> #14537: Do not link to msvcrt.dll
> -+-
> Reporter:  johndoe   |Owner:  (none)
> Type:  feature request   |   Status:  closed
> Priority:  normal|Milestone:
>Component:  Compiler  |  Version:  8.2.1
>   (Linking)  |
>   Resolution:  invalid   | Keywords:
> Operating System:  Windows   | Architecture:
>  |  Unknown/Multiple
>  Type of failure:  Other |Test Case:
>   Blocked By:| Blocking:
>  Related Tickets:|  Differential Rev(s):
>Wiki Page:|
> -+-
>
> Comment (by johndoe):
>
>  Well, I tried to modify `specs` directly, but project failed to build:
>  I've got messagebox that `setup.exe` crashed.
>  That was on my local machine, but the same you can see on appveyor:
>  https://ci.appveyor.com/project/johnd0e/pandoc/build/1.0.79
>
>  I am not sure, maybe my `specs` file is not good:
>  https://github.com/johnd0e/pandoc/blob/master/specs
>  But it worked well with other project (not GHC).
>
> --
> Ticket URL: 
> GHC 
> The Glasgow Haskell Compiler
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Bringing back #ghc IRC logging

[Phyx]

Herbert said he submitted the request months ago.

As botbot.me site says they don't honor every request, so I guess we
weren't selected.

I'd still prefer botbot.me as it seems more stable than ircbrowse.net which
also doesn't seem to work on mobile (Android, get a server error)

On Wed, Oct 25, 2017, 02:55 Ben Gamari  wrote:

> Niklas Hambüchen  writes:
>
> > Hey Ben,
> >
> > I haven't heard of a single concern of enabling botbot.me over the last
> > 3 months.
> >
> > Can we go ahead and enable it?
> >
> > All we need is a channel op to fill out https://botbot.me/request/.
> >
> Yep, let's do it. Thanks for following up on this!
>
> Herbert, can you complete the request?
>
> Cheers,
>
> - Ben
>
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs