Re: testsuite broken

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> Yikes.  I can’t run the testsuite on Linux (debian ? I think…).  See below.
> I installed python3 by saying
> apt-get install python3
> And indeed
>
> python3 --version
>
> Python 3.2.3

For the record I've opened #12909 to track this. There is a fix in
D2778 which I'll merge shortly.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 8.0.2 release candidate 1

[Ben Gamari]

Jens Petersen <juhpeter...@gmail.com> writes:

> On 26 November 2016 at 07:38, Ben Gamari <b...@well-typed.com> wrote:
>
>> http://downloads.haskell.org/~ghc/8.0.2-rc1/
>>
>
> Thank you, I built it for Fedora 25 (just released last week) and Rawhide:
>
> https://copr.fedorainfracloud.org/coprs/petersen/ghc-8.0.2
>
> Hopefully there will be a build for F24 soon too.
>
Thanks Jens!

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Ben Gamari]

Michal Terepeta  writes:

> Interesting! I must have missed this proposal.  It seems that it didn't meet
> with much enthusiasm though (but it also proposes to have a completely
> separate
> repo on github).
>
> Personally, I'd be happy with something more modest:
> - A collection of modules/programs that are more representative of real
>   Haskell programs and stress various aspects of the compiler.
>   (this seems to be a weakness of nofib, where >90% of modules compile
>   in less than 0.4s)

This would be great.

> - A way to compile all of those and do "before and after" comparisons
>   easily. To measure the time, we should probably try to compile each
>   module at least a few times. (it seems that this is not currently
>   possible with `tests/perf/compiler` and
>   nofib only compiles the programs once AFAICS)
>
> Looking at the comments on the proposal from Moritz, most people would
> prefer to
> extend/improve nofib or `tests/perf/compiler` tests. So I guess the main
> question is - what would be better:
> - Extending nofib with modules that are compile only (i.e., not
>   runnable) and focus on stressing the compiler?
> - Extending `tests/perf/compiler` with ability to run all the tests and do
>   easy "before and after" comparisons?
>
I don't have a strong opinion on which of these would be better.
However, I would point out that currently the tests/perf/compiler tests
are extremely labor-intensive to maintain while doing relatively little
to catch performance regressions. There are a few issues here:

 * some tests aren't very reproducible between runs, meaning that
   contributors sometimes don't catch regressions in their local
   validations
 * many tests aren't very reproducible between platforms and all tests
   are inconsistent between differing word sizes. This means that we end
   up having many sets of expected performance numbers in the testsuite.
   In practice nearly all of these except 64-bit Linux are out-of-date.
 * our window-based acceptance criterion for performance metrics doesn't
   catch most regressions, which typically bump allocations by a couple
   percent or less (whereas the acceptance thresholds range from 5% to
   20%). This means that the testsuite fails to catch many deltas, only
   failing when some unlucky person finally pushes the number over the
   threshold.

Joachim and I discussed this issue a few months ago at Hac Phi; he had
an interesting approach to tracking expected performance numbers which
may both alleviate these issues and reduce the maintenance burden that
the tests pose. I wrote down some terse notes in #12758.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Ben Gamari]

Michal Terepeta  writes:

> Hi everyone,
>
> I've been running nofib a few times recently to see the effect of some
> changes
> on compile time (not the runtime of the compiled program). And I've started
> wondering how representative nofib is when it comes to measuring compile
> time
> and compiler allocations? It seems that most of the nofib programs compile
> really quickly...
>
> Is there some collections of modules/libraries/applications that were put
> together with the purpose of benchmarking GHC itself and I just haven't
> seen/found it?
>
Sadly no; I've put out a number of calls for minimal programs (e.g.
small, fairly free-standing real-world applications) but the response
hasn't been terribly strong. I frankly can't blame people for not
wanting to take the time to strip out dependencies from their working
programs. Joachim and I have previously discussed the possibility of
manually collecting a set of popular Hackage libraries on a regular
basis for use in compiler performance characterization.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Github repos for boot libraries

[Ben Gamari]

Edward Kmett  writes:

> For reference, the master repository for transformers is at
>
> http://hub.darcs.net/ross/transformers
>
> We should probably edit the 'website' link for that github repository to at
> least point there.
>
> I don't have access to do so, however.
>
> Subtly pinging Herbert, by adding him here. =)
>
I have fixed the GitHub repo URL.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[ANNOUNCE] Formation of the initial GHC Steering Committee

[Ben Gamari]

Dear Haskell community,

Over the past months we have discussed changes to GHC's process for
collecting, discussing, and considering new language extensions,
compiler features, and the like. Happily, we are now ready to move
forward with our new proposal process.

Towards this end, we have formed the GHC Steering Committee which will
be responsible for evaluating the proposals that run through the
process. The committee consists of the following members (with GitHub
user names given parenthetically),

 * Chris Allen (@bitemyapp)
 * Joachim Breitner (@nomeata)
 * Manuel M T Chakravarty (@mchakravarty)
 * Iavor Diatchki (@yav)
 * Atze Dijkstra (@atzedijkstra)
 * Richard Eisenberg (@goldfirere)
 * Ben Gamari (@bgamari)
 * Simon Marlow (@simonmar)
 * Ryan Newton (@rrnewton)
 * Simon Peyton-Jones (@simonpj)

The body will be chaired jointly by Simon Marlow and Simon Peyton-Jones.

Since the ghc-proposals repository was created, it has accumulated
nearly thirty pull requests describing a variety of compelling changes.
We will consider these proposals to be at the beginning of their
four-week discussion period. The goal of this discussion is to find and
eliminate weaknesses of the proposal. The final proposal should
address all valid points raised in the discussion. When you believe the
proposal has converged, bring it to the steering committee and summarize
the discussion in a pull request comment.

If you would like to contribute a new proposal, please refer to the
directions given in the ghc-proposals' repository README [1] and
proposal submission guidelines [2].

Cheers,

- Ben, on behalf of the GHC Steering Committee


[1] https://github.com/ghc-proposals/ghc-proposals
[2] 
https://github.com/ghc-proposals/ghc-proposals/blob/master/proposal-submission.rst


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Phabricator upgrade tomorrow

[Ben Gamari]

Hello everyone!

Currently our Phabricator installation is quite old, based on a commit
from last July. Given that I have a bit of breathing room now between
8.0.2 and 8.2.1, I'd like to take this opportunity to do an upgrade
tomorrow if no one objects. Given that this is the first time I have
attempted an upgrade, I expect this may take a few hours in the middle
of the day EST. I would expect that the GHC Phabricator instance would
be down for much of this time.

Let me know if this will cause you undue burden.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Trac to Phabricator (Maniphest) migration prototype

[Ben Gamari]

Matthew Pickering  writes:

> Dear devs,
>
Hi Matthew and Dan,

First, thanks for your work on this; it is an impressive effort.
Reconstructing a decade of tickets with broken markup, tricky syntax,
and a strange data model is no easy task. Good work so far!

On the whole I am pleasantly surprised by how nicely Maniphest seems to
hang together. I have pasted my notes from my own reflection on the pros
and cons of both systems below. On re-reading them, it seems clear that
Trac does leave us with a number of issues which Phabricator may
resolve.

As I've expressed in the past, I think we should consider preservation
of ticket numbers to be an absolute requirement of any migration. To do
otherwise imposes a small cost on a large number of people for a
very long time with little benefit. GHC infrastructure exists to
support GHC, not the other way around.

However, ticket numbers notwithstanding I think I would be fine moving
in this direction if the community agrees that this is the direction we
want to go in. 

There are a few questions that remain outstanding, however:


What do others think of this?
=

Does Phabricator address the concerns that others, including those
outside of the usual GHC development community, have raised about our
issue tracking in the past? It would be interesting to hear some outside
voices.


How do we handle the stable branch?
===

One important role that the issue tracker plays is ensuring that 
patches are backported to the stable branch when appropriate. In Trac we
handle this with a "merge" state (indicating that the ticket has been
fixed in `master`, but the fix needs to be backported) and the milestone
field (to indicate which release we want to backport to).

A significant amount of GHC's maintenance load is spent backporting and
updating tickets, so it's important that this process works smoothly and
reliably. I think this is may be an area where Phabricator could
improve the status quo since the workflow currently looks something like
this,

 1. Someone merges a patch to `master` fixing an issue; if the commit
message mentions the ticket number then our infrastructure
automatically leaves a comment referencing the commit on the ticket.

 2. Someone (usually the committer) places the ticket in `merge` state
and sets the milestone appropriately

 3. I merge the patch to the stable branch

 4. I close the ticket and manually leave a comment mentioning the SHA
of the backported commit.

In particular (4) is more work than it needs to be; ideally comment
generation would be automated as it is for commits to `master` but
Trac's comment-on-commit functionality is a bit limited, so this is
currently not an option.

I'm not sure what Phabricator's analogous workflow to the above might
look like. It seems that Phabricator's Releeph module may be in part
intended for this use-case, but it seems to have some unfortunate
limitations (e.g. the inflexibility in branch naming) that make it hard
to imagine it being usable in our case.

Setting aside Releeph, perhaps the best solution would be to continue
with our current workflow: we would retain the "status" state and
milestone projects would take the place of the current "milestone"
field. If I'm not mistaken Phabricator can be configured to mention
commits on stable branches in the ticket history, so this should help
point (4).


Which fields should be preserved?
=

Our Trac instance associates a lot of structured metadata with each
ticket. I generally think that this is a good thing, especially compared
to the everything-is-a-tag model which I have found can quickly become
unmaintainable. Unfortunately, Trac makes our users pay for these fields
with a confusing ticket form.

It appears that Phabricator's Transactions [1] module may allow us to have
our cake and eat it too: we can define one form for users to create
tickets and another, more complete form for developer use. In light of
this I don't see why we would need to fall back to the
everything-is-a-tag. Matthew, what did you feel was
less-than-satisfactory about the proper-fields approach? I fear that
relevant metadata like GHC version, operating system and architecture
simply won't be provided unless the user is explicitly prompted; I
personally find the cue to be quite helpful. Presumably contributors can
set Herald rules for notification on these fields if they so desire.

In the particular case of the "Component" field, I personally try to
set this correctly when possible and have certainly found it useful as a
search criterion. However, I suspect it would be fine as a tag. I also
know that Simon PJ is quite fond of the test case field (although
few others as as diligent in keeping this one up to date).


How would we migrate and what will become of Trac?
==

The mechanics of migration will 

Re: Phabricator upgrade underway

[Ben Gamari]

Ben Gamari <b...@well-typed.com> writes:

> Hello everyone,
>
> I'll be bringing down Phabricator for an upgrade in a few minutes. I'll
> let you know when things are back up.
>
Hello everyone,

The upgrade should now be complete. Feel free to resume your typical
Phabrication. I've done some testing but let me know if you encounter
any trouble.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Contributing Examples to the Documentation

[Ben Gamari]

Sunjay Varma  writes:

> Hi,
> I'm considering contributing examples to the documentation. I wanted to
> start with something like Data.List because it is one of the modules I end
> up using the most. I think a few examples for each function would help
> users understand them better. I find myself referring back to books like
> learn you a haskell because I don't remember exactly what I'm supposed to
> do with a function.
>
> Doing one module seems like a good start and hopefully we can have some
> other people begin to add their own examples too.
>
> Is this a worthwhile contribution? I haven't contributed before and so I
> think it's prudent to ask before I add something no one wants.
>
This would be amazing! Many people have remarked that GHC's library
documentation is in need of examples; we just need someone to step up
and start contributing patches.

> Are there any examples of modules with good code examples that I should use
> as a reference? I want to include both the code and output of the example
> as if the user was running ghci. Are there any guidelines for contributing
> documentation?
>
I think Edward's lens library is probably a good example here. I know
of few good examples in GHC itself.

> When I say Data.List, I really mean Data.Foldable and Data.Traversable
> since that is where the functions are actually implemented.
>
These are great places to start.

> I noticed the GitHub repo said that Pull Requests were okay for easy to
> review documentation changes. Can I open a pull request there or should I
> follow another process?
>
We prefer to take patches on Phabricator. However, to lower the barrier
to small patches like this I have suggested in the past that we accept
GitHub pull requests for small changes.

> Please let me know when you can. I don't have an exact timeline for when
> this will be done, but hopefully I'll have something in the next few weeks.
> I don't anticipate that it will take long once I sit down to do it.
>
Great. Let us know if you encounter friction.

> I've always complained about a lack of examples and never done anything
> about it. Hopefully I can practice what I preach and contribute some in
> order to make the documentation a little better for everyone.
>
> Thanks for helping to make this language so great!

And thanks to you for helping as well!

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Phabricator upgrade underway

[Ben Gamari]

Hello everyone,

I'll be bringing down Phabricator for an upgrade in a few minutes. I'll
let you know when things are back up.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: nofib on Shake

[Ben Gamari]

Michal Terepeta  writes:

> Hi all,
>
> While looking at nofib, I've found a blog post from Neil Mitchell [1],
> which describes a Shake build system for nofib. The comments mentioned
> that this should get merged, but it seems that nothing actually happened?
> Is there some fundamental reason for that?
>
Indeed there is no fundamental reason and I think it would be great to
make nofib a bit easier to run and modify.

However, I think we should be careful to maintain some degree of
compatibility. One of the nice properties of nofib is that it can be run
against a wide range of compiler versions. It would be ashame if, for
instance, Joachim's gipeda had to do different things to extract
performance metrics from logs produced by logs pre- and post-Shake
nofibs.

> We could also create a cabal and stack files for `nofib-analyse` (making
> it possible to use some libraries for it).
>
This would be great. This would allow me to drop a submodule from my own
performance monitoring tool.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Debugging GHC with GHCi

[Ben Gamari]

Thomas Jakway  writes:

> I want to be able to load certain GHC modules in interpreted mode in 
> ghci so I can set breakpoints in them.  I have tests in the testsuite 
> that are compiled by inplace/bin/ghc-stage2 with -package ghc.  I can 
> load the tests with ghc-stage2 --interactive -package ghc but since ghc 
> is compiled I can only set breakpoints in the tests themselves.  Loading 
> the relevant files by passing them as absolute paths to :l loads them 
> but ghci doesn't stop at the breakpoints placed in them (I'm guessing 
> because ghci doesn't realize that the module I just loaded is meant to 
> replace the compiled version in -package ghc).
>
> So if I use
>
> inplace/bin/ghc-stage2 --interactive -package ghc mytest.hs
> then
> :l abs/path/to/AsmCodeGen.hs
>
> and set breakpoints, nothing happens.
>
Many of us would love to be able to load GHC into GHCi. Unfortunately,
we aren't currently in a position where this is possible. The only thing
standing in our way is the inability of GHC's interpreter to run modules
which use unboxed tuples. While there are a few modules within GHC which
use unboxed tuples, none of them are particularly interesting for
debugging purposes, so compiling them with -fobject-code should be fine.
In principle this could be accomplished by,

{-# OPTIONS_GHC -fobject-code #-}

However, as explained in #10965, GHC sadly does not allow this. I spent
a bit of time tonight trying to see if I could convince GHC to first
manually build object code for the needed modules, and then load
bytecode for the rest. Unfortunately recompilation checking fought me at
every turn.

The current state of my attempt can be found here [1]. It would be great
if someone could pick it up. This will involve,

 * Working out how to convince the GHC to use the object code for
   utils/Encoding.o instead of recompiling

 * Identifying all of the modules which can't be byte-code compiled and
   add them to $obj_modules

 * Chassing down whatever other issues that might pop up along the way

I also wouldn't be surprised if you would need this GHC patch [2].

Cheers,

- Ben


[1] https://gist.github.com/bgamari/bd53e4fd6f3323599387ffc7b11d1a1e
[2] 
http://git.haskell.org/ghc.git/commit/326931db9cdc26f2d47657c1f084b9903fd46246


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Confused about the sub-modules

[Ben Gamari]

Erik de Castro Lopo  writes:

> Edward Z. Yang wrote:
>
>> Not any more. The commit just has to exist in the remote repo (that's
>> what the lint checks.)
>
> So this is where I am running into trouble. Everything for process 
> and directory is fine, but for Cabal and containers, the git repo
> on git.haskell.org is missing the commits I need.
>
Hmm, what in particular is missing? Cabal seems up-to-date (both
git.haskell.org:packages/Cabal and github.com/haskell/Cabal master
branches point to 09865f60caa55a7b02880f2a779c9dd8e1be5ac0). As does
containers (both point to 71c64747120c3cd1b91f06731167009b0e5b2454).

In general all of this should be reasonably automatic. However, when
upstreams push non-fast-forward updates to their branches a bit of
manual intervention is necessary; if in doubt just ask as you've done
here.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: User manual broken

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> I’m getting tons of this stuff from the user manual type setting. Might 
> someone fix it?

Hmm, I've tried a few environments and have so far been unable to
reproduce this. what version of sphinx-build are you using (e.g.
sphinx-build --version)?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: User manual broken

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> I’m getting tons of this stuff from the user manual type setting. Might 
> someone fix it?

Yes, I'm on it.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[ANNOUNCE] GHC 8.0.2 release candidate 2

[Ben Gamari]

Hello everyone,

The GHC team is happy to announce the second candiate of the
8.0.2 release of the Glasgow Haskell Compiler. Source and binary
distributions are available at

http://downloads.haskell.org/~ghc/8.0.2-rc2/

This is the second and likely final release candidate leading up the
8.0.2 release. This release will fix a number of bugs found in 8.0.1
including,

  * Interface file build determinism (#4012).

  * Compatibility with macOS Sierra and GCC compilers which compile 
position-independent executables by default

  * Runtime linker fixes on Windows (see #12797)

  * A compiler bug which resulted in undefined reference errors while
compiling some packages (see #12076)

  * Compatability with systems which use the gold linker

  * A number of memory consistency bugs in the runtime system

  * A number of efficiency issues in the threaded runtime which manifest
on larger core counts and large numbers of bound threads.

  * A typechecker bug which caused some programs using
-XDefaultSignatures to be incorrectly accepted.

  * More than two-hundred other bugs. See Trac [1] for a complete
listing.

This release candidate fixes a number of issues present in -rc1,

  * #12757, which lead to broken runtime behavior and even crashes in
the presence of primitive strings.

  * #12844, a type inference issue affecting partial type signatures.

  * A bump of the `directory` library, fixing buggy path
canonicalization behavior (#12894). Unfortunately this required a
major version bump in `directory` and minor bumps in several other
libraries.

  * #12912, where use of the `select` system call would lead to runtime
system failures with large numbers of open file handles.

If all goes well we should have a final 8.0.2 release out shortly after
the new year. As always, let us know if you encounter trouble. Thanks to
everyone who has contributed so far!

Happy testing,

- Ben


[1] 
https://ghc.haskell.org/trac/ghc/query?status=closed=8.0.2=id=summary=status=type=priority=milestone=component=priority


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Reading source annotations during type checking

[Ben Gamari]

Alan, did you see this?

Alejandro Serrano Mena  writes:

> Dear GHC devs,
> Is there a way to retrieve "source annotations" (as defined by
> https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/extending_ghc.html#source-annotations)
> during type checking. In particular, I am interested in reading them in
> TcExpr and TcCanonical.
>
> Regards,
> Alejandro
> ___
> ghc-devs mailing list
> ghc-devs@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Will directory 1.3.0.0 be shipped with GHC 8.0.2?

[Ben Gamari]

Andrés Sicard-Ramírez  writes:

> Dear all,
>
> I got directory 1.3.0.0 after installing GHC 8.0.2 RC2 (which hasn't
> been announced) using:
>
The source release went out last week. I'll likely announce the
availability of builds tomorrow or Wednesday.

Indeed GHC 8.0.2 will ship with directory 1.3; this is unfortunate, but
we concluded that this would be the only sensible option considering
there is a rather subtle change in semantics in this release (namely
fixing directory #63).

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Ben Gamari]

Johannes Waldmann  writes:

> Hi,
>
>> ... to compile it with a profiled GHC and look at the report?
>
> How hard is it to build hackage or stackage
> with a profiled ghc? (Does it require ghc magic, or can I do it?)
>
Not terribly hard although it could be made smoother.

To start you'll need to compile a profiled GHC. To do this you simply
want to something like the following,

 1. install the necessary build dependencies [1]
 2. get the sources [2]
 3. configure the tree to produce a profiled compiler:
   a. cp mk/build.mk.sample mk/build.mk
   b. uncomment the line `BuildFlavour=prof` in mk/build.mk
 4. `./boot && ./configure --prefix=$dest && make && make install`

Then for a particular package,

 1. get a working directory: `cabal unpack $pkg && cd $pkg-*`
 2. `args="--with-ghc=$dest/bin/ghc 
--allow-newer=base,ghc-prim,template-haskell,..."`
 3. install dependencies: `cabal install --only-dependencies $args .`
 4. run the build, `cabal configure --ghc-options="-p -hc" $args && cabal build`

You should end up with a .prof and .hp file. Honestly, I often skip the
`cabal` step entirely and just use `ghc` to compile a module of interest
directly.


[1] https://ghc.haskell.org/trac/ghc/wiki/Building/Preparation
[2] https://ghc.haskell.org/trac/ghc/wiki/Building/GettingTheSources


>> ... some obvious sub-optimal algorithms in GHC.
>
> obvious to whom? you mean sub-optimality is already known,
> or that it would become obvious once the reports are there?
>
I think "obvious" may have been a bit of a strong word here. There are
sub-optimal algorithms in the compiler and they can be found with a bit
of work. If you have a good testcase tickling such an algorithm finding
the issue can be quite straightforward; if not then the process can be a
bit trickier. However, GHC is just another Haskell program and
performance issues are approached just like in any other project.


> Even without profiling - does hackage collect timing information from
> its automated builds?
>
Sadly it doesn't. But...

> What needs to be done to add timing information in places like
> https://hackage.haskell.org/package/obdd-0.6.1/reports/1 ?
>
I've discussed the possibility with Herbert to add instrumentation in
his matrix builder [3] to collect this sort of information.

As a general note, keep in mind that timings are quite unstable,
dependent upon factors beyond our control at all levels of the stack.
For this reason, I generally prefer to rely on allocations, not
runtimes, while profiling.

As always, don't hesitate to drop by #ghc if you run into trouble.

Cheers,

- Ben


[3] http://matrix.hackage.haskell.org/packages


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Ben Gamari]

Michal Terepeta <michal.terep...@gmail.com> writes:

>> On Tue, Dec 6, 2016 at 2:44 AM Ben Gamari <b...@smart-cactus.org> wrote:
>>
>>I don't have a strong opinion on which of these would be better.
>>However, I would point out that currently the tests/perf/compiler tests
>>are extremely labor-intensive to maintain while doing relatively little
>>to catch performance regressions. There are a few issues here:
>>
>> * some tests aren't very reproducible between runs, meaning that
>>   contributors sometimes don't catch regressions in their local
>>   validations
>> * many tests aren't very reproducible between platforms and all tests
>>   are inconsistent between differing word sizes. This means that we end
>>   up having many sets of expected performance numbers in the testsuite.
>>   In practice nearly all of these except 64-bit Linux are out-of-date.
>> * our window-based acceptance criterion for performance metrics doesn't
>>   catch most regressions, which typically bump allocations by a couple
>>   percent or less (whereas the acceptance thresholds range from 5% to
>>   20%). This means that the testsuite fails to catch many deltas, only
>>   failing when some unlucky person finally pushes the number over the
>>   threshold.
>>
>> Joachim and I discussed this issue a few months ago at Hac Phi; he had
>> an interesting approach to tracking expected performance numbers which
>> may both alleviate these issues and reduce the maintenance burden that
>> the tests pose. I wrote down some terse notes in #12758.
>
> Thanks for mentioning the ticket!
>
Sure!

> To be honest, I'm not a huge fan of having performance tests being
> treated the same as any other tests. IMHO they are quite different:
>
> - They usually need a quiet environment (e.g., cannot run two different
>   tests at the same time). But with ordinary correctness tests, I can
>   run as many as I want concurrently.
>
This is absolutely true; if I had a nickel for every time I saw the
testsuite fail, only to pass upon re-running I would be able to fund a
great deal of GHC development ;)

> - The output is not really binary (correct vs incorrect) but some kind of a
>   number (or collection of numbers) that we want to track over time.
>
Yes, and this is more or less the idea which the ticket is supposed to
capture; we track performance numbers in the GHC repository in git
notes and have Harbormaster (or some other stable test environment)
maintain them. Exact metrics would be recorded for every commit and we
could warn during validate if something changes suspiciously (e.g. look
at the mean and variance of the metric over the past N commits and
squawk if the commit bumps the metric more than some number of sigmas).

This sort of scheme could be implemented in either the testsuite or
nofib. It's not clear that one is better than the other (although we
would want to teach the testsuite driver to run performance tests
serially).

> - The decision whether to fail is harder. Since output might be noisy, you
>   need to have either quite relaxed bounds (and miss small
>   regressions) or try to enforce stronger bounds (and suffer from the
>   flakiness and maintenance overhead).
>
Yep. That is right.

> So for the purpose of:
>   "I have a small change and want to check its effect on compiler
>   performance and expect, e.g., ~1% difference"
> the model running of benchmarks separately from tests is much nicer. I
> can run them when I'm not doing anything else on the computer and then
> easily compare the results. (that's what I usually do for nofib). For
> tracking the performance over time, one could set something up to run
> the benchmarks when idle. (isn't that's what perf.haskell.org is
> doing?)
>
> Due to that, if we want to extend tests/perf/compiler to support this
> use case, I think we should include there benchmarks that are *not*
> tests (and are not included in ./validate), but there's some easy tool
> to run all of them and give you a quick comparison of what's changed.
>
When you put it like this it does sound like nofib is the natural choice
here.

> To a certain degree this would be then orthogonal to the improvements
> suggested in the ticket. But we could probably reuse some things
> (e.g., dumping .csv files for perf metrics?)
>
Indeed.

> How should we proceed? Should I open a new ticket focused on this?
> (maybe we could try to figure out all the details there?)
>
That sounds good to me.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Measuring performance of GHC

[Ben Gamari]

Johannes Waldmann  writes:

> Hi Ben, thanks,
>
>
>>  4. run the build, `cabal configure --ghc-options="-p -hc" $args && cabal 
>> build`
>
> cabal configure $args --ghc-options="+RTS -p -hc -RTS"
>
Ahh, yes, of course. I should have tried this before hitting send.

>> You should end up with a .prof and .hp file.
>
> Yes, that works. - Typical output starts like this
>
> COST CENTRE   MODULE   %time %alloc
>
> SimplTopBinds SimplCore 60.7   57.3
> OccAnal   SimplCore  6.06.0
> Simplify  SimplCore  3.00.5
>
Ahh yes. So one of the things I neglected to mention is that the
profiled build flavour includes only a few cost centers. One of the
tricky aspects of the cost-center profiler is that it affects
core-to-core optimizations, meaning that the act of profiling may
actually shift around costs. Consequently by default the 
the build flavour includes a rather conservative set of cost-centers to
avoid distorting the results and preserve compiler performance.

Typically when I've profiled the compiler I already have a region of
interest in mind. I simply add `OPTIONS_GHC -fprof-auto` pragmas to the
modules involved. The build system already adds this flag to a few
top-level modules, hence the cost-centers which you observe (see
compiler/ghc.mk; search for GhcProfiled).

If you don't have a particular piece of the compiler in mind to study,
you certainly can just pepper every module with cost centers by adding
-fprof-auto to GhcStage2HcOpts (e.g. in mk/build.mk). The resulting
compiler may be a bit slow and you may need to be just a tad more
careful in evaluating the profile.

It might be nice if we had a more aggressive profiled build flavour
which added cost centers to a larger fraction of machinery of the
compiler, which excluding low-level utilities like FastString, which are
critical to the compiler's performance.

>
> These files are always called ghc.{prof,hp},
> how could this be changed? Ideally, the output file name
> would depend on the package being compiled,
> then the mechanism could probably be used with 'stack' builds.
>
We really should have a way to do this but sadly do not currently.
Ideally we would also have a way to change the default eventlog
destination path.

> Building executables mentioned in the cabal file will
> already overwrite profiling info from building libraries.
>
Note that you can instruct `cabal` to only build a single component of a
package. For instance, in the case of the `text` package you can build
just the library component with `cabal build text`.

> When I 'cabal build' the 'text' package,
> then the last actual compilation (which leaves
> the profiling info) is for cbits/cbits.c
>
Ahh right. Moreover, there is likely another GHC invocation after that
to link the final library. This is why I typically just use GHC
directly, perhaps stealing the command line produced by `cabal` (with
`-v`).

> I don't see how to build Data/Text.hs alone
> (with ghc, not via cabal), I am getting
> Failed to load interface for ‘Data.Text.Show’
>
Hmm, I'm not sure I see the issue. In the case of `text` I can just run
`ghc` from the source root (ensuring that I set the #include path with
`-I`),

$ git clone git://github.com/bos/text
$ cd text
$ ghc Data/Text.hs -Iinclude


However, some other packages (particularly those that make heavy use of
CPP) aren't entirely straightforward. In these cases I often find myself
copying bits from the command line produced by cabal.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC 8.2.1 status

[Ben Gamari]

Hello everyone,

Here is your periodic release update. After a number of false-starts,
the tree is finally approaching a releasable state. Last week I cut
the ghc-8.2 branch and things have been stabilizing ever since. At this
point we are waiting on only a couple more fixes before the rc1 source
release can be cut,

 * #13474: To be addressed by Richard soon
 * #13410: To be addressed by Simon PJ shortly
 * #1: To be addressed by Ben shortly
 * #13433: Simon Marow has a fix

There are also a few issues that we have deemed not to block -rc1, but
which must be addressed prior to the final release,

 * #13220: David will verify that 
 * #13233: Ben will do a bit more digging, Simon PJ will think about it

With luck we will be able to finish the -rc1 blockers in the next couple
of days, allowing us to cut a source release by Tuesday or Wednesday.

Cheers,

 - Your friendly release manager


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.2.1 status

[Ben Gamari]

George Colpitts  writes:

> Thanks Ben. When you have a chance it would be good to update the status
> page  wrt what we
> will be getting, specifically will Backpack be in 8.2.1
>
Hi George,

I just reviewed the status page and cleaned a few things up. Otherwise it looks 
up to date.

Regarding Backpack, all of the major pieces are currently in the tree
and will be present in 8.2. It would be great if users could play these
bits in -rc1; there has been a steady stream of fixes from Edward in the
past weeks, so there is no doubt more work to do here and it would be
great to find issues before the final release if possible.

Edward: Do you think you could add some details to the Backpack entry on
the status page?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [commit: ghc] master: rts linker: Introduce MachOTypes (8ed29b5)

[Ben Gamari]

Moritz Angermann  writes:

> Hi Gabor,
>
> thanks! And sorry for causing trouble.  I’ll look into the MachO
> case.
>
Thanks for handling this Gabor and Moritz!

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Building head on Openbsd

[Ben Gamari]

Karel Gardas  writes:

> On 03/26/17 04:54 PM, Sergei Trofimovich wrote:
>> On Sun, 26 Mar 2017 15:08:50 +0200
>> Karel Gardas  wrote:
>>
>>> On 03/26/17 03:04 PM, Sergei Trofimovich wrote:
 I guess openbsd does not have definition of EM_PPC64 (yet?).
>>>
>>> IIRC it does not. I even remembering to fix this in the past but then
>>> probably forgotten to submit patch...
>>>
>>> Anyway, attempting to duplicate on outdated OpenBSD current from Dec
>>> 2016, but still post 6.0 code so this should be ok...
>>
>> I've tested the following patch
>>  
>> https://git.haskell.org/ghc.git/commitdiff/6c73504ac5f4e951062d5e868fa2b69b028b6e79
>> on amd64-unknown-openbsd6.0.
>>
>> Hope it does not breaks things for you.
>
> Well, my patch looks exactly like yours. Anyway, thanks for such fast 
> solution and commit.
>
I have cherry-picked the fix into 8.2 as
d4694b85be23c8a014225271060765c062502ed2.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

D3316 status

[Ben Gamari]

Hi Richard,

It took me a bit longer than expected to get back to D3316 but I think I
am now done. It wasn't quite as trivial as I thought it might be, so I
thought I would summarize what was needed,

 1. tcSplitTyConApp_maybe and tcRepSplitTyConApp_maybe have been moved
from TcType to Type to avoid import loops. I'm not terribly happy
with this change, but I spent a fair amount of time looking for
alternatives and it seems this is the least evil. Moreover, there is
precedent for this in tcRepSplitAppTy_maybe. It would be nice to
refactor TcType and friends to avoid this loop.

 2. TypeMap now respects the Constraint/Type distinction. I believe this
is safe for the TypeMap uses in the typechecker itself. The only
other use AFAICT is Specialise.CallKeySet, which I believe should
also be fine under this change.

 3. TcInteract.matchTypeable now handles Constraint explicitly.

With all of this the T11715 test passes. I've uploaded my changes
relative to your patch as D3396 for your review.

However, I am a bit concerned with (2) as it runs contrary to your
original implementation, which changed coreViewOneStarKind to coreView.
What was the reasoning behind this?

The motivation for moving to tcView here is the dictionary cache, which
uses a TypeMap. Namely, if we solve for `Typeable Type`, and later look
for `Typeable Constraint` in the dictionary cache, we will incorrectly
find the `Typeable Type` dictionary.

Thoughts?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Stat too good

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> Ben,
> I still get these four stat-too-good "failures" on 64-bit Linux.
>
> Unexpected stat failures:
>
>/tmp/ghctest-ca0gfq/test   spaces/./perf/compiler/T13035.run  T13035 [stat 
> too good] (normal)
>
>/tmp/ghctest-ca0gfq/test   spaces/./perf/compiler/T12425.run  T12425 [stat 
> too good] (optasm)
>
>/tmp/ghctest-ca0gfq/test   spaces/./perf/compiler/T1969.run   T1969 [stat 
> too good] (normal)
>
>/tmp/ghctest-ca0gfq/test   spaces/./perf/compiler/T9233.run   T9233
>[stat too good] (normal)
>
> Don't you?

I'm not seeing anything like this at the moment but I have seen a great
deal of variability in testsuite results recently. In fact, I had to bump the
acceptance window size of T4029 as my local machine, Harbormaster, and
the OS X build bot differed by nearly 10% in max_bytes_used.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Why are there no Show instances for internal types

[Ben Gamari]

Tom Sydney Kerckhove  writes:

> On 19-03-17 02:08:56, Rahul Muttineni wrote:
>> Syd, can you tell us what kind of things you were trying to print out?
>
> Maybe I wasn't very clear.
> I'm trying to visualise the internal structure of some of the
> typechecker's output.
> I specifically do NOT need to see the output of Outputable's functions.
> They show the human-readibly version and not the internal structure.
>
Indeed I am sympathetic to this request. In my time working on GHC I
have written raw several variants of `Type -> SDoc`, each exposing
various levels of detail. These are handy and can be a good way to gain
insight into the AST, but I feel like it is hard to come up with
something that is generally applicable; I find each time I need to
expose slightly different details about the internal structure of the
representation.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Adding a module

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> If you add a module to GHC, I know you need to extend ghc.cabal.in.   But 
> also ghc.mk? Otherwise I get hits
>
>
> If you add a module to GHC, I know you need to extend ghc.cabal.in.   But 
> also ghc.mk? Otherwise I get hits
> /usr/bin/ar: creating rts/dist/build/libHSrts_debug.a
>
> Reachable modules from DynFlags out of date
>
> Please fix compiler/ghc.mk, or building DLLs on Windows may break (#7780)
>
> Extra modules: CoreOpt
>
> make[1]: *** [compiler/stage2/dll-split.stamp] Error 1
>
> make[1]: *** Waiting for unfinished jobs
>
> make: *** [all] Error 2
>
>
> Is it necessary to have two places to extend?
>
Well, I suppose we might be able to compute the list of accessible
modules instead of listing them by hand. However, I'm not sure that this
would be worth the complexity it would introduce.

Moreover, I believe that Tamar is working towards being able to entirely
lift the need for this list (please correct me if I'm wrong, Tamar). If
things go well his work should be in 8.4.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Why are there no Show instances for internal types

[Ben Gamari]

Rahul Muttineni  writes:

> I think another way to go about this problem is to figure out an
> alternative to baking in DynFlags to SDocContext (which I feel is the core
> problem here). The only use of those DynFlags is via sdocWithDynFlags and
> 94 call sites use them.
>
Indeed, I would love to see this happen. This exact request is being
tracked as #10143.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Why are there no Show instances for internal types

[Ben Gamari]

On March 18, 2017 9:03:48 AM EDT, Tom Sydney Kerckhove 
 wrote:

Snip.
>
>My questions for you:
>
>- Is there a reason that there are no derived 'Show' instances for most
>  types?

As Richard mentioned, we don't derive Show due to code size and compilation 
time concerns. Show in particular is rather expensive to derive and seeing as 
we already have Outputable I don't it would make sense to derive it by default. 
I would really like to avoid introducing more CPP into the code base for this 
particular problem.

One alternative which will work in many cases is to simply derive Show yourself 
using StandaloneDeriving. Does this help?

Cheers,

- Ben 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

Siddhanathan Shanmugam  writes:

>> I would be happy to advise if you would like to pick this up.
>
> Thanks Ben!
>
>> This would mean that Haskell libraries compiled with different flags
>> would not be ABI compatible.
>
> Wait, can we not maintain ABI compatibility if we limit the target
> features using a compiler flag? Sometimes (for performance reasons)
> it's reasonable to request the compiler to only generate SSE
> instructions, even if AVX2 is available on the target. On GCC we can
> use the flag -msse to do just that.
>
I think the reasoning here is the following (please excuse the rather
contrived example): Consider a function f with two variants,

module AvxImpl where
{-# OPTIONS_GHC -mavx #-}
f :: DoubleX4# -> DoubleX4# -> Double

module SseImpl where
{-# OPTIONS_GHC -msse #-}
f :: DoubleX4# -> DoubleX4# -> Double

If we allow GHC to pass arguments with SIMD registers we now have a bit
of a conundrum: The calling convention for AvxImpl.f will require that
we pass the two arguments in YMM registers, whereas SseImpl.f will
be via passed some other means (perhaps two pairs of XMM registers).

In the C world this isn't a problem AFAIK since intrinsic types map
directly to register classes. Consequently, I can look at a C
declaration type,

double f(__m256 x, __m256 y);

and tell you precisely the calling convention that would be used. In
GHC, however, we have an abstract vector model and therefore the calling
convention is determined by which ISA the compiler is targetting.

I really don't know how to fix this "correctly". Currently we assume
that there is a static mapping between STG registers and machine
registers. Giving this up sounds quite painful.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

Edward Kmett  writes:

> Hrmm. In C/C++ I can tell individual functions to turn on additional ISA
> feature sets with compiler-specific __attribute__((target("avx2"))) tricks.
> This avoids complains from the compiler when I call builtins that aren't
> available at my current compilation feature level. Perhaps pragmas for the
> codegen along those lines is what we'd ultimately need? Alternately, if we
> simply distinguish between what the ghc codegen produces with one set of
> options and what we're allowed to ask for explicitly with another then
> user-land tricks like I employ would remain sound.
>
I'm actually not sure that simply distinguishing between the user- and
codegen-allowed ISA extensions is quite sufficient. Afterall, AFAIK LLVM
doesn't make such a distinction itself: AFAIK if you write a vector
primitive and compile for a target that doesn't have an appropriate
instruction the code-generator will lower it with software emulation.

However, adding a pragma to allow per-function target annotations seems
quite reasonable and easily doable. Moreover, contrary to my previous
assertion, it shouldn't require any splitting of compilation units. I
ran a quick experiment, compiling this program,

__attribute__((target("sse2"))) int hello() {
  return 1; 
}

With clang. It produced something like,

define i32 @hello() #0 {
  ret i32 1
}

attributes #0 = { "target-cpu"="x86-64" 
"target-features"="+fxsr,+mmx,+sse,+sse2,+x87" ... }

So it seems LLVM is perfectly capable of expressing this; in hindsight
I'm not sure why I ever doubted this.

There are a number of details that would need to be worked out regarding
how such a pragma should behave. Does the general direction sound
reasonable? I've opened #13427 [1] to track this idea.

Cheers,

- Ben


[1] https://ghc.haskell.org/trac/ghc/ticket/13427


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

It's a bit unclear from this comment whether this statement is a
critique of a particular implementation strategy for adding SIMD support
to the NCG or a more general reflection on SIMD interfaces. From your
later messages I infer the latter in my response; feel free to disregard
if I misinterpreted.

Carter Schonwald  writes:

> agreed. and the generic vector size stuff in llvm is both pretty
> naive, AND not the sane/tractable way to add SIMD support to the NCG,
>
I don't see why this is true. I think it's fair to say that the LLVM
folks have put a lot more thought into SIMD support than any of us here;
consequently I tend to put a fair amount of trust in what they have to
say about the matter. Moreover, it seems to me like they came up with a
pretty sensible abstraction from which they can produce very good code.

Is the abstraction perfect? Of course not; they poke holes where
necessary to expose truly platform specific functionality. However, it
seems they rarely find it necessary to use these holes: In playing
around with Clang I found that almost all of the standard vector
operations lowered to the "naive" abstract operations.

I don't see why we can't provide a similar approach: provide abstract
types and some basic operations (as we already do), supplemented with
tailored primops far target-specific functionality.

My generally, I think we should have a very good reason before we go off
and chart our own course here.

> i'm totally ok with my vector sizes that are available depending on the
> target CPU or whatever. Operating systems have very sane errors for that
> sort of mishap,
>
If the user wants to be more careful about using precisely the vector
support that their target offers then that is their perogative. Unless
I'm missing something there is nothing stopping them under the current
scheme.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

Carter Schonwald  writes:

> No matter *how* ghc ultimately bundles simd for high level
> programming, it *will* have to bottom out into these target specific
> operations at code gen time, and LLVM is *not* an abstraction for it.
>
I am very interested to hear what you mean by this; please do elaborate.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: PSA: perf.haskell.org/ghc temporarily out of order

[Ben Gamari]

Joachim Breitner  writes:

> Hi,
>
> a recent change to nofib
> (https://phabricator.haskell.org/rNOFIB313812d319e009d698bc1a4d2e8ac26d4dfe3c0a)
> broke the perf.haskell.org builder, so we won’t be getting perf
> warnings until that is fixed.
>
I've pushed the michalt's fix. Thanks for the quick turnaround, michalt!

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

Edward Kmett  writes:

> Currently if you try to use a DoubleX4# and don't have AVX2 turned on, it
> deliberately crashes out during code generation, no?

I very well be missing something, but I don't believe this is true. This
program compiles just fine with merely -fllvm -msse,

{-# LANGUAGE MagicHash #-}
{-# LANGUAGE UnboxedTuples #-}
module Hi where
import GHC.Prim
import GHC.Float

addIt :: DoubleX4# -> DoubleX4# -> DoubleX4#
addIt x y = plusDoubleX4# x y
{-# NOINLINE addIt #-}

It produces the following assembler,,

movupd 0x10(%rbp),%xmm0
movupd 0x0(%rbp),%xmm1
movupd 0x30(%rbp),%xmm2
movupd 0x20(%rbp),%xmm3
addpd  %xmm1,%xmm3
addpd  %xmm0,%xmm2
movupd %xmm2,0x30(%rbp)
movupd %xmm3,0x20(%rbp)
mov0x40(%rbp),%rax
lea0x20(%rbp),%rbp
jmpq   *%rax

The reason for this is that the LLVM code generator just blindly
translates DoubleX4# to LLVM's <4 x double> type. The LLVM code
generator then does whatever it can to produce the code we ask of it,
even if the target doesn't have support for this vector variety.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

Carter Schonwald  writes:

> solution: lets call these registers what they are, instead of pretending
> they're portable. we are not going to find the right abstraction in the
> first go. lets not do that. first get it working sanely, then figure out
> proper abstractions
>
I'm not sure I understand what you are suggesting here. Are you
suggesting we rename the types and primops in the Haskell interface?
Some deeper change in semantics? Their treatment in the compiler
backend? Something else entirely?

I'm lost.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Translation of GHC typechecker output to haskell-src-exts's 'Type'

[Ben Gamari]

Tom Sydney Kerckhove  writes:

> Dear GHC Devs,

Hi Tom,

> Because of the unwieldy nature of the data that the GHC type checker
> outputs, I am trying to convert a GHC 'Type' [1] to a haskell-src-ext
> 'Type' [2].
>
> The translation does not need to be perfect for now, but I would at
> least like to be able to translate function types and types that involve
> type-class constraints. (See my initial attempt in attachment)
>
> Has this ever been done before?

I'm not aware of any implementations, but I can't claim to have seen
every usage of haskell-src-exts.

> Could you point me to some documentation on GHC's 'Type' [1] that could
> help me with writing this function? (The comments in code aren't nearly
> enough for me.)
>
I'm afraid the notes in TyCoRep are really all we have. Note that you
should also familiarize yourself with the TyCon type (for reasons you'll
see below).

> In particular, I am having trouble finding type class constraints in the
> 'Type'.
>
During typechecking class constraints are turned into dictionary
arguments. If I'm not mistaken you can pick these out by looking for
AlgTyCons (a constructor of the TyCon type, see TyCon.hs) with a
algTcParent matching (ClassTyCon cls _). cls will be the name of the
associated class.

I hope this helps.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC 8.2.1-rc1 source tarball availability

[Ben Gamari]

tl;dr: If you would like to produce a binary distribution for GHC
   8.2.1-rc1 then let me know, grab the source distribution and
   start building. The binary distributions will be announced one
   week from today.

Hello GHC packagers,

I am happy to announce the release of the 8.2.1-rc1 source distribution
to binary packagers. You will find the usual source artifacts at

http://downloads.haskell.org/~ghc/8.2.1-rc1/

As usual, the sooner we can get the binary distributions together the
better, but I will hold off on announcing the distributions until next
Sunday to ensure we're all on the same page. It would be appreciated if
you could reply to this message confirming that you intend to offer a
binary distribution this release.

Otherwise, let me know if you have any trouble building your
distribution. I have yet to push the ghc-8.2.1-rc1 tag in case we
encounter unexpected issues but all of my builds with this tarball
thusfar have gone swimmingly save a few known issues (namely #13426,
#13233, and #13509).

Good luck and thanks for all of your work!

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC 8.2.1-rc1 source tarball availability

[Ben Gamari]

Alan & Kim Zimmerman  writes:

> There is no tag in the source tree, which commit has been used?
>
As I mention in the message, I intentionally held off on pushing the tag
to avoid the need to force push in case this commit ends up being bad.
The tarball was produced from d67f0471cd3584c5a548ff30c9023b599b598d3e.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+)

[Ben Gamari]

Phyx  writes:

> We have discussed a few options on what to do, but haven't gotten a
> conclusion/concensus yet.
>
Tamar and I have discussed this on IRC in the passed. I tend to agree
that we will need to, at the very least, put out a new 8.0.2. I'm not
sure about 7.10. Moreover, there is reportedly a non-negligible fraction
of users still using 7.8 releases, although I'm not keen on going back
that far given that this was pre-submodules.

> Last we talked one option was presented to just re-tar the 7.10.3 and 8.0.2
> tarballs with the fix. The gcc wrapper does not affect code generation of
> the compiler, it's just there to correct paths that are hard-coded in gcc
> because it was built to be used in MSYS2.
>
> 7.10.3 would be needed since we still support bootstrapping using it. This
> would be the easiest solution.
>
> Ben what do you think about changing the already released tarballs? Or
> would you prefer a full new release?
>
We certainly can't change the existing tarballs. However, we could just
push out a 7.10.4 (or perhaps 7.10.3b, I'm unsure) release with binary
distributions for Windows only. There are a few unrelated patches on the
ghc-7.10 branch that weren't present in 7.10.3, but I believe they are
quite low-risk.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows 10 Creators Update will break GHC (Was FW: GHC on Windows 10 15019+)

[Ben Gamari]

Ryan Trinkle  writes:

> I would strongly urge that any new tarballs be released under new names,
> and that the old tarballs be kept in place.  Changing existing tarball URLs
> silently causes breakage for Nix, and, I would imagine, for any other build
> systems that are particularly concerned with reproducibility.  Nix packages
> based on GHC use an SHA-256 hash of the tarball to check it when
> downloading, which of course will be broken by any change, no matter how
> minor.
>
I completely agree here; I avoid updating existing artifacts whenever
possible.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 8.2.1 release candidate 1

[Ben Gamari]

"Boespflug, Mathieu"  writes:

> Hi Ben,
>
> this is great news! I'm particularly keen on learning more about two points
> you mentioned in your email:
>
> * Compiler performance: do you have any numbers to quantify what 8.0 vs 8.2
> is likely to look like?

I'm afraid the best I can provide at the moment is [1]. On closer
inspection of these I'm a bit suspicious of the 8.0 numbers; I'll try to
reproduce them (and characterize the current ghc-8.2 branch, which fixes
a significant memory leak present in -rc1) shortly. That being said,
there were a few major fixes in 8.2.

> How much has the work that's been done affect performance across the
> board? Or are these mostly pathological cases (e.g. ghc --make with
> high number of cores, large number of definitions in a module, large
> amount of let nesting in definitions, etc)

The fixes span the gamut but I generally tried to concentrate on issues
that looked like they might affect a large fraction of programs. I fixed
at least one major regression present in 8.0 which significantly
inflated allocations of typeclass instance matching. I've also done
quite a bit of refactoring around our handling of names. These include
improvements in interface file serialization efficiency and name lookup.
These are just some of the major reworks; there are also numerous
smaller fixes which I don't have time to cover here.

Compilation performance has also generally improved as a result of some
of the work in the simplifier. In particular, GHC now performs an early
inlining phase which, quite surprisingly, tends to result in smaller
programs earlier in simplification, reducing the amount of work that the
compiler needs to carry out. Simon summarized some of his preliminary
numbers here [2].

> * DWARF support: could you clarify at a very high-level what typical uses
> cases can be expected to work and which ones won't? Would be eager to read
> any resources you could point me at to help me understand what still needs
> to be done on this front.
>
At this point if GHC compiles your program with -g you should have a
pretty good assurance that the resulting DWARF information is
reasonable. This means that users can use the ExecutionStack mechanism,
RTS stack-trace functionality, and GDB without fear of the act of
capturing a stacktrace leading to a crash.

After starting to write more here, I realized that you will likely not
be the last person to ask about this and updated the DWARF status page
with additional discussion [3]. Let me know if you have any questions.

Cheers,

- Ben

[1] https://gist.github.com/bgamari/fbd3e55047bd041d8208ebe820c0f493
[2] https://mail.haskell.org/pipermail/ghc-devs/2017-February/013818.html
[3] https://ghc.haskell.org/trac/ghc/wiki/DWARF/Status


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

GHC and linker choice

[Ben Gamari]

Niklas Hambüchen  writes:

> I have some suggestions for low hanging fruits in this effort.
>
> 1. Make ghc print more statistics on what it spending time on
>
> When I did the linking investigation recently
> (https://www.reddit.com/r/haskell/comments/63y43y/liked_linking_3x_faster_with_gold_link_10x_faster/)
> I noticed (with strace) that there are lots of interesting syscalls
> being made that you might not expect. For example, each time TH is used,
> shared libraries are loaded, and to determine the shared library paths,
> ghc shells out to `gcc --print-file-name`. Each such invocation takes 20
> ms on my system, and I have 1000 invocations in my build. That's 20
> seconds (out of 2 minutes build time) just asking gcc for paths.
>
> I recommend that for every call to an external GHC measures how long
> that call took, so that it can be asked to print a summary when it's done.
>

> That might give us lots of interesting things to optimize. For example,
> This would have made the long linker times totally obvious.
>
For what it's worth, some of us have recognized for quite some time that
BFD ld is a known slow spot in the compilation pipeline. However, up
until now we have considered this to be more of an issue to be handled
at the packaging level than in GHC issue. Currently, GHC relies on `gcc`
to link and gcc has its own idea of the default linker. Many users (e.g.
Debian packagers, users who are cross compiling) are quite unhappy when
GHC coerces the system toolchain into changing its default behavior.

Consequently we have maintained the status quo and silently wished that
Linux distributions would start moving to gold by default. Sadly there
continues to be little progress on this matter.

However, given that so many users are now relying on GHC binary
distributions and that BFD ld is an increasingly significant issue as
dependency counts increase, I think the status quo is increasingly
untenable. I have proposed (#13541) that we introduce configure
logic to use gold when available. There are outstanding questions however

 * What interface (i.e. configure flags) do we expose to the user to
   allow them to turn on or off this logic?

 * Do we want to enable the logic by default in binary
   distributions to ensure that users see the benefits of gold if it is
   available?

 * Do we want to also enable it in source distributions as well? If yes
   then developers may be caught by surprise; if no then there will be
   an inconsistency between the default configuration of the source tree
   and binary distributions (although there is plenty of precedent for this).

Anyways, let me know if you have thoughts on any of this. Hopefully
we'll be able to get this done for 8.4.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: FW: [Haskell-cafe] [Call for Contributions] Haskell Communities and Activities Report, May 2017 edition (32nd edition)

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> Ben 
>
> Will you coordinate the GHC status report for HCAR?
>
Of course.

> Everyone: please don’t leave Ben doing all the heavy lifting. He'll
> sketch a structure but it's up to the rest of us to fill it in. These
> six monthly reports are widely read, I think.
>
Thanks!

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Translation of GHC typechecker output to haskell-src-exts's 'Type'

[Ben Gamari]

Tom Sydney Kerckhove  writes:

> On 02-04-17 14:40:05, Tom Sydney Kerckhove wrote:
>> Is there a way to access the types before this translation happens?
>> It's okay if I have to assume that type-checking succeeds...
>
> I have found a way to do what I want, even after this translation.
> It relies on the fact that type class constraints always occur on the
> left side of the result of splitFunTy.
>
Yep, that is what I would do as well.

> Thank you for your help!
>
No worries and good luck.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: testsuite not in GHC 8.2.1-rc1 source tarball ?

[Ben Gamari]

George Colpitts  writes:

> Thanks Brandon
>
> After downloading the source tarball and doing a build successfully I
> wanted to run the testsuite.
>
You can indeed run the testsuite.

However, note that the testsuite is not included in the "-src" tarball
to keep the distribution size down. There is a separate "-testsuite"
tarball which includes the testsuite/ subtree. This can be extracted
into the parent directory of the source tree. You should find this
tarball in the usual place [1].

Cheers,

- Ben


[1] 
https://downloads.haskell.org/~ghc/8.2.1-rc1/ghc-8.2.0.20170404-testsuite.tar.xz


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Phabricator upgrade tonight

[Ben Gamari]

Ben Gamari <b...@well-typed.com> writes:

> Ben Gamari <b...@well-typed.com> writes:
>
>> Unless someone objects I will be performing an upgrade of Phabricator
>> tonight. I'll likely start around 23:00 EST (4:00 UTC). As usual,
>> Phabricator may be down for anywhere from 10 minutes to a few hours,
>> depending upon how things go.
>>
> As there has been no objection I'll be bringing down Phabricator in
> about ten minutes for the upgrade.
>
Phabricator is once again up. As usual, let me know if anything has
broken.

I should note that thanks to mpickering's valiant efforts the GHC Trac
Tickets Differential field is now once again working. Thanks Matthew!

Chers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Phabricator upgrade tonight

[Ben Gamari]

Ben Gamari <b...@well-typed.com> writes:

> Unless someone objects I will be performing an upgrade of Phabricator
> tonight. I'll likely start around 23:00 EST (4:00 UTC). As usual,
> Phabricator may be down for anywhere from 10 minutes to a few hours,
> depending upon how things go.
>
As there has been no objection I'll be bringing down Phabricator in
about ten minutes for the upgrade.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: linux build broken

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> My linux build is broken.  See below.  Help!

All of this is due to the recent CPP cleanup, unfortunately. I think
I will revert it for now and we can work out the issues elsewhere.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[ANNOUNCE] GHC 8.2.1 release candidate 1

[Ben Gamari]

Hello everyone,

The GHC team is very pleased to announce the first candidate of the
8.2.1 release of the Glasgow Haskell Compiler. Source and binary
distributions are available at

https://downloads.haskell.org/~ghc/8.2.1-rc1/

This is the first of a few release candidates leading up the final 8.2.1
release. This release will feature,

  * A new type-indexed Typeable implementation

  * The long awaited Backpack

  * Deriving strategies for disambiguating DeriveAnyClass,
GeneralizedNewtypeDeriving, and stock mechanisms

  * Overloaded record fields

  * Improved compiler performance

  * Better code generation through more robust tracking of join points

  * Compact regions for more efficient garbage collection and serialization

  * Better support for machines with non-uniform memory architectures

  * More robust support for levity (e.g. RuntimeRep) polymorphism

  * A simple interface for streaming eventlog data from live processes

  * Further refinement of DWARF support

Unfortunately there are still a few known issues in this release,
including a few compiler panics (#13233, #13509) and a memory leak in
the simplifier (#13426) which may adversely affect heap sizes and
compilation time for some modules. This memory leak unfortunately made
it impossible to provide a 32-bit Windows distribution for this
candidate; this will be resolved in -rc2.

As always, please let us know if you have difficulty. Thanks to everyone
who has contributed to this release!

Happy testing,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 8.2.1 release candidate 1

[Ben Gamari]

Phyx  writes:

> Hi Ben,
>
> The windows build is unusable. The settings file has $TopDir expanded and
> hard-coded to the build path on drydock.
>
Hmm, this is very odd and sounds like a build system bug since I used
the same scripts as usual to generate this bindist. I'll investigate.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: testsuite not in GHC 8.2.1-rc1 source tarball ?

[Ben Gamari]

George Colpitts  writes:

> Are there any guidelines on how much memory is required to run the tests?
> On my Mac I have 12 GB of RAM and a few hundred gigabytes of free disk
> space. The first time I did
>
>
>- make THREADS=4 test
>
> and I got a message
>
>
>- Your system has run out of application memory
>
> so I reran with
>
>- make test
>
> and left it running unattended. My machine crashed about an hour later.
>
Well, unfortunately one of the changes that just barely missesd the
window for -rc1 was a fix to a rather serious memory leak in the compiler
(#13426). I suspect this is the reason you are seeing such high memory
footprints.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: testsuite not in GHC 8.2.1-rc1 source tarball ?

[Ben Gamari]

Erik de Castro Lopo <mle...@mega-nerd.com> writes:

> Ben Gamari wrote:
>
>> Well, unfortunately one of the changes that just barely missesd the
>> window for -rc1 was a fix to a rather serious memory leak in the compiler
>> (#13426).
>
> Thats going to fixed for rc2?
>
Yes, the patch is already in the ghc-8.2 branch.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: downloads.haskell.org unavailable for a bit

[Ben Gamari]

Ben Gamari <b...@well-typed.com> writes:

> Ben Gamari <b...@well-typed.com> writes:
>
>> Hello all,
>>
>> Unfortunately, due to a botched rsync, the tarballs on
>> downloads.haskell.org will be unavailable for a few hours. I'm currently
>> working on restoring the 8.0.2 tarballs first to ensure that they remain
>> available to users. I will follow up with status updates as they come.
>> Many apologies for the inconvenience.
>>
> The 8.0.2 tarballs are now again available.
>
All of the tarballs are now once again available. Sorry again for the
inconvenience!

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: ghc-stage2 —interactive segfaults on Mac OS X 10.11.6 (build flavour = prof)

[Ben Gamari]

On April 14, 2017 3:33:21 PM EDT, Alfredo Di Napoli 
 wrote:
>Ok, I had success by removing “-debug” in favour of “-DDEBUG”. After
>compiling GHC I fired GDB and this is the output:
>
>Starting program:
>/Users/adinapoli/programming/haskell/ghc/inplace/lib/bin/ghc-stage2
>-B/Users/adinapoli/programming/haskell/ghc/inplace/lib --interactive
>GHCi, version 8.3.20170413: http://www.haskell.org/ghc/  :? for help
>[New Thread 0x120f of process 19786]
>[New Thread 0x1403 of process 19786]
>[New Thread 0x1503 of process 19786]
>[New Thread 0x1603 of process 19786]
>
>Thread 1 received signal SIGSEGV, Segmentation fault.
>0x000104cdd81a in ocInit_MachO () at rts/linker/MachO.c:141
>141 if(NULL != oc->info->nlist) {
>
>Maybe it does ring a bell to any of you. In case not, I’m happy to
>continue
>digging.
>
It doesn't ring a bell and I'm not near a computer at the moment, but it sounds 
like you are hot on the trail; that is a far more tractable error. 

I'll have a look at the code to see if I can see any obvious things to check 
when I get back home.


___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[ANNOUNCE] GHC tarballs for Windows 10 Creators Update

[Ben Gamari]

Hello GHC users,

Over the next few weeks Microsoft will be pushing out an update (the
so-called Creators Update) to Windows 10 which will break the GCC
toolchain shipped with all GHC releases prior to the yet-to-be-released
8.2.1 (see Trac [1] for details).

Over the last few weeks our resident Windows expert Tamar and I have
been working on preparing a set of patched Windows tarballs with
retrofitted GCC toolchains covering GHC 7.2.1 through 8.0.2. These
tarballs are now available on downloads.haskell.org. The new tarballs
are distinguished from the original releases with a `-win10` suffix. For
instance, the 64-bit 8.0.2 binary distribution can be found at,


https://downloads.haskell.org/~ghc/8.0.2/ghc-8.0.2-x86_64-unknown-mingw32-win10.tar.xz

Other than an updated mingw/bin/gcc.exe, these are identical to the
original releases. More details about the patched toolchain can be found
in Tamar's ghc-compat repository [2]. As always, let me know if you
encounter trouble.

Finally, we should all thank Tamar for his work on this; without his
effort GHC-on-Windows would be in a sad state indeed. Thanks Tamar!

Cheers,

- Ben


[1] https://ghc.haskell.org/trac/ghc/ticket/13411
[2] https://github.com/Mistuke/ghc-compat


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC tarballs for Windows 10 Creators Update

[Ben Gamari]

Niklas Larsson  writes:

> Hi!
>
> That's great! I see that the download page at
> http://www.haskell.org/ghc/download still links to the old tarballs
> for 8.0.2. Will those links be updated?
>
Yes, they will in due time.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

downloads.haskell.org unavailable for a bit

[Ben Gamari]

Hello all,

Unfortunately, due to a botched rsync, the tarballs on
downloads.haskell.org will be unavailable for a few hours. I'm currently
working on restoring the 8.0.2 tarballs first to ensure that they remain
available to users. I will follow up with status updates as they come.
Many apologies for the inconvenience.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: downloads.haskell.org unavailable for a bit

[Ben Gamari]

Ben Gamari <b...@well-typed.com> writes:

> Hello all,
>
> Unfortunately, due to a botched rsync, the tarballs on
> downloads.haskell.org will be unavailable for a few hours. I'm currently
> working on restoring the 8.0.2 tarballs first to ensure that they remain
> available to users. I will follow up with status updates as they come.
> Many apologies for the inconvenience.
>
The 8.0.2 tarballs are now again available.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: ghc-stage2 —interactive segfaults on Mac OS X 10.11.6 (build flavour = prof)

[Ben Gamari]

Alfredo Di Napoli  writes:

> Hey all,
>
> I’m trying to compile GHC HEAD (cloning from master) with the `prof` build
> flavour on a Mac OS X 10.11.6 machine and I have noticed that, despite
> ghc-stage2 works as expected, when invoked with —interactive it starts
> before crashing with a segmentation fault:
>
> ```
> ☁  compiler [master] ⚡ ../inplace/bin/ghc-stage2 --interactive
> GHCi, version 8.3.20170413: http://www.haskell.org/ghc/  :? for help
> [1]79176 segmentation fault  ../inplace/bin/ghc-stage2 --interactive
> ```
>
> Did it happen to somebody else or it’s just me? Shall I try throwing gdb at
> it to try and see what’s going on?

Hmm, interesting. I've not seen crashes like this locally nor in CI. It
would be great if you could try to get some insight. Is this crash
perfectly reproducible?

It may be worth adding -dcore-lint to GhcStage2HcOpts to ensure the code
we are producing is sane.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Harbormaster out to sea?

[Ben Gamari]

While I haven't checked, I suspect that the builders are simply busy as I've 
pushed a number of patches over the last few days.

Cheers,

- Ben

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows build failing in a new way

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> My windows build is more broken than usual.  I can't even build a GHC.
> Please, could someone fix this?  I'm getting desperate.

This is very odd; Harbormaster is also seeing it but I've been unable to
reproduce locally. It seems that the libffi build is failing but it's
quite unclear why it would fail for you yet succeed for me. I'll try to
reproduce on the Harbormaster machine.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows build failing in a new way

[Ben Gamari]

Phyx  writes:

> My CI server is also reproducing it while I also haven't been able to
> locally. Weird indeed.
>
Ahhh, I just noticed that Reid stumbled upon the culprit yesterday. See
[1].

Cheers,

- Ben


[1] https://phabricator.haskell.org/rGHCe901ed1c5d66


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: Windows build failing in a new way

[Ben Gamari]

loneti...@gmail.com writes:

> Ah great,
>
> Triples again.. I still wonder why it is suddenly an issue. We haven’t
> touched the .m4 file in a while and no one changed libffi either
> right? This is just like last time the normalization bit us. Causing
> days of broken builds on different targets while everyone fixed the
> one they were interested in.
>
Well, the patch that Reid points out indeed does change the triple which
we pass to subproject configures. However, I have been utterly unable to
reproduce this locally nor on the Harbormaster machine (both with
./validate).

Nevertheless, I have a hypothesis for the cause and a proposed fix in
D3304.

> Why do we do the normalization? It doesn’t seem to give us any
> benefits at all.. Maybe we should stop doing it after the branch for
> 8.4?
>
Well, there are a few different normalizations which you might mean
here. The patch in question affects autoconf's canonicalization. The
patch in question actually removes what may be the last reference to the
autoconf-canonicalized triple. However, my proposed fix then
reintroduces the need for it, since I suspect the cause is that we are
passing an empty triple to subproject configures.

There is also the matter of GHC's own triple normalization (e.g.
GHC_CONVERT_OS and friends, defined in aclocal.m4). I'm frankly not
entirely sure this is doing much harm and replacing it with autoconf's
canonicalized triple would be a non-trivial amount of work. However, if
you want to try I wouldn't be opposed.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: Windows build failing in a new way

[Ben Gamari]

Ben Gamari <b...@smart-cactus.org> writes:

> loneti...@gmail.com writes:
>
>> Ah great,
>>
>> Triples again.. I still wonder why it is suddenly an issue. We haven’t
>> touched the .m4 file in a while and no one changed libffi either
>> right? This is just like last time the normalization bit us. Causing
>> days of broken builds on different targets while everyone fixed the
>> one they were interested in.
>>
> Well, the patch that Reid points out indeed does change the triple which
> we pass to subproject configures. However, I have been utterly unable to
> reproduce this locally nor on the Harbormaster machine (both with
> ./validate).
>
> Nevertheless, I have a hypothesis for the cause and a proposed fix in
> D3304.
>
I believe at this point Harbormaster has demonstrated [1] that the fix
is effective. I'll go ahead and merge.

Cheers,

- Ben


[1] https://phabricator.haskell.org/harbormaster/build/23078/


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: can't validate on Linux

[Ben Gamari]

Richard Eisenberg  writes:

> Hi devs,
>
> When I try validating my patch, validation fails with
>
>> ...
>> Registering library for ghc-prim-0.5.0.0..
>> ghc-cabal: '/home/rae/ghc/ghc-valid/bindisttest/install
>> dir/lib/ghc-8.3.20170312/bin/ghc-pkg' exited with an error:
>> ghc-pkg: Couldn't open database /home/rae/ghc/ghc-valid/bindisttest/install
>> dir/lib/ghc-8.3.20170312/package.conf.d for modification: {handle:
>> /home/rae/ghc/ghc-valid/bindisttest/install
>> dir/lib/ghc-8.3.20170312/package.conf.d/package.cache.lock}: hLock: invalid
>> argument (Bad file descriptor)
>> make[3]: *** [ghc.mk:990: install_packages] Error 1
>> make[2]: *** [Makefile:51: install] Error 2
>> make[1]: *** [bindisttest/ghc.mk:33: test_bindist] Error 2
>> make: *** [Makefile:127: test_bindist] Error 2
>
This is when I typically `make distclean`. That being said, this does
look a bit suspicious. Let me know if cleaning doesn't help.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: New commits

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> Edward, Ben, others
> I have just pushed a patch series
>
> 2209d5e6 Comments only
>
> 4eeb3273 Drop redundant import
>
> 2d3cb34a Define TcSimplify.simplifyTopImplic and use it
>
> af6ed4a6 Fix constraint simplification in rules
>
> 48d1866e Improve error messages for skolems
>
> 7e96526a Fix TcSimplify.decideQuantification for kind variables
>
> bc0f3abd Deal with JoinIds before void types
>
> 900cfdc2 Do not generate a data-con wrapper for !Int#
> Issues:
>
> * bkpcabal03 is failing... I don't see how it can possibly have 
> anything to do with me, so I've pushed anyway.  Edward, might you look?
>
> * I get these stat failures - all improvements
>
>perf/compiler/T13035.runT13035 [stat too good] (normal)
>
>perf/compiler/T12425.runT12425 [stat too good] (optasm)
>
>perf/compiler/T9675.run T9675 [stat too good] (optasm)
>
>perf/space_leaks/T4029.run  T4029 [stat too good] (ghci)
>
>perf/should_run/T10359.run  T10359 [stat too good] (normal)
>
>perf/compiler/T1969.run T1969 [stat too good] (normal)
>
> Some of them are from before (I reported this and suggested
> re-centreing the numbers), but the last two are new, I think. Hurrah!
> Ben: might you look?

Sure.



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Removing core-spec.pdf from repository?

[Ben Gamari]

Hello everyone,

Currently there is a typeset copy of the Core specification in the GHC
repository. This means any time someone changes the specification the
repository grows by around 300kB. While this isn't the end of the
world, it's generally considered bad form to put generated files under
version control.

Of course, the tools required to typeset the specification (ott and
LaTeX) are non-trivial to install, so there is considerable convenience
that comes from having a typeset version readily available.

I suggest that we remove the PDF from the repository but instead I can
start including it in my nightly documentation builds. Any objections?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: LLVM calling convention for AVX2 and AVX512 registers

[Ben Gamari]

Edward Kmett  writes:

> That, rather tangentially, reminds me: If we do start to teach the code
> generator about how to produce these sorts of things from simpler parts,
> e.g. via enabling something like LLVM's vectorization pass, or some
> internal future ghc compiler pass that checks for, say, Superword-Level
> Parallelism
> 
> in the style of Jaewook Shin, then we need to differentiate between flags
> for what ghc/llvm is allowed to produce via optimization, etc. and what the
> end user is allowed to explicitly emit. e.g. in my own code I can safely
> call avx2 primitives after I set up guards to check that I'm on a CPU that
> supports them, but I can only currently emit that code after I tell GHC
> that I want it to allow the avx2 instructions. If I build a complicated
> dispatch mechanism in Haskell for picking the right ISA and emitting code
> for several of them, I'm going to need to tell ghc to let me build with all
> sorts of instruction sets that the machine the final executable runs on may
> not fully support. We should be careful not to conflate these two things.
>
Indeed this is tricky.

The obvious stop-gap solution is to simply move your various platform
dependent implementations into multiple modules. However, as you say
this quickly breaks down once GHC itself starts to learn vectorisation.
At that point you will need to draw the distinction you mention,
separating the ISA available to the user and that available to the
compiler.

Another related question is whether you eventually want a way to specify
an ISA per-function (via pragma, for instance). This would allow you to
set a conservative `-march` for the module on the whole, but allow use
of ISA extensions precisely when necessary. This is a bit tricky in the
face of inlining; perhaps you want to require only `NOINLINE` functions
can be decorated with such a thing.

I suspect in the case of LLVM this will require breaking modules up into
multiple compilation units and linking together the resulting objects.
This will certainly require a fair bit of engineering effort but nothing
terribly difficult.

Regarding dispatch, GCC has a function multi-versioning mechanism [1]
which is seems relevant to mention here. However, it's not entirely
clear to me whether the complexity here is worthwhile for GHC.

Anyways, there are plenty of possible options here; it would be helpful
to have a feature request ticket for the "user/compiler ISA" idea you
propose where we can collect ideas. Perhaps you could open one?

Cheers,

- Ben


[1] https://lwn.net/Articles/691666/


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Trac password change interface restored

[Ben Gamari]

Hello everyone,

For the last few weeks Trac's password change functionality has been
disabled due to passwords being leaked through the ghc-tickets mailing
list.

I have updated the Trac configuration to fix this leakage and reenabled
this interface. Moreover, we shouldn't see any further email address
verifications messages on ghc-tickets.

Thanks to everyone for their patience while this was handled.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: can't validate on Linux

[Ben Gamari]

Richard Eisenberg  writes:

> Hi devs,
>
> When I try validating my patch, validation fails with
>
>> ...
>> Registering library for ghc-prim-0.5.0.0..
>> ghc-cabal: '/home/rae/ghc/ghc-valid/bindisttest/install
>> dir/lib/ghc-8.3.20170312/bin/ghc-pkg' exited with an error:
>> ghc-pkg: Couldn't open database /home/rae/ghc/ghc-valid/bindisttest/install
>> dir/lib/ghc-8.3.20170312/package.conf.d for modification: {handle:
>> /home/rae/ghc/ghc-valid/bindisttest/install
>> dir/lib/ghc-8.3.20170312/package.conf.d/package.cache.lock}: hLock: invalid
>> argument (Bad file descriptor)

Andrzej, it looks like this is another ghc-pkg issue. Do you have any
idea what is going on here?

Richard, this is on Linux, yes?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

RE: Windows build broken again

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> Windows build still broken.  Please please could someone fix?
> It's something to do with the testsuite Python script

This is #13375. I have a fix in D3289. It's currently validating.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows build broken again

[Ben Gamari]

Phyx  writes:

> https://ghc.haskell.org/trac/ghc/ticket/13375
>
Are people not receiving my messages pointing out this ticket? I've
mentioned it twice now but I get the impression that these messages
aren't being seen.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Windows build broken again

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> The Windows build is broken again.  Here's the tail of the log
>
Yes, I opened a ticket (#13375) about this earlier. Running,

"C:/msys64/home/ben/ghc/inplace/bin/ghc-pkg.exe" recache

is sufficient to work around the issue it seems.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Better perf

[Ben Gamari]

Simon Peyton Jones via ghc-devs  writes:

> I've just committed this patch sequence
>
> fb9ae288088a3eabc4e1bb4e86fa473a3881d2e2 Make FloatOut/SetLevels idemoptent 
> on bottoming functions
>
> 995ab74b3c55fe3a0299bd94b49e948c942e76d6 Comments only
>
> 1163f4f2fe9aabd722c963497c67c5f8c71ef71b Tiny refactor
>
> 9b2c73ea8082199245bfa6a28390b70b38f87fd1 Make TH_Roles2 less fragile
>
> 9304df5230a7a29d3e992916d133e462b854e55f Fix CSE (again) on literal strings
> In my final validate run (after updating to HEAD) I saw
>
> Unexpected stat failures:
>
>perf/space_leaks/T4029.run  T4029 [stat too good] (ghci)
>
This one is rather interesting since I have also been seeing this for
quite some time locally, but Harbormaster consistently disagrees.
Moreover, it passes locally if my machine isn't under load. I
haven't yet investigated why allocations are so inconsistent.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Phabricator upgrade tonight

[Ben Gamari]

Unless someone objects I will be performing an upgrade of Phabricator
tonight. I'll likely start around 23:00 EST (4:00 UTC). As usual,
Phabricator may be down for anywhere from 10 minutes to a few hours,
depending upon how things go.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Where do I start if I would like help improve GHC compilation times?

[Ben Gamari]

Alfredo Di Napoli  writes:

> Hey folks,
>
Hi Alfredo!

First, thanks for writing. More eyes looking at GHC's compiler
performance is badly needed.

> maybe I’m setting up for something too ambitious for me, but I would like
> to take an active stance to the overlasting “GHC compilation times are
> terrible” matter, instead of simply stare at the screen with despair
> whenever GHC compiles a sufficiently large Haskell program ;)
>
> To make this even more interesting, I have never contributed to GHC either!
> The max I have pushed myself into was 2 years ago when I successfully built
> GHC head from source and tried to fix an Haddock “easy” ticket I don’t even
> recall (full disclosure, eventually I didn’t :D ).
>
> Specifically, I would love community recommendations & guidance about:
>
> 1. Is this simply too daunting for somebody like me? Maybe is better to
> first start contributing more regularly, take confidence with the code base
> AND then move forward?
>
As with any software project, it is possible to treat the compiler as a
black box, throw a profiler at it and see what hotspots show up. This
gives you a place to focus your effort, allowing you to learn a small
area and broaden your knowledge as necessary.

However, I think it's fair to say that you will be significantly more
productive if you first develop a basic understanding of the compilation
pipeline. I'd recommend having a look at the GHC Commentary [1] for a
start.

I think it also helps to have a rough idea of what "slow" means to you.
I find it is quite helpful if you have a particular program which you
feel compiles more slowly than you would like (especially if it even
compiles slowly with -O0, since then much less of the compiler is
involved in compilation). Another approach is to look for programs whose
compilation time has regressed over the course of GHC releases. It is
not hard to find these examples and it is often possible to bisect your
way back to the regressing commit.

Also, note that I have collected some notes pertaining to compiler
performance on the Wiki [2]. Here you will find a number of tickets of
interest (as well a some rough themes which I've noticed), some nofib
results which might guide your efforts, as well as a list of some
fixes which have been committed in the past.

[1] https://ghc.haskell.org/trac/ghc/wiki/Commentary/Compiler 
[2] https://ghc.haskell.org/trac/ghc/wiki/Performance/Compiler

> 2. Are compilation times largely dependant from the target platform (I’m on
> Darwin) or there is something which can be done “globally” so that the
> benefits can be experienced by everybody?
>
There are some external considerations (e.g. the platform's compiler and
linking toolchain) which contribute to GHC's runtime. For instance, it
is known that the BFD ld linker implementation that many Linux
distributions use by default is a great deal slower than it could be.
This particular issue has come up recently and I'm currently working on
allowing us to use the more performant gold linker when available.

However, I think it's fair to say that for most programs GHC's runtime
is largely independent of platform. I would invite you to try compiling
a package which you consider GHC to compile "slowly" with GHC's -v flag
(and GHC 8.0.1 or newer). This will give you a rough breakdown of where
time is spent. For many packages you will find that the simplifier
and/or typechecker dominate, followed (often distantly) by native code
generation. Of these steps native code generation is the only one with a
strong platform dependence.

> 3. Is there any recommended workflow to profile GHC compilation times? Is
> there any build flavour one should prefer when doing so? (Maybe the full,
> slowest one?)
>
There are a few options here:

 * As of GHC 8.0 the compiler will output timing and allocation
   information for its various stages if run with -v. This can be
   extremely helpful to get a high-level picture of where the compiler
   is spending its time while compiling your program. This is almost
   always the right place to start.

 * As with any Haskell program, the cost centre profiler can be used to
   characterize the memory and CPU behavior of various parts of the
   compiler.

   GHC's source tree includes a "prof" build flavour which builds the
   compiler with profiling enabled. However it only includes a handful
   of cost-centres and is best used when you already have a rough idea
   where you are looking and can add further cost-centres to drill down
   to your hotspot.

   Simply enabling -fprof-exported across the entire tree just doesn't
   work in my experience: not only is the resulting compiler quite slow,
   but the profile you get is far too unwieldy to learn from.

 * Occassionally the ticky-ticky profiler can be helpful in identifying
   allocation hotspots without the full overhead of the cost-centre
   profiler.

 * In principle our newly-stable DWARF debug information can be used 

Re: testsuite failures on Mac

[Ben Gamari]

Richard Eisenberg <r...@cs.brynmawr.edu> writes:

>> On Apr 7, 2017, at 3:27 PM, Ben Gamari <b...@smart-cactus.org> wrote:
>> 
>> How clean is clean? distclean?
>
> maintainer-clean. I never settle for anything less. :)
>
Hmm, that is quite peculiar. Harbormaster is currently behind master by
three commits, but the last completely build finished successfully.
Moreover, I've never seen any issue quite like what you report.

Are you using the GMP shipped with integer-gmp or your system's GMP? If
the latter, have you updated or otherwise touched it recently?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: testsuite failures on Mac

[Ben Gamari]

Richard Eisenberg  writes:

> Hi devs,
>
> On today's HEAD, I built from a cleaned tree and then ran the
> testsuite.

How clean is clean? distclean?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: 8.2.1: Ord TyCon is gone?

[Ben Gamari]

Levent Erkok  writes:

> I'm trying to port some plugin code from GHC 8.0.2 to GHC 8.2.1; alas I'm
> getting an error suggesting that TyCon is no longer an instance of Ord:
>
> Data/SBV/Plugin/Analyze.hs:580:33: error:
> • No instance for (Ord TyCon) arising from a use of ‘M.lookup’
>   There are instances for similar types:
> instance Ord GHC.Types.TyCon -- Defined in ‘GHC.Classes’
> • In the expression: k `M.lookup` tcMap
>   In the expression:
> case k `M.lookup` tcMap of
>   Just knd -> return knd
>   Nothing -> unknown
>   In a case alternative:
>   Just k
> -> case k `M.lookup` tcMap of
>  Just knd -> return knd
>  Nothing -> unknown
> |
> 580 |  Just k -> case k `M.lookup` tcMap of
> | ^^
>
Hmm. I am unable to reproduce this,

$ ghci
λ> import Type.Reflection
λ> let tc = typeRepTyCon (typeRep @Int)
λ> tc == tc
True

Does that work for you?

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: 8.2.1: Ord TyCon is gone?

[Ben Gamari]

Brandon Allbery <allber...@gmail.com> writes:

> On Sat, Jul 29, 2017 at 10:07 AM, Ben Gamari <b...@smart-cactus.org> wrote:
>
>> Hmm. I am unable to reproduce this,
>>
>> $ ghci
>> λ> import Type.Reflection
>> λ> let tc = typeRepTyCon (typeRep @Int)
>> λ> tc == tc
>> True
>>
>> Does that work for you?
>>
>
> Maybe I'm missing something, but doesn't that only test Eq, not Ord?
>
Oh dear, I somehow understood that you were referring to Typeable's
TyCon, not the ghc library's TyCon. Ignore my message.

Indeed the TyCon Ord instance is gone to help enforce determinism within
GHC. If you need a map use UniqFM.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHC release timing and future build infrastructure

[Ben Gamari]

Bardur Arantsson  writes:

> On 2017-08-02 21:26, Bardur Arantsson wrote:
>
>> I'd like to add a few thoughs (or just to underscore the ones you've
>> already brought forth, as the case may be):
>> 
> [--snip--]
>> reasons -- I wouldn't presume to know. Also note that this is
>> *basically* how Rust also works, it's just that they keep the "unstable"
>> bits behind a feature flag (until they're deemed 'stable') instead of
>> actually having different code bases.
>> 
>
> Sorry about the self-reply and excesive bolding-for-emphasis. The point
> of this past paragraph was that *perhaps* GHC could move towards
> (short-lived!) "feature flags" for the compiler[1]. Again, I have no
> experience with the GHC development process so maybe it's completely
> absurd to even contemplate such a thing (in terms of effort).
>
It's not clear to me that this would work. Afterall, not all commits are
new features and, perhaps more importantly, not all bugs stem from new
features.

Rather, many of GHCs commits are in fact refactorings. This is one of
the major reasons why the codebase hasn't collapsed under its own weight
despite the last two decades of development. Sometimes these
refactorings are merely cleanups, sometimes they fix bugs, but sometimes
they also introduce bugs.

In principle, the role of a stable branch is to serve as a "known good"
state which we can place fixes that we are quite certain fix a bug
without knowingly introducing others. Of course, in part due to our long
release schedule, we have in practice been a bit more liberal in
backporting. That being said, I do fear that including all new commits, even
those that don't fix bugs, in all new releases may have the effect of
destabilizing those releases.

Of course, feel free to clarify if I've misunderstood the proposal
(which I fear I may have).

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Limited availability next week

[Ben Gamari]

Hello everyone,

I will be travelling starting tomorrow through next week. During this
time I'll still try to check in on things once a day to make sure
nothing is burning down. However, I will be a bit less available. Still
feel free to ping me via email or IRC, however. I'll try to get back to
you.

Cheers,

- Ben




signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Restructuring hsSyn

[Ben Gamari]

Shayan Najd  writes:

> Currently AST declarations, their relate utilities, and `Outputable`
> instances are defined in the same files.
> Does anyone object to moving `Outputable` instances to separate files?
> The purpose is to gradually identify reusable functionalities, group them
> together, polish them (e.g., remove some unnecessary dependencies), and
> expose them to the end-users.
> At this stage, I don't expect any changes outside hsSyn.
>
Sounds reasonable to me.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Compilation time

[Ben Gamari]

Ryan Scott  writes:

> Another things to note about 8.2 build times is that a bulk of the time is
> probably being spent during the linking phase. This is because GHC 8.2 has
> switched over to split-sections by default, but unfortunately, old versions
> of the BFD linker take a long time to link code that uses split-sections
> (see the discussion at [1]). There is ongoing work (which should hopefully
> land before the final 8.2 release) to use gold instead of BFD as the
> default linker when available, which will reduce linking times dramatically.
>
Indeed it has already landed. See #13541.

In short, ./configure will now choose to use ld.gold or ld.lld if
available (although this can be disabled using the --disable-ld-override
configure flag).

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[ANNOUNCE] GHC 8.2.1 release candidate 3 available

[Ben Gamari]

Hello everyone,

I am pleased to at long last announce the third (and almost certainly
last) release candidate of GHC 8.2.1. Binary and source distributions
can be found at,

https://downloads.haskell.org/~ghc/8.2.1-rc3/

This release candidate fixes a number of regressions from 8.0.2 found in
-rc2, as well as a major correctness bug (#13615) present in several previous 
GHC
major releases. Users taking advantage of parallelism in their programs
will be strongly encouraged to upgrade to 8.2.1 once it is released.
Among the other issues resolved in this candidate are,

 * Functionality allowing GHC to automatically use the ld.gold linker
   when available (see #13541). Not only has there recently been
   significant user demand for this feature, but this also serves to
   work around a performance bug in BFD ld affecting GHC 8.2 (#13739)

 * A regression resulting in the package cache file to be out-of-date
   after binary distribution installation (#13375)

 * Numerous type-checker bugs (#13625, #1370, #13782, #13871, #13594,
   #13879, #13881, #13875)

 * A bug wherein a thread blocked on a BLACKHOLE may not be woken up (#13751)

 * and many more...

There are a few changes in release-engineering matters that should be
mentioned,

 * Binary distributions for CentOS 6.7 have been dropped due to the
   release of CentOS 7.0 which can use Debian 8 binaries. If you would
   like us to continue to produce CentOS 6.7 bindists please let me
   know.

 * GHC HQ now also provides FreeBSD and OpenBSD distributions for amd64;
   this will allow us to more quickly ship distributions to users by
   eliminating the need for a long lag time between source release
   availability and having all binary distributions available.

 * There is a technology-preview of an AArch64 Linux binary
   distribution, as well as an ARM Linux build (although the latter
   won't be uploaded until a few hours from now). AArch64 support is
   quite preliminary but should be stable in 8.4 thanks to further
   linker fixes by Moritz Angerman. ARM should be stable.

As always, let us know if you encounter problems. If all goes well we
will hopefully have a final 8.2.1 release in the next two weeks.

Finally, thanks for your patience during this admittedly quite drawn out
release cycle. While it has been long, we are confident that the result
will be worth the wait. Moreover, we have been steadily working on
infrastructure which should shrink future release cycles and give us
better testing between releases. More details on this coming soon.

Happy testing,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Compilation time

[Ben Gamari]

Evan Laforge <qdun...@gmail.com> writes:

> On Fri, Jul 7, 2017 at 9:28 AM, Ben Gamari <b...@smart-cactus.org> wrote:
>>
>> In short, ./configure will now choose to use ld.gold or ld.lld if
>> available (although this can be disabled using the --disable-ld-override
>> configure flag).
>
> Just from curiosity, does this apply to OS X?  Of course, gold is
> ELF-only, so it surely doesn't apply, but there's still lld.  OS X
> uses clang to compile so I thought it might already use lld, but the
> 'ld -v' output looks different, and lld.llvm.org implies it's a
> replacement for OS X ld, so maybe not.  But it doesn't look like GNU
> ld either, so maybe it's not affected by the BFD bug?
>
> I'll try 8.2 on OS X and see if the link time changes.

8.2 will prefer both gold and lld over bfd ld. However two conditions
must hold for these to be used,

 * The ld.lld/ld.gold executable must be in $PATH (or explicitly named
   by passing the LD variable to configure)

 * $CC must understand the `-fuse-ld={gold,lld}` option. For (IMHO quite
   silly) political reasons, gcc doesn't support `-fuse-ld=lld`. Debian
   happens to patch gcc to add support but I don't know how common this
   is in other distributions.

Unfortunately, some earlier `gcc` versions didn't fail if given a
`-fuse-ld` option that they didn't understand. Sadly we have no reliable
way to detect this, so in this case we may end up passing a `-fuse-ld`
option that gcc simply ignores.

In the case of OS X we use Apple's own home-grown linker. I'm not sure
whether/how OS X's gcc wrapper treats `-fuse-ld` (beyond that it doesn't
error if the flag is given). I also don't know whether lld is currently a
capable replacement for OS X ld.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Building GHC reference manual?

[Ben Gamari]

Iavor Diatchki  writes:

> Hello,
>
> is it possible to build just the HTML for the GHC reference manual, without
> building the whole of GHC, and if so how do I do it?
>
It is. Just run `make html` from the source root.

Note, however, that BUILD_SPHINX_HTML must be set to YES for this to
work.

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Compilation time

[Ben Gamari]

Moritz Angermann  writes:

> For those of us who are on macOS,

...

> Thus, if you end up having the llvm tools (and ld.lld) in your PATH, you
> will need to set `--disable-ld-override` during configure on macOS, or your
> ghc build will fail, because you end up trying to link MachO object files
> with an ELF only linker.
>
Yikes, this is quite bad. We'll need to teach autoconf to recognize this
before the release. Thanks for bringing this up!

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Bringing back #ghc IRC logging

[Ben Gamari]

Niklas Hambüchen  writes:

> Hi,
>
> as you may have noticed, #ghc IRC logging has been broken for months
> (though the channel topic still says "Logs: http://ircbrowse.net/ghc;).
>
I am very much in favor of doing something about this. Unfortunately,
Chris Done stepped back from ircbrowse.net around the time that #ghc
started using it for logging. Moving to a third-party service like
botbot.me sounds quite reasonable to m.e

Cheers,

- Ben



signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Trac down for a bit

[Ben Gamari]

Hello everyone,

Sorry for the interruption but Trac is having some trouble at the
moment. I'm actively working on restoring service and will let you know
when things are back to normal.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: Trac down for a bit

[Ben Gamari]

Ben Gamari <b...@well-typed.com> writes:

> Hello everyone,
>
> Sorry for the interruption but Trac is having some trouble at the
> moment. I'm actively working on restoring service and will let you know
> when things are back to normal.

I believe things should now be back to normal. As always, let me know if
you encounter trouble.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: GHCi tests hanging on Mac

[Ben Gamari]

Richard Eisenberg  writes:

> Hi devs,
>
> Some GHCi tests (but far from all!) hang on my Mac. I can't reproduce
> this behavior outside the testsuite driver, so I think it's the Python
> code, not the Haskell.
>
> Failing tests:
>   ghci/prog017/prog017
>   ghci/scripts/T6018ghcifail
>   ghci/scripts/T6018ghcirnfail
>
> And that's it! My Python reports that it's 2.7.13.
>
Hmm, well that is concerning. As far as I can tell the only thing that
these tests share is that their scripts are relatively long. However,
they aren't unique in this regard; if this were the cause I would have
also expected, for instance, T2182ghci and T11524a to fail as well.

Is it really always this set of tests that fail? Do they fail reliably?

Are you certain you are running the test in the same way that the
testsuite driver is (specifically, piping the script to stdin of the
ghci process)? I just looked at the driver implementation and it seems
to me that it could be simplified a bit. Does D3735 help? It's
admittedly a stab in the dark, but Python has surprised me in the past.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

[ANNOUNCE] GHC 8.2.1 available

[Ben Gamari]

   ===
The Glasgow Haskell Compiler -- version 8.2.1
   ===


The GHC developers are very happy to announce the long-awaited 8.2.1
release of Glasgow Haskell Compiler. Binary and source distributions can
be found at

https://downloads.haskell.org/~ghc/8.2.1/

This is the second release in the 8.0 series. As such, the focus of this
release is performance, stability, and consolidation. Consequently
numerous cleanups can be seen throughout the compiler including,

 * Significant improvements in compiler performance

 * More robust support for levity polymorphism

 * Reliable DWARF debugging information

 * Improved runtime system performance on NUMA systems

 * Retooling of the cost-center profiler, including support for live
   streaming of profile data via the GHC event log

 * Interface file determinism

 * More robust treatment of join points, enabling significantly better
   code generation in many cases

 * Numerous improvements in robustness on Windows

 * and the resolution of over 500 other tickets

In addition, there are a number of new features,

 * A new, more type-safe type reflection mechanism

 * The long-awaited Backpack module system

 * Deriving strategies to disambiguate between GHC's various instance
   deriving mechanisms

 * Unboxed sum types, for efficient unpacked representation of sum data
   types

 * Compact regions, allowing better control over garbage collection
   in the presence of large heaps containing many long-lived objects.

 * Colorful messages and caret diagnostics for more legible errors

A more thorough list of the changes in this release can be found in the
release notes,

  https://haskell.org/ghc/docs/8.2.1/docs/html/users_guide/8.2.1-notes.html

There are a few changes in release-engineering that should be noted,

 * Binary distributions for 32-bit CentOS 6.7 have been dropped.
   Moreover, there are no dedicated CentOS 7.0 distributions as CentOS 7
   can use can use Debian 8 binaries. If you would like us to continue
   to produce 32-bit CentOS 6.7 distributions please let us know.

 * GHC HQ now builds FreeBSD and OpenBSD distributions for amd64; this
   comes after many years of these distributions being faithfully
   provided by Karel Gardas and Pali Gabor Janos, who we should heartily
   thank for their contributions.

   GHC HQ building these distributions ourselves will allow us to more
   quickly ship distributions to users by eliminating the need for a
   long lag time between source release availability and having all
   binary distributions available.

 * There is a technology-preview of an AArch64 Linux binary
   distribution, as well as an ARM Linux build. AArch64 support is quite
   preliminary but should be stable in 8.4 thanks to further linker
   fixes by Moritz Angerman. ARM should be stable.

 * GHC now tries to use the gold and lld linkers by default. These
   linkers are significantly faster than the BFD linker implementation
   that most Linux distributions use by default. If gold or lld are not
   available GHC will use the system's default linker. GHC can be forced
   to use the default linker by passing --disable-ld-override to
   configure.

This release has been the result of over a year of hard work by over 150
code contributors. Thanks to everyone who has helped in writing patches,
testing, reporting bugs, and offering feedback over the last year.

This release cycle was admittedly quite drawn out, significantly longer
than expected or desired. While we are confident that the result is
worth the wait, we have been steadily working on infrastructure which
should help shrink future release cycles and give us better testing
between releases. More details on this coming soon.

As always, let us know if you encounter trouble.


How to get it
~

The easy way is to go to the web page, which should be self-explanatory:

http://www.haskell.org/ghc/

We supply binary builds in the native package format for many
platforms, and the source distribution is available from the same
place.

Packages will appear as they are built - if the package for your
system isn't available yet, please try again later.


Background
~~

Haskell is a standard lazy functional programming language.

GHC is a state-of-the-art programming suite for Haskell.  Included is
an optimising compiler generating efficient code for a variety of
platforms, together with an interactive system for convenient, quick
development.  The distribution includes space and time profiling
facilities, a large collection of libraries, and support for various
language extensions, including concurrency, exceptions, and foreign
language interfaces. GHC is distributed under a BSD-style open source license.

A wide variety of Haskell related resources (tutorials, libraries,
specifications, documentation, compilers, interpreters, references,
contact 

Re: Frontend Plugins Proposal

[Ben Gamari]

Matthew Pickering  writes:

> 4 years ago Edsko proposed a patch to add support for frontend plugins
> to GHC. You can read his proposal here -
> https://ghc.haskell.org/trac/ghc/wiki/FrontendPluginsProposal#no1
>
> Something like core to core plugins but for typechecked code would be
> useful for tool writers. Does anyone know why the patch was never
> merged?
>
This was before I started following closely, however, knowing how these
things tend to go, I doubt that there was any strong objection to
merging. It probably just got dropped.

If we were to pick up this proposal again today it would certainly help
if it provided some concrete motivation for the new plugin type.

> Confusingly Edward Yang implemented another feature called frontend
> plugins which does something quite different (allowing users to write
> custom major modes rather than hook into the frontend). These features
> are quite different to each other.
>
To be fair, Edsko's proposal refers to the proposed plugin type as
a "source plugin" not a "frontend plugin", despite the Wiki page name. I
think "source plugin" is a better name anyways.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Harbormaster update

[Ben Gamari]

Hello everyone,

Note that the Harbormaster builders now bootstrap with GHC 8.2.1 and
LLVM 3.9. I'll be on the lookout for any failures but ping me if you see
Harbormaster fail to build your otherwise perfectly good patch.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs

Re: [ANNOUNCE] GHC 8.2.1 available

[Ben Gamari]

Wolfgang Jeltsch <wolfgang...@jeltsch.info> writes:

> Am Samstag, den 22.07.2017, 23:03 -0400 schrieb Ben Gamari:
>> In addition, there are a number of new features,
>> 
>>  * A new, more type-safe type reflection mechanism
>> 
>>  * The long-awaited Backpack module system
>> 
>>  * Deriving strategies to disambiguate between GHC's various instance
>>    deriving mechanisms
>> 
>>  * Unboxed sum types, for efficient unpacked representation of sum 
>>    data types
>> 
>>  * Compact regions, allowing better control over garbage collection
>>    in the presence of large heaps containing many long-lived objects.
>> 
>>  * Colorful messages and caret diagnostics for more legible errors
>> 
>> A more thorough list of the changes in this release can be found in
>> the release notes,
>> 
>>   https://haskell.org/ghc/docs/8.2.1/docs/html/users_guide/8.2.1-notes.html
>
> It seems that the release notes mention the new type reflection
> mechanism und colorful messages only in the “Highlights” section, not in
> the “Full details” section, and that they do not mention the Backpack
> module system and unboxed sums at all.
>
Yes, indeed these were oversights. They are fixed in the ghc-8.2 branch
and I will try to push newly generated documentation shortly.

Cheers,

- Ben


signature.asc
Description: PGP signature
___
ghc-devs mailing list
ghc-devs@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs