Re: [ANNOUNCE] GHC 8.2.1 release candidate 2

[Alberto Valverde]

Hi,

I'm trying to build an app with the new release candidate and I'm running
into a couple of issues, some which I can fix or workaround, some are
worrisome and others are blocking me. I'm using Nix, if that matters.

The fixable
---

- The expected too strict version bounds. Worked around using doJailbreak,
will send PRs to the respective packages with relaxed bounds.
- A weird kind error when using ConstraintKinds in a propietary package
which didn't manifest itself with ghc < 8.2:

src/Sigym4/Propag/Types.hs:1071:4: error:
• Expected a type, but
  ‘(PropagIOConstraint l a,
Missing (PropagIOVector l) (PropagIONullable l a),
Elem (PropagIONullable l a) ~ a)’ has kind
  ‘Constraint’
• In the type ‘((PropagIOConstraint l a,
 Missing (PropagIOVector l) (PropagIONullable l a),
 Elem (PropagIONullable l a) ~ a))’
  In the type declaration for ‘CanSerialize’
 |
1071 |   (( PropagIOConstraint l a
 |...

src/Sigym4/Propag/Types.hs:1077:4: error:
• Expected a constraint,
but ‘(CanSerialize l Double, CanSerialize l Int16)’ has kind ‘*’
• In the type ‘(CanSerialize l Double, CanSerialize l Int16)’
  In the type declaration for ‘CanSerializePropagTypes’
 |
1077 |( CanSerialize l Double
 |^^^...

I cannot link to the source for this package since it belongs to my
employer but I think that the interesting code is:

type CanSerialize l a =
( PropagIOConstraint l a
, Missing (PropagIOVector l) (PropagIONullable l a)
, Elem (PropagIONullable l a) ~ a
)

where PropagIOConstraint, PropagIONullable and PropagIOVector are
"standalone" type families and Elem is an associated type family (not from
IsList)

Both errors disappear if I give an explicit kind signature like this: "type
CanSerialize l a = (. :: Constraint)". Is this expected behaviour?
Should I try to isolate and open a ticket?

The worrisome


- I had to disable the tests for two packages since they seem to "hang"
 (ie: they never finish running and don't seem to consume any CPU time).
These packages are lens-4.15.1 and fingertree-0.1.1.0. Maybe it's a Nix
environmental issue, I'm not sure. Can anyone reproduce this?

The blockers
-

- I can't manage to install several packages which include executables
(namely, update-nix-fetchgit and snap-server, for the moment) because Cabal
says that it cannot find the source for the main module of the executables:

"Setup: can't find source for Main in ."

It seems that the "hs-source-dir" directive in the .cabal file is not being
honored. Maybe a Nix-only issue? Can anyone reproduce this? Any ideas on
how can I fix it?

Thanks very much for GHC, btw :)

Alberto


On Tue, May 16, 2017 at 4:47 AM, Ben Gamari  wrote:

>
> Hello everyone,
>
> The GHC team is very pleased to announce the second candidate of the
> 8.2.1 release of the Glasgow Haskell Compiler. Source and binary
> distributions are available at
>
> https://downloads.haskell.org/~ghc/8.2.1-rc2/
>
> This is the second of what will likely be either two or three release
> candidates leading up the final 8.2.1 release. This release will
> feature,
>
>   * A new type-indexed Typeable implementation
>
>   * The long awaited Backpack
>
>   * Deriving strategies for disambiguating DeriveAnyClass,
> GeneralizedNewtypeDeriving, and stock mechanisms
>
>   * Overloaded record fields
>
>   * Improved compiler performance
>
>   * Better code generation through more robust tracking of join points
>
>   * Compact regions for more efficient garbage collection and serialization
>
>   * Better support for machines with non-uniform memory architectures
>
>   * More robust support for levity (e.g. RuntimeRep) polymorphism
>
>   * A simple interface for streaming eventlog data from live processes
>
>   * Further refinement of DWARF support
>
> This candidate fixes most of the issues present in release candidate
> one including,
>
>  * #13233: typePrimRep panic while compiling GHC with profiling enabled
>  * #13509: type error involving unboxed tuples
>  * #13426: compile-time memory-usage regression
>  * #13560: Windows binary distributions carry absolute paths to toolchain
>  * #13585: Control.Lens.Wrapped.ala causes compiler panic
>  * #13623: Join points produce bad code for stream fusion
>
> As always, please let us know if you have difficulty. Thanks to everyone
> who has contributed!
>
> Happy testing,
>
> - Ben
>
> ___
> ghc-devs mailing list
> ghc-d...@haskell.org
> http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
>
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

Re: 8.2.1-rc2 upgrade report

[Alberto Valverde]

Hi Simon,

Thanks for the pointer. I re-did both builds with -dshow-passes and made a
small script to plot the results of the lines which summarize the elapsed
time and allocated memory per phase and module. I've uploaded the raw logs,
a plot of the results and the script I wrote to generate it to
https://gist.githubusercontent.com/albertov/145ac5c01bfbadc5c9ff55e9c5c2e50e
.

The plotted results live here
https://gist.githubusercontent.com/albertov/145ac5c01bfbadc5c9ff55e9c5c2e50e/raw/8996644707fc5c18c1d42ad43ee31b1817509384/bench.png

Apparently, the biggest slowdown in respect to 8.0.2 seems to occur in the
SpecConstr and Simplifier passes in the Propag (where the "main" function
is) and the Sigym4.Propag.Engine (where the main algorithm lives) modules.

Any other tests that would be helpful for me to run? I'm not sure where to
start to create a reproducible case but I'll see if I can come up with
something soon...

Alberto

On Tue, Jun 6, 2017 at 1:58 PM, Simon Peyton Jones <simo...@microsoft.com>
wrote:

> Thanks for the report.
>
>
>
> Going from 67G to 56G allocation is a very worthwhile improvement in
> runtime!  Hurrah.
>
>
>
> However, trebling compile time is very bad.  It is (I think) far from
> typical: generally 8.2 is *faster* at compiling than 8.0 so you must be
> hitting something weird.  Anything you can do to make a reproducible case
> would be helpful.  -dshow-passes shows the size of each intermediate form,
> which at least sometimes shows where the big changes are.
>
>
>
> Simon
>
>
>
> *From:* Glasgow-haskell-users [mailto:glasgow-haskell-users-
> boun...@haskell.org] *On Behalf Of *Alberto Valverde
> *Sent:* 06 June 2017 12:39
> *To:* GHC users <glasgow-haskell-users@haskell.org>
> *Subject:* 8.2.1-rc2 upgrade report
>
>
>
> Hi,
>
>
>
> I've finally managed to upgrade all the dependencies of the proprietary
> app I mentioned some days ago in this list and there are good and bad
> differences I've noticed between 8.0.2 that I'd like to share.
>
>
>
> The bad
>
> ---
>
>
>
> * An optimized cold build (-O2)  is about 3 times slower (~53s vs. ~2m55s)
> and consumes more memory (~2Gb vs. ~7Gb) at it's peak.
>
>
>
> The good
>
> -
>
>
>
> * An un-optimized cold build (-O0) takes about the same time (~21s, phew!
> :) It's maybe even slightly faster with 8.2 (too few and badly taken
> measurements to really know, though)
>
> * The optimized executable is slightly faster and allocates less memory.
> For this app it makes up for the performance regression of the optimized
> build (which is almost always done by CI), IMHO.
>
>
>
> I did only a couple of runs and only wrote down [1] the last run results
> (which were similar to the previous results) so take these observations
> with a grain of salt (except maybe the optimized build slowdown, which
> doesn't have much margin for variance to be skewing the results). I also
> measured the peak memory usage by observing "top".
>
>
>
> In case gives a clue: The app is a multi-threaded 2D spread simulator
> which deals with many mmapped Storable mutable vectors and has been pretty
> optimized for countless hours (I mean by this that it has (too) many INLINE
> pragmas. Mostly on polymorphic functions to aid in their specialization). I
> think some of this information can be deduced from the results I'm linking
> at the footer. I believe the INLINEs are playing a big part of the slowdown
> since the slowest modules to compile are the "Main" ones which put
> everything together, along with the typical lens-th-heavy "Types" ones.
>
>
>
> I'd like to help by producing a reproducible and isolated benchmark or a
> better analysis or ... so someone more knowledgeable than me on GHC
> internals can someday hopefully attack the regression. Any pointers on what
> would help and where can I learn to do it?
>
>
>
> Thanks!
>
>
>
>
>
> [1] https://gist.github.com/albertov/46fbb13d940f67a569f9a25c1cb8154c
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Falbertov%2F46fbb13d940f67a569f9a25c1cb8154c=02%7C01%7Csimonpj%40microsoft.com%7Cce8170b01de84bacdab308d4acd0aef3%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636323459695695689=iRWChz2fzNFuXBAhiYABO5OepSXYBLxvvIQ8CAiABYg%3D=0>
>
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users

8.2.1-rc2 upgrade report

[Alberto Valverde]

Hi,

I've finally managed to upgrade all the dependencies of the proprietary app
I mentioned some days ago in this list and there are good and bad
differences I've noticed between 8.0.2 that I'd like to share.

The bad
---

* An optimized cold build (-O2)  is about 3 times slower (~53s vs. ~2m55s)
and consumes more memory (~2Gb vs. ~7Gb) at it's peak.

The good
-

* An un-optimized cold build (-O0) takes about the same time (~21s, phew!
:) It's maybe even slightly faster with 8.2 (too few and badly taken
measurements to really know, though)
* The optimized executable is slightly faster and allocates less memory.
For this app it makes up for the performance regression of the optimized
build (which is almost always done by CI), IMHO.

I did only a couple of runs and only wrote down [1] the last run results
(which were similar to the previous results) so take these observations
with a grain of salt (except maybe the optimized build slowdown, which
doesn't have much margin for variance to be skewing the results). I also
measured the peak memory usage by observing "top".

In case gives a clue: The app is a multi-threaded 2D spread simulator which
deals with many mmapped Storable mutable vectors and has been pretty
optimized for countless hours (I mean by this that it has (too) many INLINE
pragmas. Mostly on polymorphic functions to aid in their specialization). I
think some of this information can be deduced from the results I'm linking
at the footer. I believe the INLINEs are playing a big part of the slowdown
since the slowest modules to compile are the "Main" ones which put
everything together, along with the typical lens-th-heavy "Types" ones.

I'd like to help by producing a reproducible and isolated benchmark or a
better analysis or ... so someone more knowledgeable than me on GHC
internals can someday hopefully attack the regression. Any pointers on what
would help and where can I learn to do it?

Thanks!


[1] https://gist.github.com/albertov/46fbb13d940f67a569f9a25c1cb8154c
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://mail.haskell.org/cgi-bin/mailman/listinfo/glasgow-haskell-users