Re: [GHC] #4012: Compilation results are not deterministic

2012-11-17 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by igloo):

 Hmm, OK. I was going to suggest a final renaming pass (in which only
 generated names get renamed) and ''not'' throwing the result away, so that
 it's easier to understand when the hash is or isn't changing.

 But I think that would be worse, because then the -ddump-simpl etc names
 wouldn't match the final names.

 So your !DeBruijn idea sounds best to me.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-12 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonmar):

 @igloo: I'm not sure if that would fix it.  A name that was generated at
 the top-level might be floated in, and conversely a non-top-level name
 might be floated out.  So wouldn't the same problem still occur?

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-08 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+
Changes (by igloo):

  * owner:  igloo =>


-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-08 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by igloo):

 I think this is only really a problem when ghc ''generates'' the same name
 at both the top level and in an expression. Is it feasible to generate
 e.g. 'tlvl' for top-level names, and 'lvl' for let/lambda-bound names?

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-07 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonmar):

 ... or, if we like to keep the original names for readability, we could do
 local renaming only when computing the hash, and throw away the renamed
 version afterwards (basically hash the !DeBruijn representation).

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-07 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonmar):

 Actually you're right.  I said that we do alpha-renaming, but in fact the
 current algorithm keeps the original names when they don't clash, and that
 gives rise to the `lvl2/lvl3` difference in your example.

 We should not be using the original `OccName` when tidying a local binder,
 we should just always use "x", or use a different `OccName` for different
 kinds of binders (e.g. "x" for let-binders, "y" for lambdas, and "z" for
 case binders).  Relevant code is in `coreSyn/CoreTidy.lhs:tidyIdBndr`.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-07 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by nomeata):

 Sure. But it is desireable, and low hanging fruit that fixes the ABI of
 different code (such as normalizing the interface wrt alpha-renaming) will
 probably make it more robust against odd changes when compiling the same
 code twice.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-07 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonmar):

 Note that compiling ''different code'' and hoping to get the same ABI hash
 is a much more difficult proposition than getting the same ABI hash by
 repeatedly compiling the same code (which is what this ticket was
 originally about).  I realise it's a reasonable thing to want to do, but I
 just wanted to point out the distinction.  We need to solve the easier
 problem first before we can tackle the harder one.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-07 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by nomeata):

 Replying to [comment:26 simonpj]:
 > Do Ian's changes help?

 Please excuse me if I don’t set up HEAD in a way to build all the packages
 required to test the code above. But I did find a small test case that
 could potentially be added to the test suite. With the attached Test.hs
 and GHC-7.4.1, I get this result:

 {{{
 $ ghc --make -O Test.hs && cp Test.hi Test1.hi && ghc --make -DVARIANT -O
 Test.hs && diff -u <(ghc --show-iface Test1.hi) <(ghc --show-iface
 Test.hi)
 [1 of 1] Compiling Test ( Test.hs, Test.o )
 [1 of 1] Compiling Test ( Test.hs, Test.o )
 --- /dev/fd/63  2012-11-07 20:45:57.719385850 +0100
 +++ /dev/fd/62  2012-11-07 20:45:57.719385850 +0100
 @@ -5,11 +5,11 @@
  Way: Wanted [],
   got[]
  interface main:Test 7041
 -  interface hash: f70ea3ebb89cd1fb5fd38e711dc3a86b
 -  ABI hash: 3d46e40ea31fe2b5209a9cfa0e8c55a2
 +  interface hash: 0c3a556b0b6f4bd91ecf6b9097dd0914
 +  ABI hash: fbf17c77263ac40a0a46a1da9debd563
export-list hash: 10500eb025dddf2ab63ae606468fa772
orphan hash: 693e9af84d3dfcc71e640e005bdc5e2e
 -  flag hash: fc61acebd75ed0e7e110c288131c5ae4
 +  flag hash: eda85656077ab2758c42e66fbdb9f82a
used TH splices: False
where
  exports:
 @@ -33,15 +33,15 @@
   ds :: ((), a) ->
 case ds of wild { (,) ds1 z ->
 case ds1 of wild1 { () -> Test.x @ a $dEq $dNum z } })
 -}
 -9ca884031f6b6682163a35c8b033ebe1
 +3170054a493c32af50e7d3df196daeae
b :: forall a. [a] -> [a] -> [a]
  {- Arity: 1, HasNoCafRefs, Strictness: L,
 Unfolding: (\ @ a x1 :: [a] ->
 let {
 - lvl2 :: [a]
 + lvl3 :: [a]
   = let { y :: [a] = GHC.Base.++ @ a x1 x1 } in
 GHC.Base.++ @ a y y
 } in
 -   \ z :: [a] -> GHC.Base.++ @ a z lvl2) -}
 +   \ z :: [a] -> GHC.Base.++ @ a z lvl3) -}
  2d4ae0d356c36e2c96109ac55a0ff610
x :: forall a.
 GHC.Classes.Eq a -> GHC.Num.Num a -> a -> GHC.Types.Bool
 }}}

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-06 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonpj):

 Do Ian's changes help?

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-02 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by ian@…):

 commit ba38f995d6312aa0cfe15873c8e5e9475e03f19c
 {{{
 Author: Ian Lynagh 
 Date:   Fri Nov 2 22:54:12 2012 +

 Avoid putting uniqs in specconstr rules; part of #4012

 There's no need to have the uniq in the rule, but its presence can
 cause spurious ABI changes.

  compiler/specialise/SpecConstr.lhs |   10 +++---
  1 files changed, 7 insertions(+), 3 deletions(-)
 }}}

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-11-02 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by ian@…):

 commit 095b9bf4ed418c43216cfca2ae271c143e555f1d
 {{{
 Author: Ian Lynagh 
 Date:   Fri Nov 2 03:45:15 2012 +

 Don't put uniqs in ghc wrapper function names; part of #4012

 The wrapper functions can end up in interface files, and thus are
 part of the ABI hash. But uniqs easily change for no good reason
 when recompiling, which can lead to an ABI hash needlessly changing.

  compiler/deSugar/DsForeign.lhs |   20 +++-
  compiler/main/DynFlags.hs  |   11 ---
  2 files changed, 23 insertions(+), 8 deletions(-)
 }}}

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-10-30 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by nomeata):

 Replying to [comment:21 simonpj]:
 > As he says, the a59/a60 diff in your example is a bit mysterious.
 Perhaps show us both interface files?  And preferably a way to reproduce.

 To reproduce, fetch haskell-warp-1.2.1.1 and compile with 7.4.1; once
 unmodified and once with the (to be attached) patch applied. Interface
 files also attached.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-10-29 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  igloo   
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+
Changes (by igloo):

  * owner:  => igloo


-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-10-29 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonpj):

 So to summarise Simon M's comments
  * A full and total fix is beyond us at the moment
  * But it seems likely that some modest fixes that are relatively easy
 (notably ccall wrappers and the "SC.." rule names) may go a long way.  Ian
 is going to have a go at those.

 As he says, the a59/a60 diff in your example is a bit mysterious.  Perhaps
 show us both interface files?  And preferably a way to reproduce.

 Simon

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-10-29 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by simonmar):

 We do alpha-normalise the whole code of the module before we hash it -
 that's why you're seeing `a60` rater than the internal names that look
 like `a_s3fj`.  It's a mystery to me why there are differences here, so we
 ought to look into it.  Ian has agreed to fix the differences due to
 `ghc_wrapper` Ids that were mentioned above, and to look into the
 differences in "SC:..." rule names, which should fix two of the known
 issues here.

 Fixing the CSE ordering issue is a tricky one, because it means keeping
 the bindings in a deterministic order throughout the compiler.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-10-27 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.2   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by nomeata):

 Again this is biting me. We are trying to backport a fix in warp to Debian
 testing, without having to rebuild everything based on it. And indeed the
 patch does not really change any of the exported information (e.g. no
 function eligible for inlining). But nevertheless the ABI hash changes.
 Judging from the diff of ```ghc --show-iface``` (attached), this is only
 due to renaming of internal variables.

 Maybe the ABI hashing should first alpha-normalize the expression?

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-06-15 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.1   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by nomeata):

 Let me illustrate one consequence of this problem. In Debian, the freeze
 for wheezy is near. We have just managed to migrate ghc-7.4.1 to wheezy,
 but there are a few remaining bugs of various severity (#5991, #6156) and
 having GHCi on ARM would be nice as well. Alas, we cannot upgrade to
 ghc-7.4.2 which provides all this, as it would require rebuilding
 everything, which is not possible at this stage any more
 (http://lists.debian.org/debian-haskell/2012/06/msg00038.html).

 What we will probably do is to backport fixes from 7.4.2 onto 7.4.1, but
 this is not nice either, as it might introduce new bugs and makes our ghc
 deviate more and more from upstream.

 Ideally, if 7.4.2 does not change anything about the actual ABI, it would
 not force it to change completely. Currently, it does, by encoding the
 version number in every hash (#5328; granted, I filed that, but that does
 not mean its the best solution :-)). It should do something like haddock:
 Keep an internal counter, separate from the version number, that is
 increased if indeed everything needs to be rebuilt, e.g. changes in the
 .hi file format, in the calling convention etc. (← identifying this list
 is probably non-trivial). And if a new compiler version is released that
 does not do any of these, the number stays the same and previously
 compiled packages can be re-used.

 This would also require the file names of installed packages to not
 include the ghc version but the ghc abi interface number; but configuring
 this can be left to the distributions.

 (Hmm, this is not _really_ this bug, but closely related.)

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-02-13 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  high  |  Milestone:  7.6.1   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+
Changes (by simonmar):

  * priority:  low => high


Old description:

> Short story: if you use ghc-6.12.1.20100318 (or similar, probably
> ghc-6.12.1 release will produce the same results) to bootstrap
> ghc-6.12, and then use that ghc-6.12 to bootstrap another ghc-6.12,
> those two instances of ghc-6.12 will have different ABI hashes and
> interfaces in the ghc package. If you use ghc-6.10 for the
> bootstrapping, you'll even get differences in the ghc, base and
> Cabal packages.
>
> Long story: see logfiles and descriptions at http://darcs.volkswurst.de
> /boot-tests/ (note that the logfiles are quite large, I really don't want
> to attach 150 MB of logs to this ticket).

New description:

 There are some issues with non-determinism in the output of GHC, which
 means that compilations are not repeatable.  This affects some users (e.g.
 Debian packagers) who need to be able to get repeatable hashes for the
 packages of a GHC build.

 The cases we know about that lead to non-deterministic results are:

  * The `spec_ids` (specialised Ids) attached to an Id have a non-
 deterministic ordering
  * CSE can give different results depending on the order in which the
 bindings are considered, and since the ordering is non-deterministic, the
 result of CSE is also non-deterministic.  e.g. in `x = z; y = z; z = 3`,
 where `y` and `x` are exported, we can end up with either `x = y; y = 3`
 or `y = x; x = 3`.
  * There seems to be something unpredictable about the order of arguments
 to SpecConstr-generated specialisations, see
 [http://www.haskell.org/pipermail/glasgow-haskell-
 users/2011-April/020287.html]
  * The wrappers generated by the `CApiFFI` extension have non-
 deterministic names. (see comment:15 below).

 '''Old ticket description follows'''

 Short story: if you use ghc-6.12.1.20100318 (or similar, probably
 ghc-6.12.1 release will produce the same results) to bootstrap
 ghc-6.12, and then use that ghc-6.12 to bootstrap another ghc-6.12,
 those two instances of ghc-6.12 will have different ABI hashes and
 interfaces in the ghc package. If you use ghc-6.10 for the
 bootstrapping, you'll even get differences in the ghc, base and
 Cabal packages.

 Long story: see logfiles and descriptions at http://darcs.volkswurst.de
 /boot-tests/ (note that the logfiles are quite large, I really don't want
 to attach 150 MB of logs to this ticket).

--

Comment:

 I've updated the description following the latest evidence.  Let's try to
 do something about this for the next major release.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2012-02-10 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  low   |  Milestone:  7.6.1   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Os:  Unknown/Multiple  |   Architecture:  Unknown/Multiple
   Failure:  Other | Difficulty:  Difficult (2-5 days)
  Testcase:|  Blockedby:  
  Blocking:|Related:  
---+

Comment(by nomeata):

 This has bit us again with 7.4.1. I forgot about this issue and we built
 lots of libraries against the first upload of GHC 7.4.1 (built with
 7.0.4), all of which will have to be rebuild after the next minor GHC
 upload, because that will be built with 7.4.1 and the base API will likely
 change again.

 This is the interface diff causing the ABI change in base, in case anybody
 wonders:
 {{{
 /usr/lib/ghc/base-4.5.0.0/System/Posix/Internals.hi
 --- /dev/fd/63  2012-02-10 20:33:16.717639938 +
 +++ /dev/fd/62  2012-02-10 20:33:16.717639938 +
 @@ -5,11 +5,11 @@
  Way: Wanted [],
   got[]
  interface base:System.Posix.Internals 7041
 -  interface hash: fef49c410428b674b72ebd8b1a93bd62
 -  ABI hash: 33b2adf3f92d97c87fbcbd52d7f22781
 +  interface hash: 13159d537315369ddfe00efa59167f8a
 +  ABI hash: fe56115a605d2758561d089b972bb8bb
export-list hash: 83b224804aef34838bb7767a01e8aaa7
orphan hash: 693e9af84d3dfcc71e640e005bdc5e2e
 -  flag hash: 28bf87c2d7df91e45dad874fc0a5931f
 +  flag hash: 865402a98b08183763ca20b5e9837ae0
used TH splices: False
where
  exports:
 @@ -241,7 +241,7 @@
  import  -/  integer-gmp:GHC.Integer.Type 254721fe3c053c778036ed1a1fa4248d
  addDependentFile "libraries/base/include/HsBaseConfig.h"
  addDependentFile "libraries/base/dist-
 install/build/autogen/cabal_macros.h"
 -3e0a4be3b609e3ac62226d26fd8dbfa2
 +9e62726c5155bee7f03712b80c79
$wa :: GHC.Prim.Int#
   -> GHC.Prim.State# GHC.Prim.RealWorld
   -> (# GHC.Prim.State# GHC.Prim.RealWorld, GHC.IO.IOMode.IOMode
 #)
 @@ -258,7 +258,7 @@
System.Posix.Internals.fdGetMode3
System.Posix.Internals.fdGetMode2
(\ ds2 :: GHC.Prim.State# GHC.Prim.RealWorld ->
 -   case {__pkg_ccall base ghc_wrapper_d2ju_fcntl
 GHC.Prim.Int#
 +   case {__pkg_ccall base ghc_wrapper_d2jn_fcntl
 GHC.Prim.Int#
 -> GHC.Prim.Int#
 -> GHC.Prim.State#
 GHC.Prim.RealWorld
 @@ -386,7 +386,7 @@
0
->
 System.Posix.Internals.fileType2
 w } } } } } } } } } }
 } }) -}
 -178f30c6a50e12552309d6f35e96cfa3
 +1e8eed47078d0056cc7209ae4be4cd83
$wa2 :: GHC.Prim.Int#
-> GHC.Prim.State# GHC.Prim.RealWorld
-> (# GHC.Prim.State# GHC.Prim.RealWorld, () #)
 @@ -404,7 +404,7 @@
 GHC.Prim.RealWorld,
 GHC.Prim.Int# #)}
GHC.Prim.realWorld# of wild1 { (#,#) ds2 ds3 ->
 -   case {__pkg_ccall base ghc_wrapper_d2ji_fcntl
 GHC.Prim.Int#
 +   case {__pkg_ccall base ghc_wrapper_d2jb_fcntl
 GHC.Prim.Int#
   ->
 GHC.Prim.Int#
   ->
 GHC.Prim.Int#
   ->
 GHC.Prim.State#
 @@ -440,7 +440,7 @@
   (GHC.Types.NTCo:IO (Refl
 Foreign.C.Types.CInt))
 ds6 of wild5 { (#,#) new_s1 a76 ->
  (# new_s1, GHC.Tuple.() #) } } } } } }) -}
 -20c72f17c3f40d20ed52c7a2c116
 +ce1394a8e53df017e302d2cc305d0231
$wa3 :: GHC.Prim.Int#
-> GHC.Types.Bool
-> GHC.Prim.State# GHC.Prim.RealWorld
 @@ -459,7 +459,7 @@
System.Posix.Internals.fdGetMode3
System.Posix.Internals.setNonBlockingFD3
(\ ds2 :: GHC.Prim.State# GHC.Prim.RealWorld ->
 -   case {__pkg_ccall base ghc_wrapper_d2ju_fcntl
 GHC.Prim.Int#
 +   case {__pkg_ccall base ghc_wrapper_d2jn_fcntl
 GHC.Prim.Int#
 -> GHC.Prim.Int#
 -> GHC.Prim.State#
 GHC.Prim.RealWorld
 @@ -494,7 +494,7 @@
 GHC.Prim.RealWorld,
 GHC.Prim.Int# #)}
  GHC.Prim.realWorld# of wild6 { (#,#)
 ds2 ds3 ->
 - case {__pkg_ccall base
 ghc_wrapper_d2ji_fcntl GHC

Re: [GHC] #4012: Compilation results are not deterministic

2011-04-05 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  normal|  Milestone:  7.2.1   
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
  Testcase:|  Blockedby:  
Difficulty:  Difficult (2-5 days)  | Os:  Unknown/Multiple
  Blocking:|   Architecture:  Unknown/Multiple
   Failure:  Other |  
---+
Changes (by nomeata):

 * cc: mail@… (added)


Comment:

 Just a minor comment: This is hurting distributions quite a bit. Is there
 a chance to at least avoid this particular problem (cBooterVersion vs
 cProjectVersion) in the next ghc release?

 Thanks,
 Joachim

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler

___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2010-04-29 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  normal|  Milestone:  6.14.1  
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Difficulty:  Difficult (2-5 days)  | Os:  Unknown/Multiple
  Testcase:|   Architecture:  Unknown/Multiple
   Failure:  Other |  
---+
Changes (by igloo):

  * milestone:  => 6.14.1


-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2010-04-26 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  normal|  Milestone:  
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Difficulty:  Difficult (2-5 days)  | Os:  Unknown/Multiple
  Testcase:|   Architecture:  Unknown/Multiple
   Failure:  Other |  Patch:  0   
---+

Comment(by simonmar):

 Replying to [comment:6 kili]:

 > For example, if two persons are working on a ghc package for their
 operating system, and also on updates for all the stuff that needs ghc
 (xmonad, alex, happy, monadius, to mention the most important ones), it's
 a pity if they're not able to exchange packages.

 For this to work I think you'd really need stable ABIs, not just
 deterministic compilation.  If you only had deterministic compilation, it
 would be hard to guarantee that all the inputs to the compilation were
 identical between the two systems: there are a lot of ways that
 differences can accidentally creep in (different optimisation flags,
 system configuration settings, etc.).

 See [wiki:Commentary/Packages#AllowingABIcompatibilty] for more on this.

 > You mentioned that even a `make clean; make' may change the interfaces.
 Indeed, I remember at least one case where ghc segfailted during the build
 (this doesn't happen often, only every 20th or 30th build)

 FWIW, we never see random segfaults in GHC here, so I expect that really
 is a bug specific to your OS or system that needs investigating.

 > and I just restarted the build -- and got interface changes.

 Yes - unfortunate, but not unexpected.

 > > Anyway, I do agree that having deterministic compilation results is a
 desirable thing, so on second thoughts I've re-opened the bug and retitled
 it.
 >
 > Thanks. But is this really *one* Bug? There are the problems with non-
 determinisms, but I think that the CSE on inlined constants (like
 cBooterVersion and cProjectVersion) is a separate problem.

 Ian pointed out that you really have source differences here, which I
 hadn't spotted.  (Actually, I'm not sure why `cBooterVersion` isn't set to
 the version of the stage1 compiler when building stage2, it doesn't seem
 useful to record the stage0 version in the stage2 compiler.)

 So the underlying problem leading to the CSE issue is that the compiler
 doesn't use a deterministic ordering for bindings internally.  It uses the
 `Unique` ordering, which is semi-random.  The results of CSE may depend on
 the order of bindings, but it's entirely possible that there are other
 non-deterministic consequences of the same kind elsewhere.  Making the
 order deterministic would fix all of them.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2010-04-25 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  normal|  Milestone:  
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Difficulty:  Difficult (2-5 days)  | Os:  Unknown/Multiple
  Testcase:|   Architecture:  Unknown/Multiple
   Failure:  Other |  Patch:  0   
---+

Comment(by igloo):

 Replying to [comment:6 kili]:
 > Thanks. But is this really *one* Bug? There are the problems with non-
 determinisms, but I think that the CSE on inlined constants (like
 cBooterVersion and cProjectVersion) is a separate problem.

 I don't think that's a bug at all. You are compiling different source
 code, so it's entirely reasonable that the ABI should be different.

 If the ABI was otherwise stable then it might be worth trying to work
 around `cBooterVersion`'s value affecting the ABI, though.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic

2010-04-25 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  normal|  Milestone:  
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Difficulty:  Difficult (2-5 days)  | Os:  Unknown/Multiple
  Testcase:|   Architecture:  Unknown/Multiple
   Failure:  Other |  Patch:  0   
---+

Comment(by kili):

 Replying to [comment:5 simonmar]:
 > The bootstrapping GHC is a red herring, as I mentioned - the fact is
 that compilation results aren't deterministic.  They never have been, in
 fact.

 And it's good that the problems are detected now with the hashes.

 > Even if compilation results were deterministic, under what circumstances
 would you want to have two systems "use each others packages", when
 they're both using a GHC independently built from source?  If they
 independently build GHC from source, why wouldn't they build packages from
 source too?

 For example, if two persons are working on a ghc package for their
 operating system, and also on updates for all the stuff that needs ghc
 (xmonad, alex, happy, monadius, to mention the most important ones), it's
 a pity if they're not able to exchange packages.

 You mentioned that even a `make clean; make' may change the interfaces.
 Indeed, I remember at least one case where ghc segfailted during the build
 (this doesn't happen often, only every 20th or 30th build) and I just
 restarted the build -- and got interface changes.

 > Note that packages are registered with an MD5 hash of the interface, so
 the package system will spot if you try to register an incompatible
 package.

 And that's good, but it's just a workaround, UMHO.

 > Anyway, I do agree that having deterministic compilation results is a
 desirable thing, so on second thoughts I've re-opened the bug and retitled
 it.

 Thanks. But is this really *one* Bug? There are the problems with non-
 determinisms, but I think that the CSE on inlined constants (like
 cBooterVersion and cProjectVersion) is a separate problem.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs


Re: [GHC] #4012: Compilation results are not deterministic (was: Different stage2 results depending on the version of the bootstrapping compiler)

2010-04-25 Thread GHC
#4012: Compilation results are not deterministic
---+
  Reporter:  kili  |  Owner:  
  Type:  bug   | Status:  new 
  Priority:  normal|  Milestone:  
 Component:  Compiler  |Version:  6.12.2  
Resolution:|   Keywords:  
Difficulty:  Difficult (2-5 days)  | Os:  Unknown/Multiple
  Testcase:|   Architecture:  Unknown/Multiple
   Failure:  Other |  Patch:  0   
---+
Changes (by simonmar):

  * status:  closed => new
  * os:  OpenBSD => Unknown/Multiple
  * component:  Build System => Compiler
  * difficulty:  => Difficult (2-5 days)
  * resolution:  invalid =>


Comment:

 Replying to [comment:4 kili]:

 > But this means if two people build packages, where one used a different
 bootstrapper for ghc than the other, they can't use each other's packages.
 I'd consider that a bug.

 The bootstrapping GHC is a red herring, as I mentioned - the fact is that
 compilation results aren't deterministic.  They never have been, in fact.

 Even if compilation results were deterministic, under what circumstances
 would you want to have two systems "use each others packages", when
 they're both using a GHC independently built from source?  If they
 independently build GHC from source, why wouldn't they build packages from
 source too?

 Note that packages are registered with an MD5 hash of the interface, so
 the package system will spot if you try to register an incompatible
 package.

 Anyway, I do agree that having deterministic compilation results is a
 desirable thing, so on second thoughts I've re-opened the bug and retitled
 it.

-- 
Ticket URL: 
GHC 
The Glasgow Haskell Compiler
___
Glasgow-haskell-bugs mailing list
Glasgow-haskell-bugs@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-bugs