Nofib fails with an error
When I run nofib the fft2 benchmark fails with an error (see below). This happens both with HEAD and 7.6.3. I'm setting the path to benchmarked GHC in mk/build.mk using option WithNofibHc. Anyone else is having the same problem? Janek ././fft2 512 /dev/null expected stdout not matched by reality --- fft2.stdout72013-05-15 08:01:24.014049002 +0200 +++ /tmp/runtest10927.1 2013-06-04 18:06:08.401977002 +0200 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.6712796173433053e-12 +result2 = 2.6712796173433053e-12 +result3 = 4.827444399779779e-8 real0m0.055s user0m0.049s sys 0m0.005s ././fft2 512 /dev/null expected stdout not matched by reality --- fft2.stdout72013-05-15 08:01:24.014049002 +0200 +++ /tmp/runtest10963.1 2013-06-04 18:06:08.507977002 +0200 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.6712796173433053e-12 +result2 = 2.6712796173433053e-12 +result3 = 4.827444399779779e-8 real0m0.054s user0m0.047s sys 0m0.005s ././fft2 512 /dev/null expected stdout not matched by reality --- fft2.stdout72013-05-15 08:01:24.014049002 +0200 +++ /tmp/runtest10999.1 2013-06-04 18:06:08.602977002 +0200 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.6712796173433053e-12 +result2 = 2.6712796173433053e-12 +result3 = 4.827444399779779e-8 real0m0.052s user0m0.044s sys 0m0.007s ././fft2 512 /dev/null expected stdout not matched by reality --- fft2.stdout72013-05-15 08:01:24.014049002 +0200 +++ /tmp/runtest11035.1 2013-06-04 18:06:08.712977002 +0200 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.6712796173433053e-12 +result2 = 2.6712796173433053e-12 +result3 = 4.827444399779779e-8 real0m0.052s user0m0.046s sys 0m0.006s ././fft2 512 /dev/null expected stdout not matched by reality --- fft2.stdout72013-05-15 08:01:24.014049002 +0200 +++ /tmp/runtest11071.1 2013-06-04 18:06:08.816977002 +0200 @@ -1,3 +1,3 @@ -result1 = 2.59635799135966e-12 -result2 = 2.59635799135966e-12 -result3 = 4.8279900966008427e-8 +result1 = 2.6712796173433053e-12 +result2 = 2.6712796173433053e-12 +result3 = 4.827444399779779e-8 make[2]: *** [runtests] Błąd 1 Failed making all in fft2: 1 make[1]: *** [all] Błąd 1 Failed making all in spectral: 1 make: *** [all] Błąd 1 ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
RE: how to checkout proper submodules
For the avoidance of doubt, I totally support what Austin and Johan are saying: I find the current setup confusing too. I'm totally persuaded of the merits of git bisect etc. I am the opposite of a git power-user (a git weedy-user?). I will be content to do whatever I'm told workflow-wise, provided I am told clearly in words of one syllable. I *very strongly* want to reduce barriers to entry for would-be contributors, and this is clearly a barrier we could lower. Making Kazu, Austin, Johan, etc more productive is massively valuable. There may be some history to how we arrived at this point, but that should not constrain for the future. We can change our workflow. I would want Ian and Simon to be thoroughly on board, but I regard the current setup as totally open to improvement. Please! BTW, Ian has written it up quite carefully here: http://hackage.haskell.org/trac/ghc/wiki/Repositories, and the linked page http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream. Simon | -Original Message- | From: ghc-devs-boun...@haskell.org [mailto:ghc-devs-boun...@haskell.org] | On Behalf Of Austin Seipp | Sent: 05 June 2013 07:35 | To: Johan Tibell | Cc: ghc-devs@haskell.org | Subject: Re: how to checkout proper submodules | | I absolutely agree here, FWIW. We should only do this if there is a | clear consensus on doing so and everyone doing active development is | comfortable with it. And it's entirely possible submodules are | inadequate for some reason that I'm not aware of which is a | show-stopper. | | However, the notion of impact-on-contributors cuts both ways. GHC has | an extremely small team of hackers as it stands, and we are lucky to | have *amazing* contributors like Kazu, Andreas, yourself, Simon | Simon, and numerous others help make GHC what it is. Much of this is | volunteer work. But as the Haskell community grows, and we are at a | loss of other full-time contributors like Simon Marlow, I think we are | beginning to see the strain on GHC and its current contributors. So, | it's important to evaluate what we're doing right and wrong. This | feedback loop is always present even if seasoned contributors can live | with it - but new contributors will definitely be impacted. | | In this instance, I honestly find it disheartening that the answer to | things like getting older revisions of the source code in HEAD, or | techniques like bisection is basically that doesn't work. The second | is unfortunate, but the latter is pretty legitimately worrying. It | would be one thing if this was a one-off occurrence of some odd | developer-workflow. But I have answered the fundamental question here | (submodules vs free-floating clones) a handful of times myself at | least, experienced the pain of the decision myself when doing | rollbacks, and I'm sure other contributors can say the same. | | GHC is already a large, industry-strength software project with years | of work put behind it. The barrier to entry and contribution is not | exactly small, but I think we've all done a good job. I'd love to see | more people contributing. But I cannot help but find these discussions | a bit sad, where contributors are impaired due to regular/traditional | development workflows like rollbacks are rendered useless - due to | some odd source control discrepancy that nobody else on the planet | seems to suffer from. | | I guess the short version is basically that that you're absolutely | right: the time of Simon, Ian, and other high-profile contributors is | *extremely* important. But I'd also rather not have people like Kazu | potentially spend hours or even days doing what simple automation can | achieve in what is literally a few keystrokes, and not only that - par | for the course for other projects. This ultimately impacts the | development cycles of *everybody*. And even if Kazu deals with it - | what about the next person? | | On Wed, Jun 5, 2013 at 12:12 AM, Johan Tibell johan.tib...@gmail.com | wrote: | The latest git release has improved submodules support some so if we now | thing the benefits of submodules outweigh the costs we can discuss if we | want to change to policy. I don't want to make that decision for other GHC | developers that spend much more time on GHC than I (e.g. SPJ). Their | productivity is more important than any inconveniences the lack of | consistent use of submodules might cause me. | | | -- | Regards, | Austin - PGP: 4096R/0x91384671 | | ___ | ghc-devs mailing list | ghc-devs@haskell.org | http://www.haskell.org/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
I agree with Austin and Johan. It's a bizarre setup. Submodules have their pain points (which we already have to deal with), but the ability to properly snapshot and branch the whole tree would be a serious benefit IMO. Manuel PS: While we are at it, why don't we just have the main repos on GitHub and use forks and pull requests like the rest of the world? (Using Git, but not GitHub's superb infrastructure, seems like a terrible waste to me.) Simon Peyton-Jones simo...@microsoft.com: For the avoidance of doubt, I totally support what Austin and Johan are saying: I find the current setup confusing too. I'm totally persuaded of the merits of git bisect etc. I am the opposite of a git power-user (a git weedy-user?). I will be content to do whatever I'm told workflow-wise, provided I am told clearly in words of one syllable. I *very strongly* want to reduce barriers to entry for would-be contributors, and this is clearly a barrier we could lower. Making Kazu, Austin, Johan, etc more productive is massively valuable. There may be some history to how we arrived at this point, but that should not constrain for the future. We can change our workflow. I would want Ian and Simon to be thoroughly on board, but I regard the current setup as totally open to improvement. Please! BTW, Ian has written it up quite carefully here: http://hackage.haskell.org/trac/ghc/wiki/Repositories, and the linked page http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream. Simon | -Original Message- | From: ghc-devs-boun...@haskell.org [mailto:ghc-devs-boun...@haskell.org] | On Behalf Of Austin Seipp | Sent: 05 June 2013 07:35 | To: Johan Tibell | Cc: ghc-devs@haskell.org | Subject: Re: how to checkout proper submodules | | I absolutely agree here, FWIW. We should only do this if there is a | clear consensus on doing so and everyone doing active development is | comfortable with it. And it's entirely possible submodules are | inadequate for some reason that I'm not aware of which is a | show-stopper. | | However, the notion of impact-on-contributors cuts both ways. GHC has | an extremely small team of hackers as it stands, and we are lucky to | have *amazing* contributors like Kazu, Andreas, yourself, Simon | Simon, and numerous others help make GHC what it is. Much of this is | volunteer work. But as the Haskell community grows, and we are at a | loss of other full-time contributors like Simon Marlow, I think we are | beginning to see the strain on GHC and its current contributors. So, | it's important to evaluate what we're doing right and wrong. This | feedback loop is always present even if seasoned contributors can live | with it - but new contributors will definitely be impacted. | | In this instance, I honestly find it disheartening that the answer to | things like getting older revisions of the source code in HEAD, or | techniques like bisection is basically that doesn't work. The second | is unfortunate, but the latter is pretty legitimately worrying. It | would be one thing if this was a one-off occurrence of some odd | developer-workflow. But I have answered the fundamental question here | (submodules vs free-floating clones) a handful of times myself at | least, experienced the pain of the decision myself when doing | rollbacks, and I'm sure other contributors can say the same. | | GHC is already a large, industry-strength software project with years | of work put behind it. The barrier to entry and contribution is not | exactly small, but I think we've all done a good job. I'd love to see | more people contributing. But I cannot help but find these discussions | a bit sad, where contributors are impaired due to regular/traditional | development workflows like rollbacks are rendered useless - due to | some odd source control discrepancy that nobody else on the planet | seems to suffer from. | | I guess the short version is basically that that you're absolutely | right: the time of Simon, Ian, and other high-profile contributors is | *extremely* important. But I'd also rather not have people like Kazu | potentially spend hours or even days doing what simple automation can | achieve in what is literally a few keystrokes, and not only that - par | for the course for other projects. This ultimately impacts the | development cycles of *everybody*. And even if Kazu deals with it - | what about the next person? | | On Wed, Jun 5, 2013 at 12:12 AM, Johan Tibell johan.tib...@gmail.com | wrote: | The latest git release has improved submodules support some so if we now | thing the benefits of submodules outweigh the costs we can discuss if we | want to change to policy. I don't want to make that decision for other GHC | developers that spend much more time on GHC than I (e.g. SPJ). Their | productivity is more important than any inconveniences the lack of | consistent use of submodules
Re: how to checkout proper submodules
On 5 June 2013 01:43, Manuel M T Chakravarty c...@cse.unsw.edu.au wrote: I agree with Austin and Johan. It's a bizarre setup. Submodules have their pain points (which we already have to deal with), but the ability to properly snapshot and branch the whole tree would be a serious benefit IMO. Manuel PS: While we are at it, why don't we just have the main repos on GitHub and use forks and pull requests like the rest of the world? (Using Git, but not GitHub's superb infrastructure, seems like a terrible waste to me.) I'd be all for this. We partially use the GitHub infrastructure since trac broke and I changed the emails to point to GitHub instead. I also often do code reviews with other devs on a personal GHC fork on github before merging in. I believe it would also help encourage more contributors (especially for libraries) but others have expressed disagreement with this point of view in the past and I'm not in hold of data. Either way, I'm glad git bisect may soon work. We'll finally be able to use the whole feature set of a version control tool :) (other piece was the move from darcs - git which gave us a working annotate). Simon Peyton-Jones simo...@microsoft.com: For the avoidance of doubt, I totally support what Austin and Johan are saying: I find the current setup confusing too. I'm totally persuaded of the merits of git bisect etc. I am the opposite of a git power-user (a git weedy-user?). I will be content to do whatever I'm told workflow-wise, provided I am told clearly in words of one syllable. I *very strongly* want to reduce barriers to entry for would-be contributors, and this is clearly a barrier we could lower. Making Kazu, Austin, Johan, etc more productive is massively valuable. There may be some history to how we arrived at this point, but that should not constrain for the future. We can change our workflow. I would want Ian and Simon to be thoroughly on board, but I regard the current setup as totally open to improvement. Please! BTW, Ian has written it up quite carefully here: http://hackage.haskell.org/trac/ghc/wiki/Repositories, and the linked page http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream. Simon | -Original Message- | From: ghc-devs-boun...@haskell.org [mailto: ghc-devs-boun...@haskell.org] | On Behalf Of Austin Seipp | Sent: 05 June 2013 07:35 | To: Johan Tibell | Cc: ghc-devs@haskell.org | Subject: Re: how to checkout proper submodules | | I absolutely agree here, FWIW. We should only do this if there is a | clear consensus on doing so and everyone doing active development is | comfortable with it. And it's entirely possible submodules are | inadequate for some reason that I'm not aware of which is a | show-stopper. | | However, the notion of impact-on-contributors cuts both ways. GHC has | an extremely small team of hackers as it stands, and we are lucky to | have *amazing* contributors like Kazu, Andreas, yourself, Simon | Simon, and numerous others help make GHC what it is. Much of this is | volunteer work. But as the Haskell community grows, and we are at a | loss of other full-time contributors like Simon Marlow, I think we are | beginning to see the strain on GHC and its current contributors. So, | it's important to evaluate what we're doing right and wrong. This | feedback loop is always present even if seasoned contributors can live | with it - but new contributors will definitely be impacted. | | In this instance, I honestly find it disheartening that the answer to | things like getting older revisions of the source code in HEAD, or | techniques like bisection is basically that doesn't work. The second | is unfortunate, but the latter is pretty legitimately worrying. It | would be one thing if this was a one-off occurrence of some odd | developer-workflow. But I have answered the fundamental question here | (submodules vs free-floating clones) a handful of times myself at | least, experienced the pain of the decision myself when doing | rollbacks, and I'm sure other contributors can say the same. | | GHC is already a large, industry-strength software project with years | of work put behind it. The barrier to entry and contribution is not | exactly small, but I think we've all done a good job. I'd love to see | more people contributing. But I cannot help but find these discussions | a bit sad, where contributors are impaired due to regular/traditional | development workflows like rollbacks are rendered useless - due to | some odd source control discrepancy that nobody else on the planet | seems to suffer from. | | I guess the short version is basically that that you're absolutely | right: the time of Simon, Ian, and other high-profile contributors is | *extremely* important. But I'd also rather not have people like Kazu | potentially spend hours or even days doing what simple
Re: how to checkout proper submodules
David Terei wrote: Either way, I'm glad git bisect may soon work. Having git bisect work on the GHC tree would be a plus! Erik -- -- Erik de Castro Lopo http://www.mega-nerd.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
BTW, this could also be a basis for solving another common pain point, that seems to afflict everyone: "Validate fails. Was it me?" Have the buildbots push only validating version-combinations (using submodules to make this precise) into the repository newbies go to could help solve that. If the buildbot also bisects the validation problem and sends an email pinpointing the problem, keeping the validated repository recent might be reasonably easy. Daniel On 06/05/2013 11:43 AM, Manuel M T Chakravarty wrote: I agree with Austin and Johan. It's a bizarre setup. Submodules have their pain points (which we already have to deal with), but the ability to properly snapshot and branch the whole tree would be a serious benefit IMO. Manuel PS: While we are at it, why don't we just have the main repos on GitHub and use forks and pull requests like the rest of the world? (Using Git, but not GitHub's superb infrastructure, seems like a terrible waste to me.) Simon Peyton-Jones simo...@microsoft.com: For the avoidance of doubt, I totally support what Austin and Johan are saying: I find the current setup confusing too. I'm totally persuaded of the merits of git bisect etc. I am the opposite of a git power-user (a git weedy-user?). I will be content to do whatever I'm told workflow-wise, provided I am told clearly in words of one syllable. I *very strongly* want to reduce barriers to entry for would-be contributors, and this is clearly a barrier we could lower. Making Kazu, Austin, Johan, etc more productive is massively valuable. There may be some history to how we arrived at this point, but that should not constrain for the future. We can change our workflow. I would want Ian and Simon to be thoroughly on board, but I regard the current setup as totally open to improvement. Please! BTW, Ian has written it up quite carefully here: http://hackage.haskell.org/trac/ghc/wiki/Repositories, and the linked page http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream. Simon | -Original Message- | From: ghc-devs-boun...@haskell.org [mailto:ghc-devs-boun...@haskell.org] | On Behalf Of Austin Seipp | Sent: 05 June 2013 07:35 | To: Johan Tibell | Cc: ghc-devs@haskell.org | Subject: Re: how to checkout proper submodules | | I absolutely agree here, FWIW. We should only do this if there is a | clear consensus on doing so and everyone doing active development is | comfortable with it. And it's entirely possible submodules are | inadequate for some reason that I'm not aware of which is a | show-stopper. | | However, the notion of impact-on-contributors cuts both ways. GHC has | an extremely small team of hackers as it stands, and we are lucky to | have *amazing* contributors like Kazu, Andreas, yourself, Simon | Simon, and numerous others help make GHC what it is. Much of this is | volunteer work. But as the Haskell community grows, and we are at a | loss of other full-time contributors like Simon Marlow, I think we are | beginning to see the strain on GHC and its current contributors. So, | it's important to evaluate what we're doing right and wrong. This | feedback loop is always present even if seasoned contributors can live | with it - but new contributors will definitely be impacted. | | In this instance, I honestly find it disheartening that the answer to | things like "getting older revisions of the source code in HEAD," or | techniques like bisection is basically "that doesn't work." The second | is unfortunate, but the latter is pretty legitimately worrying. It | would be one thing if this was a one-off occurrence of some odd | developer-workflow. But I have answered the fundamental question here | (submodules vs free-floating clones) a handful of times myself at | least, experienced the pain of the decision myself when doing | rollbacks, and I'm sure other contributors can say the same. | | GHC is already a large, industry-strength software project with years | of work put behind it. The barrier to entry and contribution is not | exactly small, but I think we've all done a good job. I'd love to see | more people contributing. But I cannot help but find these discussions | a bit sad, where contributors are impaired due to regular/traditional | development workflows like rollbacks are rendered useless - due to | some odd source control discrepancy that nobody else on the planet | seems to suffer from. | | I guess the short version is basically that that you're absolutely | right: the time of Simon, Ian, and other high-profile contributors is | *extremely* important. But I'd also rather not have people like Kazu | potentially spend hours or even days doing what simple automation can | achieve in what is literally a few keystrokes, and not only that - par | for the course for other projects. This ultimately impacts the | development cycles of *everybody*. And even if
Re: how to checkout proper submodules
For me the biggest plus of switching to submodules would be keeping GHC and testsuite in sync. If there are any reasons not to change in-tree library repos to submodules, then I would at least want testsuite to be changed to a submodule. I also use github for my daily work on GHC and being able to send patches via Pull Requests would make things easier. On the other hand it might be more difficult to attach files to a ticket (no such feature on Github AFAIK). Speaking of Github, perhaps we should put more stress on github folks to fix this: https://github.com/github/markup/issues/196 ? Jan ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
On 06/05/2013 10:10 AM, David Terei wrote: On 5 June 2013 01:43, Manuel M T Chakravarty c...@cse.unsw.edu.au wrote: I agree with Austin and Johan. It's a bizarre setup. Submodules have their pain points (which we already have to deal with), but the ability to properly snapshot and branch the whole tree would be a serious benefit IMO. Manuel PS: While we are at it, why don't we just have the main repos on GitHub and use forks and pull requests like the rest of the world? (Using Git, but not GitHub's superb infrastructure, seems like a terrible waste to me.) I'd be all for this. We partially use the GitHub infrastructure since trac broke and I changed the emails to point to GitHub instead. I also often do code reviews with other devs on a personal GHC fork on github before merging in. I believe it would also help encourage more contributors (especially for libraries) but others have expressed disagreement with this point of view in the past and I'm not in hold of data. As a very recent new (try-to-be-)contributor, i'ld like to weight in, in favor of this. IMHO, having to create a trac account, and submit patches by attachment (with the confusing trac UI) instead of just pushing to some repositories and issuing pull requests is quite suboptimal. I don't think it would scare anyone enough that they wouldn't contribute, but lowering the entry cost is always useful. -- Vincent ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validate fails: Var/Type length mismatch: [] [s{tv a15I} [tv]]
I think that Iavor is facing the same problems that I reported on this list on May 15th. I also see ghcpkg01 failing (when run during the validation) and passing (when run separately). This sometimes happens with other tests as well. Iavor, are you getting 'cahce is out of date' error when the test fails? Actual stderr output differs from expected: --- /dev/null 2013-05-14 15:38:10.77100 +0200 +++ ../../libraries/base/tests/IO/T3307.run.stderr 2013-05-15 09:21:45.695049002 +0200 @@ -0,0 +1,2 @@ +WARNING: cache is out of date: /dane/uczelnia/projekty/ghc-validate/bindisttest/install dir/lib/ghc-7.7.20130514/package.conf.d/package.cache + use 'ghc-pkg recache' to fix. *** unexpected failure for T3307(normal) I think that these problems are likely to be caused be build artifacts left from previous validation - package.cache files (and many others) are not removed by `make maintainer-clean`. I created a patch that fixes this, but it has not been reviewed yet: http://hackage.haskell.org/trac/ghc/ticket/7941 Perhaps it would resolve these problems. Janek There are no dependencies in the testsuite. You're either seeing: * a bug in a test, which means it is affecting something outside of the test * a bug in a test, which causes it to inconsistently fail * some problem with your machine, e.g. dodgy RAM (from most to least likely). Note that in the first case it is not necessarily the failing test that is broken. The testlog may give more clues as to what is going wrong. Thanks Ian ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
David Terei davidte...@gmail.com: On 5 June 2013 01:43, Manuel M T Chakravarty c...@cse.unsw.edu.au wrote: I agree with Austin and Johan. It's a bizarre setup. Submodules have their pain points (which we already have to deal with), but the ability to properly snapshot and branch the whole tree would be a serious benefit IMO. Manuel PS: While we are at it, why don't we just have the main repos on GitHub and use forks and pull requests like the rest of the world? (Using Git, but not GitHub's superb infrastructure, seems like a terrible waste to me.) I'd be all for this. We partially use the GitHub infrastructure since trac broke and I changed the emails to point to GitHub instead. I also often do code reviews with other devs on a personal GHC fork on github before merging in. I believe it would also help encourage more contributors (especially for libraries) but others have expressed disagreement with this point of view in the past and I'm not in hold of data. For the compiler, the barriers to contribution are probably elsewhere, but for the libraries, I'm sure, it would lower the barrier to entry. For example, to fix some documentation, I personally would never bother to create a patch file and attach it to some Trac ticket (where I first have to create an account). In contrast, a pull request on GitHub is a matter of a few clicks. Manuel PS: Anybody who doubts this needs to post their GitHub account name, so we can check that they actually ever used GitHub properly ;)___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Patch/feature proposal: Source plugins
Hi all,
The plugin mechanism gives access to the program in Core; this suffices for
many but not quite all purposes. Tools that need access to the original AST
can call typecheckModule directly, but of course this requires using the
GHC API directly. Moreover, even when using the GHC API directly anyway (as
in my case), it means that tools cannot take advantage of ghc's
infrastructure for dependency tracking, recompiling only changed modules,
etc.
Hence it would be useful to have source plugins, which can be used both
externally and when using ghc API (in the latter case I guess hooks would
be the more appropriate terminology). Currently core plugins are recorded
as part of DynFlags as
pluginModNames:: [ModuleName],
pluginModNameOpts :: [(ModuleName,String)],
This makes sense when thinking of plugins only as an external mechanism,
but is less convenient when using them as internal hooks, too. In my draft
patch I introduce a new type HscPlugin (described shortly) and added
sourcePlugins :: [HscPlugin],
to DynFlags. HscPlugin is a record of a pair of functions; having the
actual record here rather than a module name means that these functions
can have a non-empty closure, which is obviously convenient when using this
as a hook rather than an external plugin.
In my current version HscPlugin looks like
data HscPlugin = HscPlugin {
runHscPlugin :: forall m. MonadIO m
= DynFlags
- TcGblEnv
- m TcGblEnv
, runHscQQ :: forall m. MonadIO m
= Env TcGblEnv TcLclEnv
- HsQuasiQuote Name
- m (HsQuasiQuote Name)
}
runHscPlugin is the main function; it gets passed the TcGblEnv (which
contains the type checked AST as its tcd_binds field).
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Patch/feature proposal: Source plugins
Uh. I'm sorry, I don't know why that email got sent, I was still
writing it. Please ignore it for now, will send the full version later :)
On Wed, Jun 5, 2013 at 12:14 PM, Edsko de Vries edskodevr...@gmail.comwrote:
Hi all,
The plugin mechanism gives access to the program in Core; this suffices
for many but not quite all purposes. Tools that need access to the original
AST can call typecheckModule directly, but of course this requires using
the GHC API directly. Moreover, even when using the GHC API directly anyway
(as in my case), it means that tools cannot take advantage of ghc's
infrastructure for dependency tracking, recompiling only changed modules,
etc.
Hence it would be useful to have source plugins, which can be used both
externally and when using ghc API (in the latter case I guess hooks would
be the more appropriate terminology). Currently core plugins are recorded
as part of DynFlags as
pluginModNames:: [ModuleName],
pluginModNameOpts :: [(ModuleName,String)],
This makes sense when thinking of plugins only as an external mechanism,
but is less convenient when using them as internal hooks, too. In my draft
patch I introduce a new type HscPlugin (described shortly) and added
sourcePlugins :: [HscPlugin],
to DynFlags. HscPlugin is a record of a pair of functions; having the
actual record here rather than a module name means that these functions
can have a non-empty closure, which is obviously convenient when using this
as a hook rather than an external plugin.
In my current version HscPlugin looks like
data HscPlugin = HscPlugin {
runHscPlugin :: forall m. MonadIO m
= DynFlags
- TcGblEnv
- m TcGblEnv
, runHscQQ :: forall m. MonadIO m
= Env TcGblEnv TcLclEnv
- HsQuasiQuote Name
- m (HsQuasiQuote Name)
}
runHscPlugin is the main function; it gets passed the TcGblEnv (which
contains the type checked AST as its tcd_binds field).
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Patch/feature proposal: Source plugins
Sorry for the earlier mishap, here's the full email.
Hi all,
The plugin mechanism gives access to the program in Core; this suffices for
many but not quite all purposes. Tools that need access to the original AST
can call typecheckModule directly, but of course this requires using the
GHC API directly. Moreover, even when using the GHC API directly anyway (as
in my case), it means that tools cannot take advantage of ghc's
infrastructure for dependency tracking, recompiling only changed modules,
etc.
Hence it would be useful to have source plugins, which can be used both
externally and when using ghc API (in the latter case I guess hooks would
be the more appropriate terminology). Currently core plugins are recorded
as part of DynFlags as
pluginModNames:: [ModuleName],
pluginModNameOpts :: [(ModuleName,String)],
This makes sense when thinking of plugins only as an external mechanism,
but is less convenient when using them as internal hooks, too. In my draft
patch I introduce a new type HscPlugin (described shortly) and added
sourcePlugins :: [HscPlugin],
to DynFlags. HscPlugin is a record of a pair of functions; having the
actual record here rather than a module name means that these functions
can have a non-empty closure, which is obviously convenient when using this
as a hook rather than an external plugin.
In my current version HscPlugin looks like
data HscPlugin = HscPlugin {
runHscPlugin :: forall m. MonadIO m
= DynFlags
- TcGblEnv
- m TcGblEnv
, runHscQQ :: forall m. MonadIO m
= Env TcGblEnv TcLclEnv
- HsQuasiQuote Name
- m (HsQuasiQuote Name)
}
runHscPlugin is the main function; it gets passed the TcGblEnv (which
contains the type checked AST as its tcd_binds field) and gets a change to
return it modified (we don't currently take advantage of that; I did that
only to be in line with core plugins).
Unfortunately, the typechecked AST is only a subset of the renamed AST (see
http://www.haskell.org/pipermail/ghc-devs/2013-February/000540.html). The
TcGblEnv contains a tcg_rn_decls field, which is a reference to the full
renamed (as opposed to typechecked) AST, but by default this is not
initialized: the typechecker only optionally retains the renamed AST, and
this is hardcoded to by False. In my current patch I have changed this so
that it's hard coded to be True; ideally this should become an option in
DynFlags (more ideal still would be if the type checked AST would not lose
any information).
Unfortunately, even the renamer loses information: quasi-quotes get
expanded during renaming and all evidence of that there was ever a
quasi-quote there has disappeared when the renamer returns. For this
reason, the HscPlugin type that I'm using at the moment also has a hook for
quasi-quotes.
So what I have currently done is:
1. Introduced the above HscPlugin type and added a corresponding field
to DynFlags
2. Call runHscQQ in the renamer whenever a quasi-quote gets expanded.
3. Make sure that the typechecker passes the result of the renamer
through.
4. Call runHscPlugin on the result of the typechecker.
In my client code I then combine the information obtained from these three
sources (2, 3, 4).
The path of least resistance for me currently to submit this as a patch for
ghc therefore be to submit a patch that does precisely what I described
above, mutatis mutandis based on your feedback, except that I would add an
option to add to DynFlags that would tell the type checker whether or not
to pass the result of the renamer through, rather than hardcoding it.
It is a little bit messy mostly because parts of the AST get lost along the
way: quasi-quotes in the renamer, data type declarations and other things
during type checking. A more ideal way, but also more time consuming, would
be to change this so that the renamer leaves evidence of the quasi-quotes
in the tree, and the type checker returns the entire tree type checked,
rather than just a subset. I think that ultimately this is the better
approach, at least for our purposes -- I'm not sure about other tools, but
since this would be a larger change that affects larger parts of the ghc
pipeline I'm not sure that I'll be able to do it.
Any and all feedback on the above would be appreciated!
Edsko
___
ghc-devs mailing list
ghc-devs@haskell.org
http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
I very much support moving to all-submodules. In fact, I argued for all-submodules when we made the half-submodules transition last year. Being able to easily check out a consistent and complete source code tree in a repeatable way is extremely important. Checking out by date works if you have dated history in your git reflog. For example, see: http://stackoverflow.com/questions/6990484/git-checkout-by-date In general, git commits are *not* time ordered, so asking for the version at a particular time is not well-defined across different working repositories. The GHC HQ buildbots dump fingerprints in a form that is usable directly with fingerprint.py. You can get these fingerprints from the ghc-builds@ archive. Unfortunately there was a large gap after MSR moved buildings where our builds did not run, but things are more or less working now. I believe Ben's buildbot package dumps fingerprints in a form that needs to be massaged before fingerprints.py can deal with it. Geoff On 06/05/2013 11:32 AM, Niklas Larsson wrote: When I was fiddling with having to rollback everything to a known good state I patched sync-all to checkout all the repos to the state they were in on a certain date, it's pretty naive, but it should be usable for doing manual bisecting at least. I can't find the old mailing list archives, so I attach the patch here. Niklas 2013/6/5 Austin Seipp ase...@pobox.com (Warning: incoming answer, followed by a rant.) Base is not a submodule, meaning that there is essentially no way to automatically check it back out to the exact same state it was in, given some specified GHC commit - the commit IDs are not tracked. At this point, you are basically on your own. You'll have to manually checkout libraries/base to a specific commit that occurred 'around' the same time as the GHC commit. In this case, that means looking through whatever commits hit HEAD on May 7th: $ cd libraries/base $ git log --until=May 7th The resulting list will show you what happened up to may 7th. Take the latest commit in that list, and check out base to that revision. Any commits afterword happened on may 8th or later: $ git checkout -b temporary-io-fix sha1 of latest May 7th commit You're going to need to do this for every module that is not tracked as a submodule. Most of the repositories are very low-activity. base testsuite are going to be the annoying ones. You'll have to continue this 'manual bisection' by hand, with a very hefty dose of frustrating trial-and-error, in my experience. There is a secondary alternative. GHC has a script called 'fingerprint.py' (in utils/fingerprint/) which is somewhat designed to work around this deficiency (very poorly.) This script basically dumps out a text file, containing a key/value pair mapping every repository to its current HEAD commit. It can then take that text file and automatically do 'git checkout' for you in every repo. The idea is you can take fingerprints of the tree, save the results, and cleanly check out to some state later. The GHC build bots run by Ben L.'s Buildbox library automatically runs the 'fingerprint.py' script during nightly-builds, from what I remember. It may be possible to just look in the ghc-builds archives, and steal some fingerprints from the last month off one of the buildbots. I don't know who maintains the individual bots; perhaps you can ask the list. However, this will at best give you a 1-day level of granularity, rather than commit level granularity, which is still rather unsatisfying. - Answer over, rant begins. - I know we had this discussion sometime recently I think, but can someone *please* explain why we are in this situation of half submodules, half random-floating-git-repository-checkouts? It's terrible. I'm frankly surprised we've even been doing it this long, over a year or more? It is literally the worst of submodules, and free-standing-repositories put together, with none of the advantages of either. Free-standing repos are attractive because they are just there, and you don't have to 'maintain' them (sort of.) Submodules are attractive because they identify the critical points in which your repositories depend on each other. We have neither benefit right now, clearly. In particular, this makes it impossible to use tools like 'git bisect' which is *incredibly* useful for just these exact cases. Hell, you can even make 'git bisect' work almost 100% automatically with a tiny bit of shell scripting. http://mainisusuallyafunction.blogspot.com/2012/09/tracking-down-unused-variables-with.html You could just instead have a script that built the compiler, and ran the built compiler on your testcase, after every
Re: how to checkout proper submodules
I don't know much about subtrees, but that might be another possibility? There are a lot of things to recommend moving to github. I do hate (non-empty) merge commits, though, so I'm not a fan of github's pull request mechanism. Geoff On 06/05/2013 09:43 AM, Manuel M T Chakravarty wrote: I agree with Austin and Johan. It's a bizarre setup. Submodules have their pain points (which we already have to deal with), but the ability to properly snapshot and branch the whole tree would be a serious benefit IMO. Manuel PS: While we are at it, why don't we just have the main repos on GitHub and use forks and pull requests like the rest of the world? (Using Git, but not GitHub's superb infrastructure, seems like a terrible waste to me.) Simon Peyton-Jones simo...@microsoft.com: For the avoidance of doubt, I totally support what Austin and Johan are saying: I find the current setup confusing too. I'm totally persuaded of the merits of git bisect etc. I am the opposite of a git power-user (a git weedy-user?). I will be content to do whatever I'm told workflow-wise, provided I am told clearly in words of one syllable. I *very strongly* want to reduce barriers to entry for would-be contributors, and this is clearly a barrier we could lower. Making Kazu, Austin, Johan, etc more productive is massively valuable. There may be some history to how we arrived at this point, but that should not constrain for the future. We can change our workflow. I would want Ian and Simon to be thoroughly on board, but I regard the current setup as totally open to improvement. Please! BTW, Ian has written it up quite carefully here: http://hackage.haskell.org/trac/ghc/wiki/Repositories, and the linked page http://hackage.haskell.org/trac/ghc/wiki/Repositories/Upstream. Simon | -Original Message- | From: ghc-devs-boun...@haskell.org [mailto:ghc-devs-boun...@haskell.org] | On Behalf Of Austin Seipp | Sent: 05 June 2013 07:35 | To: Johan Tibell | Cc: ghc-devs@haskell.org | Subject: Re: how to checkout proper submodules | | I absolutely agree here, FWIW. We should only do this if there is a | clear consensus on doing so and everyone doing active development is | comfortable with it. And it's entirely possible submodules are | inadequate for some reason that I'm not aware of which is a | show-stopper. | | However, the notion of impact-on-contributors cuts both ways. GHC has | an extremely small team of hackers as it stands, and we are lucky to | have *amazing* contributors like Kazu, Andreas, yourself, Simon | Simon, and numerous others help make GHC what it is. Much of this is | volunteer work. But as the Haskell community grows, and we are at a | loss of other full-time contributors like Simon Marlow, I think we are | beginning to see the strain on GHC and its current contributors. So, | it's important to evaluate what we're doing right and wrong. This | feedback loop is always present even if seasoned contributors can live | with it - but new contributors will definitely be impacted. | | In this instance, I honestly find it disheartening that the answer to | things like getting older revisions of the source code in HEAD, or | techniques like bisection is basically that doesn't work. The second | is unfortunate, but the latter is pretty legitimately worrying. It | would be one thing if this was a one-off occurrence of some odd | developer-workflow. But I have answered the fundamental question here | (submodules vs free-floating clones) a handful of times myself at | least, experienced the pain of the decision myself when doing | rollbacks, and I'm sure other contributors can say the same. | | GHC is already a large, industry-strength software project with years | of work put behind it. The barrier to entry and contribution is not | exactly small, but I think we've all done a good job. I'd love to see | more people contributing. But I cannot help but find these discussions | a bit sad, where contributors are impaired due to regular/traditional | development workflows like rollbacks are rendered useless - due to | some odd source control discrepancy that nobody else on the planet | seems to suffer from. | | I guess the short version is basically that that you're absolutely | right: the time of Simon, Ian, and other high-profile contributors is | *extremely* important. But I'd also rather not have people like Kazu | potentially spend hours or even days doing what simple automation can | achieve in what is literally a few keystrokes, and not only that - par | for the course for other projects. This ultimately impacts the | development cycles of *everybody*. And even if Kazu deals with it - | what about the next person? | | On Wed, Jun 5, 2013 at 12:12 AM, Johan Tibell johan.tib...@gmail.com | wrote: | The latest git release has improved submodules support some so if we now | thing the benefits of submodules outweigh the costs we can
Patch/feature proposal: Provide access to the runStmt sandbox ThreadID
Hi all, This proposal is related to http://hackage.haskell.org/trac/ghc/ticket/1381, which Simon Marlow closed through commit https://github.com/ghc/ghc/commit/02c4ab049adeb77b8ee0e3b98fbf0f3026eee453. The problem, in a nutshell, is how do we terminate a code snippet started with runStmt? Before Simon's patch the only way was to disable ghc's sandboxing, so that the snippet would run in the same thread as the thread that called runStmt in the first place, and then send an asynchronous exception to that thread. This is the approach we used to take. It's a little tricky to get right (have to make sure that these exceptions are thrown only at the right times), but we thought we had it working okay. Until, that is, we realized we had a very nasty problem: snippets were being unexpected interrupted. To debug this, we introduced a CustomUserInterruption data type to serve as the exception that we were throwing. This had two purposes: first, we would be use that if we saw a CustomUserInterrupt that it could only have come from one particular throw, and second, the CustomUserInterrupt carried an integer which was incremented on every throw so that we never threw the same exception twice. What we realized is that snippets were being interrupted by *old* exceptions; that is, exceptions that we had thrown to *previous* snippets (and had been caught too). This should obviously never happen. Ian managed to reproduce this behaviour in a completely different setting ( http://hackage.haskell.org/trac/ghc/ticket/5902#comment:5) and we think that something similar (unsafePerformIO related) must be happening inside ghc. Moreover, a further conjecture is that this only happens when runStmt is still compiling the snippet to be executed (as opposed to the snippet actually executing) -- i.e., that the exception gets stuck in the bowels of ghc somewhere. We don't have any hard evidence for this, other than the problem has not appeared again with the proposed patch (but that might be luck, as it depends on timing). The patch as we currently have it is against 7.4.2, so pre Simon's change for the sandbox behaviour -- but I don't think that Simon's changes affect the above problem. The core of our patch is -sandboxIO :: DynFlags - MVar Status - IO [HValue] - IO Status -sandboxIO dflags statusMVar thing = +sandboxIO :: DynFlags - MVar Status - MVar (Maybe ThreadId) - IO [HValue] - IO Status +sandboxIO dflags statusMVar tidMVar thing = mask $ \restore - -- fork starts blocked - let runIt = liftM Complete $ try (restore $ rethrow dflags thing) + let thing' = gbracket (myThreadId = putMVar tidMVar . Just) + (\() - modifyMVar_ tidMVar (\_ - return Nothing)) + (\() - thing) + runIt = liftM Complete $ try (restore $ rethrow dflags thing') in if dopt Opt_GhciSandbox dflags then do tid - forkIO $ do res - runIt putMVar statusMVar res -- empty: can't block That is, sandboxIO takes an additional MVar (Maybe ThreadId): 1. Initially this MVar should be empty. The MVar gets initialized to Just the Thread ID of the sandbox when the user code starts running. This means that if an secondary thread attempts to read the MVar (in order to kill the snippet), that secondary thread will block until the user code starts running -- it will not interrupt ghc when compiling the snippet. 2. When the user code exists the MVar is updated to be Nothing. This means that if the auxiliary thread reads the MVar and finds Nothing it knows that the snippet has already terminated. 3. When the auxiliary thread finds Just a thread ID (it must use withMVar rather than readMVar to avoid a race condition) it can safely throw the asynchronous exception. The remainder of the patch just propagates these changes up so that runStmt takes this MVar as an argument, too. Probably when integrating the patch into ghc it would be better to leave runStmt along and provide a runStmt' that takes the additional argument. Again, any and all feedback on the above would be appreciated. Edsko ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
Hi Geoffrey, I don't know much about subtrees, but that might be another possibility? the main point about subtrees is, that you've just one repository and you're merging a directory of this repository with 'git subtree' with some other git repository. subtrees and submodules both try to handle the use case if you want to incorporate a third party repository into your own repository and would like to merge the changes in both directions. I think that subtrees are easier for the developer working on the repository, because there's only one repository, but it's a bit more hassle merging the third party repository. submodules are harder for the developer, because there're multiple repositories, but merging the third party repository might be a bit easier. GHC devs might have other reasons for using submodules, because they want to separate things or they're afraid that the resulting one repository might get too big, but I think that there should be good reasons for using submodules, because a lot of workflows (like branching) are such a hassle with submodules. Greetings, Daniel ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: Validate fails: Var/Type length mismatch: [] [s{tv a15I} [tv]]
On Wed, Jun 05, 2013 at 12:17:07PM +0200, Jan Stolarek wrote: I think that Iavor is facing the same problems that I reported on this list on May 15th. I also see ghcpkg01 failing (when run during the validation) and passing (when run separately). When you ran it separately, did you say BINDIST=YES ? If you didn't then it would have used the inplace package.conf.d, not the bindisttest one. I recently added some ghc-pkg check -v calls in the validate script incidentally, so if you log the output of the validate run then that might give some more clues as to when/how it goes wrong. Actual stderr output differs from expected: --- /dev/null 2013-05-14 15:38:10.77100 +0200 +++ ../../libraries/base/tests/IO/T3307.run.stderr 2013-05-15 09:21:45.695049002 +0200 @@ -0,0 +1,2 @@ +WARNING: cache is out of date: /dane/uczelnia/projekty/ghc-validate/bindisttest/install dir/lib/ghc-7.7.20130514/package.conf.d/package.cache + use 'ghc-pkg recache' to fix. *** unexpected failure for T3307(normal) I think that these problems are likely to be caused be build artifacts left from previous validation - package.cache files (and many others) are not removed by `make maintainer-clean`. test_bindist in bindisttest/ghc.mk removes the entire bindisttest/install dir tree before installing the bindist there. Thanks Ian -- Ian Lynagh, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
On Wed, 2013-06-05 at 15:24 +0200, Daniel Trstenjak wrote: because a lot of workflows (like branching) are such a hassle with submodules. As my experience with submodules is positive (though limimted), could you elaborate on the difficulties/hassle here? Thanks, Nicolas ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
Hi Nicolas, On Wed, Jun 05, 2013 at 03:27:09PM +0200, Nicolas Trangez wrote: As my experience with submodules is positive (though limimted), could you elaborate on the difficulties/hassle here? If you would like to develop some kind of feature which involves changes on multiple repositories/submodules and you would like to do it in a branch, than you have to create a branch in each repository, commit separately in each repository and than merge back each repository into its master branch. Greetings, Daniel ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
I'm back after sleep. A few points: 1) Subtree is - in my opinion - basically not an option. It has a nice workflow from the small amount of time I spent with it. But it's not installed by default with git, it's unclear if it ever will be. Although subtree gives the appearance of a unified repository from my understanding, in practice all developers will probably need to touch multiple repositories for several reasons anyway (like testsuite and base.) That means the third-party merge is pretty much always going to happen for any non-sizeable work, the person who *did* the work will be the one doing it, essentially amounting to basically everyone needing subtree in the long term. I may be wrong about this. If I am please call me out on it. And there may be alternative workflows for patch-submitters to help this. But in general, I'd rather not have to tell GHC developers they probably need a special git build in the long haul. 2) I agree with John Lato. I think the immediate problem of fixing the submodule situation is a core issue, and GitHub can come later. Or at the very least, we should discuss GitHub in its own email thread. That's because while I see the problem of our current setup is bad as rather obvious and with a clear mitigation/fix, there *are* some legitimate complaints about GitHub that won't be resolved so easily. We should tackle each separately (remember: we have thousands of existing tickets, wiki pages, historical existing links, etc. All of these are pretty important in a lot of ways. It's not clear what the movement-strategy here is and it is definitely not going to be free, or painless.) This is definitely a more touchy issue, but I can see both sides. 3) Regarding Daniel Trstenjak's complaint: submodules from a workflow perspective may suck a little, but realistically we use their *exact* workflow anyway as it stands. We just don't get any of the benefits: in practice developers will make branches in each affected repo and push them and maintain them concurrently. Eventually they will be merged into master for each respective repository. This process will not change if we move entirely to submodules as you said. Some extra food for thought: 1) We could now delete ./sync-all if this happened. It's almost 1000 lines of code dedicated to managing this stuff. Instead, we merely tell all hackers to clone with 'git clone --init --recursive' and viola! After a git clone, you can immediately start building. That'd be great. 2) One thing this *does* complicate is that currently, some repositories are optional. Submodules effectively make them 100% non-optional. Now, normally, I would say all developers should have every relevant library anyway. In this case however, it is a tad bit annoying. On my ARM machines for example, DPH regularly fails late-in-build due to a bug in the (custom) linker, because dph requires stage2+ghci. But it also takes a long time to build DPH, so in practice I just remove it to save myself that time. Some others do the same. That said, I'm potentially the vast minority here, and I'd be willing to just deal with it in the mean time if we can do this (this is the *exception* and certainly not the rule.) Not that big a deal, and it can also be fixed later. There are probably other things that I can't think of, but I'm sure you can all think of other stuff too. :) On Wed, Jun 5, 2013 at 8:49 AM, Daniel Trstenjak daniel.trsten...@gmail.com wrote: Hi Nicolas, On Wed, Jun 05, 2013 at 03:27:09PM +0200, Nicolas Trangez wrote: As my experience with submodules is positive (though limimted), could you elaborate on the difficulties/hassle here? If you would like to develop some kind of feature which involves changes on multiple repositories/submodules and you would like to do it in a branch, than you have to create a branch in each repository, commit separately in each repository and than merge back each repository into its master branch. Greetings, Daniel ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Regards, Austin - PGP: 4096R/0x91384671 ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
1) We could now delete ./sync-all if this happened. In that case I would vote for replacing sync-all with a script that aids in managing branches in multiple subrepos. I implemented such a script for myself in a very ad hoc way. Having something more robust would be great. 2) One thing this *does* complicate is that currently, some repositories are optional. (...) I believe this could be solved by changes in the build system, so that some components can be optional (yes, I also delete DPH to speed up building). Janek ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
Hi Austin, On Wed, Jun 05, 2013 at 09:41:56AM -0500, Austin Seipp wrote: But it's not installed by default with git, it's unclear if it ever will be. I think subtree has been part of git since 1.7.x . I have just installed the default git package (git 1.8.1.2) of Ubuntu 13.04 and the subtree command is just there. Although subtree gives the appearance of a unified repository from my understanding, in practice all developers will probably need to touch multiple repositories for several reasons anyway (like testsuite and base.) That means the third-party merge is pretty much always going to happen for any non-sizeable work, the person who *did* the work will be the one doing it, essentially amounting to basically everyone needing subtree in the long term. Sorry that I'm not aware of the GHC development process, but why are the testsuite and base in separate repositories? submodules are fine for tracking repositories, but if you're all the time changing multiple submodules, than it's a sign that you've a strong dependency between the repositories, so why not just having one repository? 2) One thing this *does* complicate is that currently, some repositories are optional. Submodules effectively make them 100% non-optional. Now, normally, I would say all developers should have every relevant library anyway. In this case however, it is a tad bit annoying. On my ARM machines for example, DPH regularly fails late-in-build due to a bug in the (custom) linker, because dph requires stage2+ghci. But it also takes a long time to build DPH, so in practice I just remove it to save myself that time. Some others do the same. Isn't this more a build system issue, that you're able to specify what should/shouldn't be build, than a repository issue? Greetings, Daniel ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
On Wed, Jun 5, 2013 at 10:20 AM, Daniel Trstenjak daniel.trsten...@gmail.com wrote: I think subtree has been part of git since 1.7.x . I have just installed the default git package (git 1.8.1.2) of Ubuntu 13.04 and the subtree command is just there. It's *part* of mainline git, but it is not installed with git. It's part of git's contrib functionality package which requires that your package maintainer be gracious enough to include it and install it by default, which requires extra intervention at build-time. As a counter-example, my 'git' from Ubuntu 12.04 LTS machine has no subtree and there are no existing instances of it in any 'precise' repositories. I'm hesitant to require developers en masse to use it for reasons like this. (Frankly I also don't know how this would work out on windows. Like, I don't know how to get a git-build-with-subtree-for-windows, much less if it works on windows at all.) Sorry that I'm not aware of the GHC development process, but why are the testsuite and base in separate repositories? Because GHC does not technically 'own' them by the most strict definition. testsuite and base are also useful for other compilers, such as nhc98 (and indeed, nhc uses base itself.) The same can be said of nofib. As a result, there is a separation. Now, in practice everybody working on base is a GHC hacker pretty much, and ditto with testsuite/nofib. Regardless of all that, to change *this* part of the equation is a much, much bigger argument. One I don't intend to wage at the moment. submodules are fine for tracking repositories, but if you're all the time changing multiple submodules, than it's a sign that you've a strong dependency between the repositories, so why not just having one repository? I would agree. In practice many of the submodules are touched extremely rarely - one change every several months. Sometimes, no changes at all between entire releases spanning a year. testsuite and base are definitely the exception, but they are also what most people spend their time with in terms of hacking (pareto in action; 80% of peoples work, 20% of the code.) But again, to change this is a far larger argument with historical implications, and implications beyond GHC. Malcolm would certainly have input as he maintains nhc. (In the past, from my understanding, nhc etc were more prevalent. But over time we've moved more and more to GHC, and 'cruft' has arguably lingered.) I think folding base and testsuite into GHC 'for good' is a separate discussion entirely. Isn't this more a build system issue, that you're able to specify what should/shouldn't be build, than a repository issue? Yes. It is not insurmountable, my point is more it's an immediate loss for some small reasons, but really nothing more than a minor annoyance. It's just something to remind people of, should we make the change. Greetings, Daniel ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs -- Regards, Austin - PGP: 4096R/0x91384671 ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
On Tue, Jun 04, 2013 at 09:05:58PM -0500, Austin Seipp wrote: I know we had this discussion sometime recently I think, but can someone *please* explain why we are in this situation of half submodules, half random-floating-git-repository-checkouts? Submodules are very handy for libraries that someone else maintains: We can make a local change to the library when we need something fixed, and then, when upstream has a fix too, we can jump straight to their fix without having to do any merging. However, submodules have various disadvantages, e.g. http://codingkilledthecat.wordpress.com/2012/04/28/why-your-company-shouldnt-use-git-submodules/ The main one for me is that it's fairly easy to lose local changes when using submodules. This is relatively unimportant for the libraries that someone else maintains, as we don't often make any local changes to lose. Even so, I've lost changes on a couple of occasions. So the reason we entered this state is that we didn't think the advantages outweighed the disadvantages for the other repositories. Thanks Ian -- Ian Lynagh, Haskell Consultant Well-Typed LLP, http://www.well-typed.com/ ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
I think that testsuite should be included in the main GHC repo. I don't recall any other project that has its tests placed in a separate repository. The nhc argument doesn't convince me - after all, most test that are added nowadays are GHC specific. Janek ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Dom Silv�rio - Lista dos aprovados em concurso
Dom Silvério ANA CAROLINA PINTO COSTA, LISLY KATELLY DE PAULA MARTINS, FRANCISCO HELSON DE LIMA NERES, PAULO RAFAEL PEREIRA SOARES, JOÃO CARLOS MOREIRA DE CARVALHO, DAMIÃO JOVENAL DOS SANTOS, MARIA GORETTI LIMA FREIRE, JANIMERY BARBOSA DE ABREU MELO. SHYSLAINE ARAÚJO BEZERRA, ARIANE SOARES SILVA, LUCAS MOREIRA DIAS, GILSON POLICARPO DE SÁ, REBECA DE FREITAS BARROS. Cachoeira Dourada. Altaneira AMANDA SILVA DE MELO, LEONEL LOPES FERNANDES, FRANCISCO ALBERTO DA SILVA, PÂMELA VIRGÍNIA DE SOUZA, JOÃO CARLOS MOREIRA DE CARVALHO, DAISY CHRISTINE MELO NOGUEIRA, MARIA DENISE FEITOSA, JACQUELINE RAQUEL MENDES PATRIOTA. SAULO DEMIAN FERREIRA MAIA, ÉRICA BEZERRA DA SILVA, MAYRA FIGUEIRÊDO PEREIRA, JUAN JACKSON HOLANDA PONTE RIBEIRO, WESLEY ROGERIO ALVES. Castanheira. Antônio Carlos ANA CAROLINA PINTO COSTA, LISLY KATELLY DE PAULA MARTINS, FRANCISCO HELSON DE LIMA NERES, PAULO RAFAEL PEREIRA SOARES, JOÃO CARLOS MOREIRA DE CARVALHO, DAMIÃO JOVENAL DOS SANTOS, MARIA GORETTI LIMA FREIRE, JANIMERY BARBOSA DE ABREU MELO. SHYSLAINE ARAÚJO BEZERRA, ARIANE SOARES SILVA, LUCAS MOREIRA DIAS, GILSON POLICARPO DE SÁ, REBECA DE FREITAS BARROS. Castanheira. ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
Re: how to checkout proper submodules
There are a lot of things to recommend moving to github. I do hate (non-empty) merge commits, though, so I'm not a fan of github's pull request mechanism. Please read A successful Git branching model to know why fast-forward is not used recently. Git flow: http://nvie.com/posts/a-successful-git-branching-model/ Another relating article is here: Github flow: http://scottchacon.com/2011/08/31/github-flow.html --Kazu ___ ghc-devs mailing list ghc-devs@haskell.org http://www.haskell.org/mailman/listinfo/ghc-devs
