Re: [Haskell-cafe] Recompiling Hackage to estimate the impact of a change [Was: StricterLabelledFieldSyntax]

2009-07-26 Thread Don Stewart 
johan.tibell:
 
 In general, I think it would be a good idea to provide some statistics of how
 many packages would break as the result of a backwards incompatible change.
 Without that data I find it hard to do a cost-benefit analysis. So I hereby
 suggest that we make a recompile of Hackage a requirement for any breaking
 language changes. I understand that it might not be that easy to recompile all
 of Hackage at the moment so we should try to come up with some step-by-step
 instructions on how to do it. In the future it would be nice to have a little
 script that does it and spits out some statistics.

Agreed. And it should be required as part of release processes for GHC.

Duncan has described the process of building large amounts of Hackage,

http://blog.well-typed.com/2009/03/regression-testing-with-hackage/

We used this prior to the ghc 6.10 release to find a few interesting
type system bugs, and get in steps for what breaks and why, ultimately
easing the upgrade process so that it affected only a couple of percent
of packages.

So, +1, let's take advantage of this centralization to improve overall
quality standards.

-- Don
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] Recompiling Hackage to estimate the impact of a change [Was: StricterLabelledFieldSyntax]

2009-07-26 Thread Ian Lynagh 
On Sun, Jul 26, 2009 at 08:21:00AM +0200, Johan Tibell wrote:
 On Sun, Jul 26, 2009 at 3:34 AM, Ian Lynagh ig...@earth.li wrote:
 
  http://hackage.haskell.org/trac/haskell-prime/wiki/StricterLabelledFieldSyntax
 
 
 In general, I think it would be a good idea to provide some statistics of
 how many packages would break as the result of a backwards incompatible
 change. Without that data I find it hard to do a cost-benefit analysis.

To some extent you are right, and if we had an easy to to get those
stats when I would be in favour of doing so.

But it is important to remember that a count of packages that break
won't tell you how hard it would be to fix them. For example, the
complete diff needed to fix old-time for StricterLabelledFieldSyntax
was:

-   toClockTime cal{ctMonth=month', ctYear=year'}
+   toClockTime $ cal{ctMonth=month', ctYear=year'}


Thanks
Ian

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] Recompiling Hackage to estimate the impact of a change [Was: StricterLabelledFieldSyntax]

2009-07-26 Thread Ian Lynagh 
On Sat, Jul 25, 2009 at 11:25:12PM -0700, Donald Bruce Stewart wrote:
 johan.tibell:
  
  In general, I think it would be a good idea to provide some statistics of 
  how
  many packages would break as the result of a backwards incompatible change.
 
 Agreed. And it should be required as part of release processes for GHC.
 
 Duncan has described the process of building large amounts of Hackage,
 
 http://blog.well-typed.com/2009/03/regression-testing-with-hackage/

That process requires too much effort. IIRC it took more than a day of
actual human time to get the package list set up etc, and requires you
to make some non-obvious decisions along the way.

I think a better process would be more along the lines of:

for p in $packages
cabal install p
rollback?

except smarter, so that it doesn't try to install packages that it has
already installed as a dependency of something else.


Thanks
Ian

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

Re: [Haskell-cafe] Recompiling Hackage to estimate the impact of a change [Was: StricterLabelledFieldSyntax]

2009-07-26 Thread Johan Tibell 
On Sun, Jul 26, 2009 at 5:24 PM, Ian Lynagh ig...@earth.li wrote:

 On Sun, Jul 26, 2009 at 08:21:00AM +0200, Johan Tibell wrote:
  On Sun, Jul 26, 2009 at 3:34 AM, Ian Lynagh ig...@earth.li wrote:
 
  
 http://hackage.haskell.org/trac/haskell-prime/wiki/StricterLabelledFieldSyntax
  
 
  In general, I think it would be a good idea to provide some statistics of
  how many packages would break as the result of a backwards incompatible
  change. Without that data I find it hard to do a cost-benefit analysis.

 To some extent you are right, and if we had an easy to to get those
 stats when I would be in favour of doing so.

 But it is important to remember that a count of packages that break
 won't tell you how hard it would be to fix them. For example, the
 complete diff needed to fix old-time for StricterLabelledFieldSyntax
 was:

 -   toClockTime cal{ctMonth=month', ctYear=year'}
 +   toClockTime $ cal{ctMonth=month', ctYear=year'}


Right. So once you know what breaks you can investigate why and, as a part
of the language change proposal, show how easy/hard it would be to fix
breakages. I'm not arguing against breaking changes but for using the
available data to make decisions. For example, when a redesign of
haskell.org was brought up a while back the discussion could have greatly
benefited from looking at web server logs to give valuable insight into user
behavior on the site.

-- Johan
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe