Hi Herbert, This sounds like a great idea :) Glad to see ideas from Eta inspiring new ideas in GHC.
"constantly nagging upstream maintainers to release quickfixes for unreleased GHC HEAD snapshots which will likely break again a few weeks later" ^ This is what motivated us to start eta-hackage in the first place. In our case, we didn't know whether the patches were completely correct when porting C FFI to Java FFI, at least in the corner cases, so we needed a way to quickly deploy updates to patches without having the end user wait a significant amount of time. Once they stabilise, we do plan on pushing the patches upstream to the respective packages - we've already done that for a few of them. Thanks, Rahul On Mon, Sep 18, 2017 at 8:04 AM, Herbert Valerio Riedel <hvrie...@gmail.com> wrote: > Hi GHC devs, > > A long-standing common problem when testing/dogfooding GHC HEAD is that > at some point during the development cycle more and more packages from > Hackage will run into build failures. > > Obviously, constantly nagging upstream maintainers to release quickfixes > for unreleased GHC HEAD snapshots which will likely break again a few > weeks later (as things are generally in flux until freeze comes into > effect) does not scale and only introduces a latency/coordination > bottleneck, and on top of it ultimately results in spamming the primary > Hackage Package index with releases (which has non-negligible negative > impact/costs of its own on the Hackage infrastructure, and thus ought to > be minimised). > > OTOH, we need the ability to easily test, debug, profile, and prototype > changes to GHC HEAD while things are still in motion, and case in point, > if you try to e.g. build `pandoc` with GHC HEAD today, you'll currently > run into a dozen or so of packages not building with GHC HEAD. > > To this end, I've finally found time to work on a side-project (related > to matrix.hackage.haskell.org) which implements a scheme tailored to > `cabal new-build`, which is inspired by how Eta copes with a very > related issue (they rely on it for stable versions of the compiler); > i.e., they maintain a set of patches at > > https://github.com/typelead/eta-hackage/tree/master/patches > > which fix up existing Hackage packages to work with the Eta compiler. > > > And this gave me the idea to use a similar scheme for GHC HEAD: > > https://github.com/hvr/head.hackage/tree/master/patches > > This folder already contains several of patches (which mostly originate > from Ryan, Ben and myself) to packages which I needed to patch in order > to build popular Hackage packages & tools. > > The main difference is how those patches are applied; Eta uses a > modified `cabal` which they renamed to `etlas` which is checks > availability of .patch & .cabal files in the GitHub repo linked above; > > Whereas for GHC HEAD with `cabal new-build` a different scheme makes > more sense: we simply generate an add-on Hackage repo, and use the > existing `cabal` facilities (e.g. multi-repo support or the nix-style > package store which makes sure that unofficially patched packages don't > contaminate "normal" install-plans, etc.) to opt into the opt-in Hackage > repo containing fixed up packages. > > > > I've tried to describe how to use the HEAD.hackage add-on repo in the > README at > > https://github.com/hvr/head.hackage#how-to-use > > > And finally, here's a practical example of how you can use it to build > e.g. the `pandoc` executable with GHC HEAD (can easily be adapted to > build your project of choice; please refer to > > http://cabal.readthedocs.io/en/latest/nix-local-build-overview.html > > to learn more about how to describe your project via `cabal.project` > files): > > > 0.) This assumes you have a recent cabal 2.1 snapshot built from Git > > 1.) create & `cd` into a new work-folder > > 2.) invoke `head.hackage.sh update` to update the HEAD.hackage index cache > > 3.) invoke `head.hackage.sh init` to create an initial `cabal.project` > configuration which locally activates the HEAD.hackage overlay repo > > 4.) If needed, edit the cabal.project file and change where GHC > HEAD can be found (the script currently assumes GHC HEAD is > installed from my Ubuntu PPA), e.g. > > with-compiler: /home/hvr/src/ghc-dev/inplace/bin/ghc-stage2 > > or you can add something like `optional-packages: deps/*/*.cabal` > to have cabal pick up package source-trees unpacked in the deps/ > folder, or you can inject ghc-options, relax upper bounds via > `allow-newer: *:base` etc (please refer to the cabal user guide) > > 5.) Create a `dummy.cabal` file (in future we will have `cabal > new-install` or other facilities, but for now we use this > workaround): > > > --8<---------------cut here---------------start------------->8--- > name: dummy > version: 0 > build-type: Simple > cabal-version: >=2.0 > > library > default-language: Haskell2010 > > -- library components you want cabal to solve & build for > -- and become accessible via .ghc.environment files and/or > -- `cabal new-repl` > build-depends: base, lens > > -- executable components you want cabal to build > build-tool-depends: pandoc:pandoc > > --8<---------------cut here---------------end--------------->8--- > > 6.) invoke `cabal new-build` > > 7.) If everything works, you'll find the `pandoc:pandoc` executable > somewhere in your ~/.cabal/store/ghc-8.3.*/ folder > (you can use http://hackage.haskell.org/package/cabal-plan > to conveniently list the location via `cabal-plan list-bins`) > > 8.) As for libraries, you can either use `cabal new-repl` > or you can leverage GHC's package environment files: > > `cabal new-build` will have generated a file like > > .ghc.environment.x86_64-linux-8.3.20170913 > > which brings into scope all transitive dependencies of > `build-depends: base, lens` > > Now all you need to do is simply call > > ghc-stage2 --make MyTestProg.hs > > to compile a program against those libs, or start up GHCi via > > ghc-stage2 --interactive > > and you'll be thrown into that package environment. > > > > I hope you find this useful > > Cheers, > Herbert > _______________________________________________ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs > -- Rahul Muttineni
_______________________________________________ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs