Re: Opening up Cabal development
Yep, of course. (I sometimes get the feeling that a less good way people do this is by just not documenting the feature :o) Excerpts from Paolo G. Giarrusso's message of 2016-07-17 04:27:14 -0700: > Edward Z. Yang mit.edu> writes: > > > Excerpts from Herbert Valerio Riedel's message of 2016-07-13 23:40:06 -0700: > > > I.e. write up a specification/proposal outlining motivation (i.e. what > > > problem does this solve), specify what the changes are exactly (syntax & > > > semantics), what the consequences are, and so on. > > > > > > Then we inevitably need some criteria to decide whether a proposal is > > > accepted and approved for implementation. This could be formally in the > > > hands of the core library committee together with the Hackage trustees > > > (since we do have quite some experience with .cabal files and care a lot > > > about the Hackage ecosystem). > > > > Why can't we release experimental changes to the Cabal specification? > > Neither setup-depends nor convenience libraries went through any formal > > proposal mechanism. If a feature is good people will start using it > > and we will have to continue supporting it. If a feature is bad/not > > useful we can yank it from the next release; anyone using an > > experimental feature like this isn't trying to maximize their > > Cabal compatibility anyway. > > Important nitpick: to do that, experimental features should be > marked as experimental and this policy should be made explicit. > That'd be kinder to the one user who starts using it without > realizing. > > I guess that goes without saying (probably in ways I don't > realize), but hey :-D > ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
Edward Z. Yang mit.edu> writes: > Excerpts from Herbert Valerio Riedel's message of 2016-07-13 23:40:06 -0700: > > I.e. write up a specification/proposal outlining motivation (i.e. what > > problem does this solve), specify what the changes are exactly (syntax & > > semantics), what the consequences are, and so on. > > > > Then we inevitably need some criteria to decide whether a proposal is > > accepted and approved for implementation. This could be formally in the > > hands of the core library committee together with the Hackage trustees > > (since we do have quite some experience with .cabal files and care a lot > > about the Hackage ecosystem). > > Why can't we release experimental changes to the Cabal specification? > Neither setup-depends nor convenience libraries went through any formal > proposal mechanism. If a feature is good people will start using it > and we will have to continue supporting it. If a feature is bad/not > useful we can yank it from the next release; anyone using an > experimental feature like this isn't trying to maximize their > Cabal compatibility anyway. Important nitpick: to do that, experimental features should be marked as experimental and this policy should be made explicit. That'd be kinder to the one user who starts using it without realizing. I guess that goes without saying (probably in ways I don't realize), but hey :-D ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
I pulled the trigger. If you are mad, talk to me. https://github.com/haskell/cabal/issues/3567 Edward Excerpts from Edward Z. Yang's message of 2016-07-13 16:32:39 -0700: > Hey all, > > Paolo Giarrusso suggested that the Cabal project might look into > giving more people commit bits, ala > http://felixge.de/2013/03/11/the-pull-request-hack.html > > I think we do need to give more commit bits. There's a sliding scale of > how extreme we can go: > > 1. We can adopt this wholesale: you PR, your GitHub profile checks > out, we give you bits. > 2. If we accept your PR, we give you bits. > 3. If we accept your PR for a new feature, we give you bits. > > For (2), I trawled the commit history for 2016 and here's what I think > is all the commits from people without bits. I apologize if I missed > anyone (I just went through the commit list by hand.) > > @thumphries > https://github.com/haskell/cabal/commit/ce2ffb24902816ab02f6c6b50921fb3a9a8b92aa > @sjakobi > https://github.com/haskell/cabal/commit/bb9501bc23914a50684f288bc88cf04747d69c32 > @accraze > https://github.com/haskell/cabal/commit/11650b26ecf0ba95c3a7e747c0968735e906c3b0 > @headprogrammingczar > https://github.com/haskell/cabal/commit/e9883dced04e6a16e25752cc8513ce26325b4b4b > > https://github.com/haskell/cabal/commit/0252a0aed79e881d4f6210d9296216aef3c301d0 > @randen https://github.com/haskell/cabal/commits/master?author=randen > @aisamanra > https://github.com/haskell/cabal/commit/6e2fca4314a01f07d520bc7b7669e70f70a231ce > @kolmodin > https://github.com/haskell/cabal/commit/0122e821825b875447f3844b349fba582fff39cf > @mightybyte > https://github.com/haskell/cabal/commit/ea01974be888d25ef918e7808aa6cf6b8aac1275 > @gbaz > https://github.com/haskell/cabal/commit/c69dfb8209dc7bfd9abf2a7e494704db652073c9 > @garetxe > https://github.com/haskell/cabal/commit/c72aa8dbb5a11fb4137bda62c9b7a99fb48b7649 > @komadori > https://github.com/haskell/cabal/commit/1da9b3533e6a0fae8692fa0f4f532ea63d43ccc8 > @mgsloan > https://github.com/haskell/cabal/commit/c10a4ca8c50290efc0b0ea65b34116ae165ccc9b > @lukexi > https://github.com/haskell/cabal/commit/5efc6341643e7fd98b33aea5a6ab96873d597787 > @mpkh > https://github.com/haskell/cabal/commit/fcaf5d02947c5fd853ff69e2326e08f0082530c1 > @pra85 > https://github.com/haskell/cabal/commit/a69b0ea23b0f00be30933a1dbb63fbc4f1306c17 > @tvestelind > https://github.com/haskell/cabal/commit/fe7b597542d57c74814620ed7a553e252b570ac7 > @corngood > https://github.com/haskell/cabal/commit/4214572306205ad2c20943ef2b3aacbc912c45d3 > @gelisam > https://github.com/haskell/cabal/commit/6b3457de66772d958fd5fe96066b08e93d0fb0c7 > > Why don't we give them all commit access! (If we want to do (1) also > look at the current PR queue.) > > If people want, we could also formalize some more rules about the state > of master, e.g., > > - The Travis build must always be green. We should prioritize > adding more tests for things we care about. Look into regular > Hackage tests. > - PR all your changes (so that you can check that Travis is green), > try to get approval for big changes but BE BOLD. > > Cheers, > Edward ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
Excerpts from Oleg Grenrus's message of 2016-07-14 02:08:54 -0700: > About convenience libraries I agree even more. We should discussed them more. > There are questions I’d might to ask, but I guess it too late It's not too late. They are not in any real release. They can be removed. > Or maybe not too late, but for another thread: > what is convenience library name clashes with the library on package, > say I define convenience library fancylib-lens, which is on Hackage, but > maybe also after I made my package? > I guess, some shadowing happens? Is there a warning? Convenience library shadows the one on Hackage. There's no warning but it would be easy to add one; however, I don't think there should be a warning--these names are just lexically scoped. > But we have quite formal feeling proposals about `provides`, > and `or-dependencies / multi-way-flag / depends`, and > formal process would help to get them thru (or say no, with an explanation) Yes. Especially when there are competing solutions and there is not an obvious "best one". > About the original topic, I’d require “ a history of quality contributions”, > so something between (2) or (3). > I don’t like many rules, also making fair but non-abusable ones is difficult, > but as we have this conversation, > the outcome should be well defined (also how we change the rules). > As with (2) does every accepted PR counts, like single character typo fix in > the documentation? > And I don’t know how to define “quality contribution” :( In the blog post linked, the suggestion was to apply discretion by looking at the GitHub profile / erstwhile contribution history. But yes, I DO think we should accept single character typo fixes! That's how you get new contributors who help spruce up the documentation. We should trust but verify. > (Maybe I’m just to scared about dramas happened elsewhere) Too late; there's already drama ;) > Related to this, a thought I had for some time: > I’d like to split Cabal and cabal-install repositories. > > Pros: > - `cabal-install` can be much more liberal in giving out commit bits, as you > cannot make too much damage here. > - we already have `hackage-security` as a dependency between them. > - the CI setup could be simplified, resulting into > - their tests could do more things: `Cabal` parses all `Hackage`, changes to > `cabal-install` doesn’t need to do it, > and vice-versa, fixing ui of cabal-install doesn’t need to do package tests > of Cabal. > - their other processes could be different too. > > Cons: Simultaneous development is almost impossible. And this is almost a > show-stopper. > - `cabal-install` would need to lag one release behind `Cabal` (development > agains released `Cabal`). > This could work as `Setup.hs` should work to install packages requiring > newer Cabal (isn’t it how it works nowadays?). > There will be a problem when newer `Cabal` won’t accept build-plan created > by solver > in older `cabal-install` (proposals mentioned above), but I hope it won’t > be a huge issue. > - To make this work we’d need to release more Cabal major versions, > so there aren’t many breaking and intrusive changes accumulating. But > that’s good thing, isn’t it? > > Or it could be a quality win to separate them. After all `cabal-install` is > not the only consumer of `Cabal`. > `stack` uses it very limitedly atm though (i.e. to extract dependencies and > maybe verify install plan?) > I don’t know what `ghc-pkg` does. > > IMHO we should develop Cabal-the-library with highest standards (starting > with having detailed CHANGELOG) for library development. > Now we might have been too focused on developing Cabal for cabal-install. > Cabal should provide quality plumbing support for “everyone”. > (if we could start with a clean desk, i’d make Setup.hs and cabal-install > commands different, that’s one confusing part). > > The separation into different repositories would make things a bit clearer: > Cabal-the-library and cabal-install are different things. (For example: > cabal-install doesn’t parse *.cabal files). Currently cabal-install is tightly coupled to Cabal. I'm all for looser coupling (e.g., https://github.com/haskell/cabal/issues/3549). Separate repositories force this, but in general looser coupling will lead to some short term pain. Edward ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
> On 14 Jul 2016, at 09:54, Edward Z. Yang wrote: > > Excerpts from Herbert Valerio Riedel's message of 2016-07-13 23:40:06 -0700: >> I.e. write up a specification/proposal outlining motivation (i.e. what >> problem does this solve), specify what the changes are exactly (syntax & >> semantics), what the consequences are, and so on. >> >> Then we inevitably need some criteria to decide whether a proposal is >> accepted and approved for implementation. This could be formally in the >> hands of the core library committee together with the Hackage trustees >> (since we do have quite some experience with .cabal files and care a lot >> about the Hackage ecosystem). > > Why can't we release experimental changes to the Cabal specification? > Neither setup-depends nor convenience libraries went through any formal > proposal mechanism. If a feature is good people will start using it > and we will have to continue supporting it. If a feature is bad/not > useful we can yank it from the next release; anyone using an > experimental feature like this isn't trying to maximize their > Cabal compatibility anyway. And any change to the format with > cabal-install can't parse is not going to be supportable via > custom-setup anyway. You gave a reason yourself: - Can we parse all of Hackage? With my industrial hat on, I don’t like the idea that I might need older Cabal, to be able to install some package. OTOH new-build could install older Cabal. So we’d need to maintain older Cabals, to be build able with newer GHCs, that’s not fun, but not impossible. I agree, that we could have discussed setup-depends more, hopefully no-one disagrees about them (and dcoutts took very conservative approach there). Also taking them out of Cabal, doesn’t mean that package becomes unbuildable, there will be just more luck involved. About convenience libraries I agree even more. We should discussed them more. There are questions I’d might to ask, but I guess it too late Or maybe not too late, but for another thread: what is convenience library name clashes with the library on package, say I define convenience library fancylib-lens, which is on Hackage, but maybe also after I made my package? I guess, some shadowing happens? Is there a warning? But we have quite formal feeling proposals about `provides`, and `or-dependencies / multi-way-flag / depends`, and formal process would help to get them thru (or say no, with an explanation) > > So how about this script, which is how web standards work: > >Cabal changes need to be informally proposed and discussed. >But the advancer can also write a PR and get it merged in >as an experimental extension, to be trialed for the next >major release series. Eventually it gets standardized. > > Edward About the original topic, I’d require “ a history of quality contributions”, so something between (2) or (3). I don’t like many rules, also making fair but non-abusable ones is difficult, but as we have this conversation, the outcome should be well defined (also how we change the rules). As with (2) does every accepted PR counts, like single character typo fix in the documentation? And I don’t know how to define “quality contribution” :( (Maybe I’m just to scared about dramas happened elsewhere) Related to this, a thought I had for some time: I’d like to split Cabal and cabal-install repositories. Pros: - `cabal-install` can be much more liberal in giving out commit bits, as you cannot make too much damage here. - we already have `hackage-security` as a dependency between them. - the CI setup could be simplified, resulting into - their tests could do more things: `Cabal` parses all `Hackage`, changes to `cabal-install` doesn’t need to do it, and vice-versa, fixing ui of cabal-install doesn’t need to do package tests of Cabal. - their other processes could be different too. Cons: Simultaneous development is almost impossible. And this is almost a show-stopper. - `cabal-install` would need to lag one release behind `Cabal` (development agains released `Cabal`). This could work as `Setup.hs` should work to install packages requiring newer Cabal (isn’t it how it works nowadays?). There will be a problem when newer `Cabal` won’t accept build-plan created by solver in older `cabal-install` (proposals mentioned above), but I hope it won’t be a huge issue. - To make this work we’d need to release more Cabal major versions, so there aren’t many breaking and intrusive changes accumulating. But that’s good thing, isn’t it? Or it could be a quality win to separate them. After all `cabal-install` is not the only consumer of `Cabal`. `stack` uses it very limitedly atm though (i.e. to extract dependencies and maybe verify install plan?) I don’t know what `ghc-pkg` does. IMHO we should develop Cabal-the-library with highest standards (starting with having detailed CHANGELOG) for library development. Now we might have been too focused on developing Cabal for cabal
Re: Opening up Cabal development
Excerpts from Herbert Valerio Riedel's message of 2016-07-13 23:40:06 -0700: > I.e. write up a specification/proposal outlining motivation (i.e. what > problem does this solve), specify what the changes are exactly (syntax & > semantics), what the consequences are, and so on. > > Then we inevitably need some criteria to decide whether a proposal is > accepted and approved for implementation. This could be formally in the > hands of the core library committee together with the Hackage trustees > (since we do have quite some experience with .cabal files and care a lot > about the Hackage ecosystem). Why can't we release experimental changes to the Cabal specification? Neither setup-depends nor convenience libraries went through any formal proposal mechanism. If a feature is good people will start using it and we will have to continue supporting it. If a feature is bad/not useful we can yank it from the next release; anyone using an experimental feature like this isn't trying to maximize their Cabal compatibility anyway. And any change to the format with cabal-install can't parse is not going to be supportable via custom-setup anyway. So how about this script, which is how web standards work: Cabal changes need to be informally proposed and discussed. But the advancer can also write a PR and get it merged in as an experimental extension, to be trialed for the next major release series. Eventually it gets standardized. Edward ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
On 2016-07-14 at 03:13:31 +0200, Edward Z. Yang wrote: [...] >> - I don’t break backwards compat for APIs without notice and >> discussion, and don’t break backwards compat for any element of cabal >> files without lots of discussion. [...] >3. Can we build all Setup.hs scripts on Hackage? This lets us know > which APIs in Cabal matter, and which ones we can change. > We'll need to establish base truth for this. While we shouldn't break the API without a good reason, we added custom-setup/setup-depends for the very reason to finally be able to break Setup.hs. Now we can specify `Setup.hs`'s dependencies with version constraints accurately, just like we already do with other library dependencies via build-depends. Moreover, cabal-install has a generous implicit upper bound on the Cabal lib version: -- The idea here is that at some point we will make significant -- breaking changes to the Cabal API that Setup.hs scripts use. -- So for old custom Setup scripts that do not specify explicit -- constraints, we constrain them to use a compatible Cabal version. -- The exact version where we'll make this API break has not yet been -- decided, so for the meantime we guess at 2.x. cabalCompatMaxVer = Version [2] [] So criteria 3 should be seen in the light of custom-setup/setup-depends. Otoh, maybe we should collect and batch up all ideas (including refactoring/aesthetic changes) to break the Cabal API for a big Cabal 2.0 rupture... :-) > It is a travesty that this infrastructure does not exist; it may even > be possible to do (1) and (3) on Travis, as they should not take too > long to do. So far, the criteria mentioned address mostly regressions. What about changes extending the .cabal syntax? There's frequently proposals to extend the features of .cabal files. I happen to propose extensions myself from time to time (motivated by the issues I see as Hackage Trustee). Since changes to .cabal are effectively changes to the CABAL specification future generations will have to cope with, I'd propose that changes affecting or introducing new .cabal syntax/semantics should go through a proposal process. I.e. write up a specification/proposal outlining motivation (i.e. what problem does this solve), specify what the changes are exactly (syntax & semantics), what the consequences are, and so on. Then we inevitably need some criteria to decide whether a proposal is accepted and approved for implementation. This could be formally in the hands of the core library committee together with the Hackage trustees (since we do have quite some experience with .cabal files and care a lot about the Hackage ecosystem). -- hvr pgpXTVc6nPbGB.pgp Description: PGP signature ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
On 14 July 2016 at 03:13, Edward Z. Yang wrote: >1. Can we parse all of Hackage? > >[...] > >3. Can we build all Setup.hs scripts on Hackage? This lets us know > which APIs in Cabal matter, and which ones we can change. > We'll need to establish base truth for this. If someone wants to look into this, the following links should be helpful: https://github.com/commercialhaskell/all-cabal-files https://github.com/23Skidoo/all-custom-setups ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
Excerpts from Gershom B's message of 2016-07-13 18:01:50 -0700: > On July 13, 2016 at 7:32:47 PM, Edward Z. Yang (ezy...@mit.edu) wrote: > > The general notion sounds good to me. I’m semi-indifferent between (1) > and (2) though conservatively lean towards the latter. > > > - The Travis build must always be green. We should prioritize > > adding more tests for things we care about. Look into regular > > Hackage tests. > > - PR all your changes (so that you can check that Travis is green), > > try to get approval for big changes but BE BOLD. > > I’d add > > - I don’t break backwards compat for APIs without notice and > discussion, and don’t break backwards compat for any element of cabal > files without lots of discussion. > > (and I don’t know what tests would be necessary to help notice this, > but they’d probably be useful!) Simple, we need Hackage-level CI. Here are few possible things to test: 1. Can we parse all of Hackage? 2. Can we build all of Stackage? https://groups.google.com/forum/#!msg/haskell-stack/oi-VJrAIJbE/FjAPVZTUAQAJ Attempting to build all of Hackage is dicey business because it is a combinatorial explosion, and you do not know what is supposed to build. But Stackage is supposed to build. Test that 3. Can we build all Setup.hs scripts on Hackage? This lets us know which APIs in Cabal matter, and which ones we can change. We'll need to establish base truth for this. It is a travesty that this infrastructure does not exist; it may even be possible to do (1) and (3) on Travis, as they should not take too long to do. Edward ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
On July 13, 2016 at 7:32:47 PM, Edward Z. Yang (ezy...@mit.edu) wrote: The general notion sounds good to me. I’m semi-indifferent between (1) and (2) though conservatively lean towards the latter. > - The Travis build must always be green. We should prioritize > adding more tests for things we care about. Look into regular > Hackage tests. > - PR all your changes (so that you can check that Travis is green), > try to get approval for big changes but BE BOLD. I’d add - I don’t break backwards compat for APIs without notice and discussion, and don’t break backwards compat for any element of cabal files without lots of discussion. (and I don’t know what tests would be necessary to help notice this, but they’d probably be useful!) —gershom ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Re: Opening up Cabal development
Hi, On 14 July 2016 at 01:32, Edward Z. Yang wrote: > Why don't we give them all commit access! (If we want to do (1) also > look at the current PR queue.) Fine with me. I tried something like this on a smaller scale previously, giving write access to a number of people with a history of quality contributions. > If people want, we could also formalize some more rules about the state > of master, e.g., > > - The Travis build must always be green. We should prioritize > adding more tests for things we care about. Look into regular > Hackage tests. > - PR all your changes (so that you can check that Travis is green), > try to get approval for big changes but BE BOLD. +1 on having a welcome page for new developers on the wiki. ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel
Opening up Cabal development
Hey all, Paolo Giarrusso suggested that the Cabal project might look into giving more people commit bits, ala http://felixge.de/2013/03/11/the-pull-request-hack.html I think we do need to give more commit bits. There's a sliding scale of how extreme we can go: 1. We can adopt this wholesale: you PR, your GitHub profile checks out, we give you bits. 2. If we accept your PR, we give you bits. 3. If we accept your PR for a new feature, we give you bits. For (2), I trawled the commit history for 2016 and here's what I think is all the commits from people without bits. I apologize if I missed anyone (I just went through the commit list by hand.) @thumphries https://github.com/haskell/cabal/commit/ce2ffb24902816ab02f6c6b50921fb3a9a8b92aa @sjakobi https://github.com/haskell/cabal/commit/bb9501bc23914a50684f288bc88cf04747d69c32 @accraze https://github.com/haskell/cabal/commit/11650b26ecf0ba95c3a7e747c0968735e906c3b0 @headprogrammingczar https://github.com/haskell/cabal/commit/e9883dced04e6a16e25752cc8513ce26325b4b4b https://github.com/haskell/cabal/commit/0252a0aed79e881d4f6210d9296216aef3c301d0 @randen https://github.com/haskell/cabal/commits/master?author=randen @aisamanra https://github.com/haskell/cabal/commit/6e2fca4314a01f07d520bc7b7669e70f70a231ce @kolmodin https://github.com/haskell/cabal/commit/0122e821825b875447f3844b349fba582fff39cf @mightybyte https://github.com/haskell/cabal/commit/ea01974be888d25ef918e7808aa6cf6b8aac1275 @gbaz https://github.com/haskell/cabal/commit/c69dfb8209dc7bfd9abf2a7e494704db652073c9 @garetxe https://github.com/haskell/cabal/commit/c72aa8dbb5a11fb4137bda62c9b7a99fb48b7649 @komadori https://github.com/haskell/cabal/commit/1da9b3533e6a0fae8692fa0f4f532ea63d43ccc8 @mgsloan https://github.com/haskell/cabal/commit/c10a4ca8c50290efc0b0ea65b34116ae165ccc9b @lukexi https://github.com/haskell/cabal/commit/5efc6341643e7fd98b33aea5a6ab96873d597787 @mpkh https://github.com/haskell/cabal/commit/fcaf5d02947c5fd853ff69e2326e08f0082530c1 @pra85 https://github.com/haskell/cabal/commit/a69b0ea23b0f00be30933a1dbb63fbc4f1306c17 @tvestelind https://github.com/haskell/cabal/commit/fe7b597542d57c74814620ed7a553e252b570ac7 @corngood https://github.com/haskell/cabal/commit/4214572306205ad2c20943ef2b3aacbc912c45d3 @gelisam https://github.com/haskell/cabal/commit/6b3457de66772d958fd5fe96066b08e93d0fb0c7 Why don't we give them all commit access! (If we want to do (1) also look at the current PR queue.) If people want, we could also formalize some more rules about the state of master, e.g., - The Travis build must always be green. We should prioritize adding more tests for things we care about. Look into regular Hackage tests. - PR all your changes (so that you can check that Travis is green), try to get approval for big changes but BE BOLD. Cheers, Edward ___ cabal-devel mailing list cabal-devel@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/cabal-devel