Re: 6-weeks release cycle
On 2015-06-05 12:00, Rikki Cattermole wrote: I wonder if we could integrate a bit more with travis The problem with Travis CI is that it's limited to Linux and OS X 64bit. D supports Windows and FreeBSD and 32bit versions as well. BTW, DMD is already tested by Travis CI [1], but only for the D1 branch. I'm guessing that is for Sociomantic which still uses D1 and perhaps only on Linux. and have a big list of projects compiling + testing against HEAD. Would be an excellent indicator. Maybe even have this as part of dub repo? That would be nice. [1] https://travis-ci.org/D-Programming-Language/dmd -- /Jacob Carlborg
Re: 6-weeks release cycle
Of many things that Rust got right, this is, in my opinion, something they did wrong. Quick release cycles only make sense in both bleeding edge model and with availability of excess developer resources. No amount of planning and management can compensate for things not being ready.
Re: 6-weeks release cycle
On 5/06/2015 6:00 p.m., Dicebot wrote: Of many things that Rust got right, this is, in my opinion, something they did wrong. Quick release cycles only make sense in both bleeding edge model and with availability of excess developer resources. No amount of planning and management can compensate for things not being ready. Of course we could always get the CI to auto build a full release every day available for download. It would then be just a matter of picking one set and promoting as a version. Perfect for e.g. testing.
Re: 6-weeks release cycle
On 5/06/2015 6:10 p.m., Dicebot wrote: On Friday, 5 June 2015 at 06:08:41 UTC, Rikki Cattermole wrote: Of course we could always get the CI to auto build a full release every day available for download. It would then be just a matter of picking one set and promoting as a version. Perfect for e.g. testing. Picking random commit and calling it a version is not really releasing. For me release implies strict regression testing and nice changelog at the very least. And shouldn't the CI be doing regression testing already? Changelog perhaps should be updated with a commit then?
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 06:42:06 UTC, Rikki Cattermole wrote: And shouldn't the CI be doing regression testing already? The autotester runs the unit tests that are in druntime, Phobos, and dmd. It catches a lot of stuff and generally prevents us from merging bad code. But it can't possibly catch everything. It catches the stuff that we thought to specifically test for and bugs that were fixed (since unit tests are usually added for bug fixes). But it's not that infrequent for something _new_ to break that's never broken before and is subtle enough that it doesn't get past all of the tests - especially when you're dealing with changes to the compiler. To catch all of that stuff before it goes out the door in a release, we need to test a _much_ larger code base than just the standard stuff - which is part of why we have betas. We want folks to try out their projects with the betas so that we can catch the stuff that we missed before it gets released in an official release. Simply grabbing an arbitrary commit and declaring it a release just because it's at about the time that we want to do a release would be a disaster. Too much gets through as it is simply because not enough folks test the betas and report what they find. _All_ of that would get through if we just picked a random commit and declared it to be a release. _Maybe_ someday our test suites will catch such a large portion of the regressions that we won't actually end up with any regressions getting out, but I doubt it. Even large, heavily used projects like gcc or KDE end up with regressions getting out, much as they try to avoid it. But like them, we need to do our best to have releases which have been tested well enough via betas and whatnot rather than just releasing stuff simply because it's a certain date. - Jonathan M Davis
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 06:42:06 UTC, Rikki Cattermole wrote: And shouldn't the CI be doing regression testing already? Changelog perhaps should be updated with a commit then? In perfect world tests provide full coverage and language spec is clearly defined. In practice each single beta release cycle detects a dozen or more important regressions triggered by third-party projects. Requiring changelog update for each PR is something that should be done, yes.
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 06:08:41 UTC, Rikki Cattermole wrote: Of course we could always get the CI to auto build a full release every day available for download. It would then be just a matter of picking one set and promoting as a version. Perfect for e.g. testing. Picking random commit and calling it a version is not really releasing. For me release implies strict regression testing and nice changelog at the very least.
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 06:00:46 UTC, Dicebot wrote: Of many things that Rust got right, this is, in my opinion, something they did wrong. Quick release cycles only make sense in both bleeding edge model and with availability of excess developer resources. No amount of planning and management can compensate for things not being ready. I think that it's probably fine to target a six week release cycle, but I also don't think that we should be slaves to it. I think that it's far more important to do our best to avoid regressions than to get releases out the door at given intervals. But regardless, we do need folks championing the release process, or it's not going to work very well. Martin seems to have been doing a good job of it, but all it would take is him being unavailable for a few weeks for releases to falter. And I'm not sure that we really have enough people to afford to have an actual team of folks focusing on releases rather than a single individual who's making sure that it gets done. - Jonathan M Davis
Re: 6-weeks release cycle
On 5/06/2015 8:28 p.m., Jonathan M Davis wrote: On Friday, 5 June 2015 at 06:42:06 UTC, Rikki Cattermole wrote: And shouldn't the CI be doing regression testing already? The autotester runs the unit tests that are in druntime, Phobos, and dmd. It catches a lot of stuff and generally prevents us from merging bad code. But it can't possibly catch everything. It catches the stuff that we thought to specifically test for and bugs that were fixed (since unit tests are usually added for bug fixes). But it's not that infrequent for something _new_ to break that's never broken before and is subtle enough that it doesn't get past all of the tests - especially when you're dealing with changes to the compiler. To catch all of that stuff before it goes out the door in a release, we need to test a _much_ larger code base than just the standard stuff - which is part of why we have betas. We want folks to try out their projects with the betas so that we can catch the stuff that we missed before it gets released in an official release. Simply grabbing an arbitrary commit and declaring it a release just because it's at about the time that we want to do a release would be a disaster. Too much gets through as it is simply because not enough folks test the betas and report what they find. _All_ of that would get through if we just picked a random commit and declared it to be a release. _Maybe_ someday our test suites will catch such a large portion of the regressions that we won't actually end up with any regressions getting out, but I doubt it. Even large, heavily used projects like gcc or KDE end up with regressions getting out, much as they try to avoid it. But like them, we need to do our best to have releases which have been tested well enough via betas and whatnot rather than just releasing stuff simply because it's a certain date. - Jonathan M Davis I wonder if we could integrate a bit more with travis and have a big list of projects compiling + testing against HEAD. Would be an excellent indicator. Maybe even have this as part of dub repo?
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 10:00:17 UTC, Rikki Cattermole wrote: I wonder if we could integrate a bit more with travis and have a big list of projects compiling + testing against HEAD. Would be an excellent indicator. Maybe even have this as part of dub repo? I do it for small set of projects using own jenkins box - it is very easy to setup actually. By the way dub master has been broken with dmd master for ages : https://jenkins.dicebot.lv
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 10:00:17 UTC, Rikki Cattermole wrote: On 5/06/2015 8:28 p.m., Jonathan M Davis wrote: On Friday, 5 June 2015 at 06:42:06 UTC, Rikki Cattermole wrote: And shouldn't the CI be doing regression testing already? The autotester runs the unit tests that are in druntime, Phobos, and dmd. It catches a lot of stuff and generally prevents us from merging bad code. But it can't possibly catch everything. It catches the stuff that we thought to specifically test for and bugs that were fixed (since unit tests are usually added for bug fixes). But it's not that infrequent for something _new_ to break that's never broken before and is subtle enough that it doesn't get past all of the tests - especially when you're dealing with changes to the compiler. To catch all of that stuff before it goes out the door in a release, we need to test a _much_ larger code base than just the standard stuff - which is part of why we have betas. We want folks to try out their projects with the betas so that we can catch the stuff that we missed before it gets released in an official release. Simply grabbing an arbitrary commit and declaring it a release just because it's at about the time that we want to do a release would be a disaster. Too much gets through as it is simply because not enough folks test the betas and report what they find. _All_ of that would get through if we just picked a random commit and declared it to be a release. _Maybe_ someday our test suites will catch such a large portion of the regressions that we won't actually end up with any regressions getting out, but I doubt it. Even large, heavily used projects like gcc or KDE end up with regressions getting out, much as they try to avoid it. But like them, we need to do our best to have releases which have been tested well enough via betas and whatnot rather than just releasing stuff simply because it's a certain date. - Jonathan M Davis I wonder if we could integrate a bit more with travis and have a big list of projects compiling + testing against HEAD. Would be an excellent indicator. Maybe even have this as part of dub repo? I remember there being talk about testing HEAD against vibe.d, I can't remember what happened(if anything.)
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 11:09:27 UTC, Dicebot wrote: By the way dub master has been broken with dmd master for ages : https://jenkins.dicebot.lv It usually is, which usually means that I'm forced to install a release version of the compiler just to build dub, since I'm almost always using dmd master. I wish that it were kept up-to-date with dmd master, but I guess that Sonke has enough to do that he doesn't want to put in that effort. - Jonathan M Davis
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 12:20:06 UTC, Jonathan M Davis wrote: On Friday, 5 June 2015 at 11:09:27 UTC, Dicebot wrote: By the way dub master has been broken with dmd master for ages : https://jenkins.dicebot.lv It usually is, which usually means that I'm forced to install a release version of the compiler just to build dub, since I'm almost always using dmd master. I wish that it were kept up-to-date with dmd master, but I guess that Sonke has enough to do that he doesn't want to put in that effort. - Jonathan M Davis It is a wrong way to put it. dub is part of D-Programming-Language organization now. It must be tested by auto-tester and no pull request that breaks dub should have ever been merged.
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 14:11:06 UTC, Steven Schveighoffer wrote: On 6/5/15 1:53 AM, Andrei Alexandrescu wrote: Rust will do it: http://opensource.com/business/15/6/rust-6-week-release-cycle Would be so nice if we had the resources to do that. There have been a couple of initiatives in the past, but the folks who wanted to do the release just got busy with other things. I am fine with putting a time limit on adding features, but I'd rather release based on minimized regressions. We have to get better as a group with insisting to use release/stable branches. Do we have a plan written somewhere of how we want to do branching? I know we did some this last release cycle, but I don't remember it being formally written. -Steve I agree that releases could sometimes be more frequent. However, to release for the sake of releasing is not a good idea, in my opinion (that is based on bad experience). Maybe Rust will do it, because they still have so much stuff to fix, dunno. D is more mature and fixes/improvements have to be pondered on and tested thoroughly. High quality releases you can rely on are more important than frequent releases. I use Manjaro Linux (ArchLinux based) at home and I appreciate their efforts to test things thoroughly before passing possibly buggy things on to users. If certain bug fixes are a priority for a given project, I'd recommend offering nightly (or weekly) builds with a well-documented change log.
Re: 6-weeks release cycle
On 6/5/15 1:53 AM, Andrei Alexandrescu wrote: Rust will do it: http://opensource.com/business/15/6/rust-6-week-release-cycle Would be so nice if we had the resources to do that. There have been a couple of initiatives in the past, but the folks who wanted to do the release just got busy with other things. I am fine with putting a time limit on adding features, but I'd rather release based on minimized regressions. We have to get better as a group with insisting to use release/stable branches. Do we have a plan written somewhere of how we want to do branching? I know we did some this last release cycle, but I don't remember it being formally written. -Steve
Re: 6-weeks release cycle
On 2015-06-05 08:10, Dicebot wrote: Picking random commit and calling it a version is not really releasing. For me release implies strict regression testing and nice changelog at the very least. Very good point. -- /Jacob Carlborg
Re: 6-weeks release cycle
On 6/5/15 2:03 PM, Dicebot wrote: Martin was going for http://wiki.dlang.org/DIP75 OK, good, this needs an executive decision. We need a process to be blessed first before we talk about release frequency. Then everything backfills from there. -Steve
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 14:11:06 UTC, Steven Schveighoffer wrote: On 6/5/15 1:53 AM, Andrei Alexandrescu wrote: Rust will do it: http://opensource.com/business/15/6/rust-6-week-release-cycle Would be so nice if we had the resources to do that. There have been a couple of initiatives in the past, but the folks who wanted to do the release just got busy with other things. I am fine with putting a time limit on adding features, but I'd rather release based on minimized regressions. We have to get better as a group with insisting to use release/stable branches. Do we have a plan written somewhere of how we want to do branching? I know we did some this last release cycle, but I don't remember it being formally written. -Steve Martin was going for http://wiki.dlang.org/DIP75
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 12:21:20 UTC, Dicebot wrote: On Friday, 5 June 2015 at 12:20:06 UTC, Jonathan M Davis wrote: On Friday, 5 June 2015 at 11:09:27 UTC, Dicebot wrote: By the way dub master has been broken with dmd master for ages : https://jenkins.dicebot.lv It usually is, which usually means that I'm forced to install a release version of the compiler just to build dub, since I'm almost always using dmd master. I wish that it were kept up-to-date with dmd master, but I guess that Sonke has enough to do that he doesn't want to put in that effort. - Jonathan M Davis It is a wrong way to put it. dub is part of D-Programming-Language organization now. It must be tested by auto-tester and no pull request that breaks dub should have ever been merged. I don't disagree, but that's the way that it's been historically, and nothing seems to have changed once it got put into the D-Programming-Language organization. Similarly, I don't think that any of the stuff in the tools repo is built as part of the autotester either (e.g. when the change to if(arr) was made recently, dustmite was broken until the change was reverted). So, our approach there is bad. - Jonathan M Davis
Re: 6-weeks release cycle
On 6/5/15 3:19 PM, Steven Schveighoffer wrote: On 6/5/15 2:03 PM, Dicebot wrote: Martin was going for http://wiki.dlang.org/DIP75 OK, good, this needs an executive decision. We need a process to be blessed first before we talk about release frequency. Then everything backfills from there. I re-read this. I think I'd prefer something like this: 1. A bugfix *or* feature release is done every month. 2. A feature release should be planned for every 2 months, but can extend up to 3 months if necessary. Exceptions are allowed to this rule, but only with extenuating circumstances. Just because sometimes features are not really added or ready. I hate to be a slave to the schedule, but we should commit to releasing *something* every month. -Steve
Re: 6-weeks release cycle
On Friday, 5 June 2015 at 19:19:33 UTC, Steven Schveighoffer wrote: On 6/5/15 2:03 PM, Dicebot wrote: Martin was going for http://wiki.dlang.org/DIP75 OK, good, this needs an executive decision. We need a process to be blessed first before we talk about release frequency. Then everything backfills from there. -Steve The important bit is to have someone stand up and build the necessary testing infrastructure. I am to blame for not having done much on this front yet, due to heavy work load. Time base release schedules work fine, no matter how tight, but you require good infrastructure to find regressions and revert if necessary. If people accept that deltas can be very small and that reverts will happen to remove an unstable feature, than a 6-week time based release schedule is fine. In fact I believe there is a lot to gain from it.
Re: 6-weeks release cycle
On 6/5/15 12:28 PM, David Soria Parra wrote: On Friday, 5 June 2015 at 19:19:33 UTC, Steven Schveighoffer wrote: On 6/5/15 2:03 PM, Dicebot wrote: Martin was going for http://wiki.dlang.org/DIP75 OK, good, this needs an executive decision. We need a process to be blessed first before we talk about release frequency. Then everything backfills from there. -Steve The important bit is to have someone stand up and build the necessary testing infrastructure. I am to blame for not having done much on this front yet, due to heavy work load. Time base release schedules work fine, no matter how tight, but you require good infrastructure to find regressions and revert if necessary. If people accept that deltas can be very small and that reverts will happen to remove an unstable feature, than a 6-week time based release schedule is fine. In fact I believe there is a lot to gain from it. I agree. To put this in context: OK, good, this needs an executive decision. We need a process to be blessed first before we talk about release frequency. Then everything backfills from there. We need a champion to do this and stay with it, not an executive decision. _Then_ everything backfills from there. Andrei
Re: 6-weeks release cycle
On 6/5/15 4:31 PM, Andrei Alexandrescu wrote: On 6/5/15 12:28 PM, David Soria Parra wrote: On Friday, 5 June 2015 at 19:19:33 UTC, Steven Schveighoffer wrote: OK, good, this needs an executive decision. We need a process to be blessed first before we talk about release frequency. Then everything backfills from there. We need a champion to do this and stay with it, not an executive decision. _Then_ everything backfills from there. I wasn't aware the DIP was approved, but I see now that it was. Sorry about that question! -Steve
