I want to write again to correct something I wrote in the previous email: "If we had done internal testing before forking 1.12, ..."
I'm not saying we didn't do any testing, shadowm points out on irc that he was actively testing his campaign at this time and didn't see big problems. But, I think it would be good if we had a more official process, or perhaps an "alpha" phase, because the following situation can be confusing. Suppose that we find that in 1.11 there is a new system in the engine to do something, that doesn't seem to work right / isn't very clearly documented and is now an orphan. In a "stabilization" beta branch, as I understand anyways, you normally are avoiding making big changes to the code, so is it appropriate to even consider reverting a bunch of commits, or a large commit? My guess would be not. Nevertheless, I think there needs to be a point where we make judgment calls like that, and during beta doesn't seem to be the right time. On the other hand, we won't ever get lots of testing unless we have users beta-testing things, so maybe during beta is the only opportunity to find and then do something like that anyways? Maybe the real answer is that we need to extend our automatic tests to also test some of these save-loading problems, as they tend to be pretty complicated and tedious to test. Also, let me add: I also don't mean to pick on the author of the commit I pointed out, I just wanted to make an example of a commit that seems, after we bisect, to cause a problem, but where it is also hard to figure out what is going on and the author seems to be gone. There are other examples of commits like this that I could have pointed out from my experience. Also, let me add to this: "Maybe it's better if there aren't specific people whose job it is, but we make a policy not to fork stable or tag a release until 3 people on irc have done this..." We could also ask on the #wesnoth channel, my guess is that there may be a few people who idle there would be willing to briefly test a specific revision if we asked them. Best Regards, Chris Beck On Sat, May 31, 2014 at 2:23 PM, chris beck <beck...@gmail.com> wrote: > > I think the real problem is not people splitting time between master / > 1.12 inappropriately, or favoring feature development over bugfix. Actually > gfgtdf and I have been working almost exclusively on bugfix for months. > > The problem is that when we forked 1.12 off of master, master was in a > very bad state and was not stable at all. In fact there were major > refactors of the code, especially the code that saves and loads games, like > this one. > > > https://github.com/wesnoth/wesnoth/commit/8f6a3a5490245c46268f8f821292da33879c2002 > > There are many strange things about this commit. For one it is enormous > and the commit message is very uninformative, and there are no code > comments. Many strange things happen after this commit. For example, the > "game_state" object, which has the purpose of representing internal engine > data, requires a pointer to the GUI object when it wants to write its > config during saving. This doesn't really make any sense... Moreover, after > this commit, we became completely unable to load Start of Scenario saves in > multiplayer -- they crash immediately with a wml error "invalid scenario > id". (We are hoping we will be able to fix this soon however, but it isn't > the point.) > > Unfortunately the person who made the commit has been apparently on > wesbreak for as long as I have been active, so the rest of us are left > without any documentation, trying to decipher what happened and figure out > what to do. > > If we had done internal testing before forking 1.12, we could have > realized that things became broken and that the authors were not around to > take responsibility, and then considered reverting commits or taking other > steps. And this is not isolated -- really, go checkout the revision where > we branched 1.12, compile it, and play around with it for a few minutes if > you want to see what the state of things was. > > Forking at a very unstable time also hinders the rest of the project > generally -- the stabilization changes generally need to go on both master > and 1.12, but if people actually push significant features to master, the > change names or argument types of functions, then we won't be able to > cherry-pick the changes between the two branches without getting constant > conflicts. So I'm finding that even if I wanted to push changes to master > right now that won't go to 1.12, I'm very hesitant to do so because of the > additional work it will create in trying to stabilize 1.12. > > > And yes, I agree that more testing for each release would of course be > great. > > I just know that I am not the one having the time to actually do so. IMO > this > > is something which should be spread among all devs by focusing on > stabilizing > > branches/1.12 (aka finding and fixing bugs). > > I agree, its the responsibility of all of us. > > From what I understood from irc, the actual reason that we didn't release > 1.11.14 is that after mattsc built the OSX package, he learned of the > clicking bug because he has a friend who helps him by briefly playtesting > the OSX build for him to catch any problems. His friend found the problems, > then he reported them upstream to us. (I think this is approximately > correct? Sorry if some details are not quite accurate.) > > Maybe we should have a similar system for the whole project, like say 3 > people on the release team whose job it is to boot up the game and just > click around on random things for 10 minutes, then write 2-3 sentences of > what they did and whether they saw any problems. > > Maybe it's better if there aren't specific people whose job it is, but we > make a policy not to fork stable or tag a release until 3 people on irc > have done this. I'm not really sure about the specifics. > > Surely we can collectively spare 30 minutes before events like these -- if > there's really a point in time where we can't release because we can't find > 3 people, then maybe it just means we are too busy to release at that time. > > Best Regards, > Chris Beck > > On Sat, May 31, 2014 at 11:21 AM, Nils Kneuper <crazy-ivano...@gmx.net> > wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Am 30.05.2014 12:29, schrieb Charles Dang: >> > I agree with Chris. The current state of our 1.12 beta releases has not >> > been what a user would expect from a beta release. Serious game breaking >> > bugs shouldn't be present. IMO we should devote more time to testing >> > *before* releases, not after. Our users should not be the ones who >> discover >> > game-breaking bugs. >> >> And this is the clear sign for me that feature development in parallel to >> stabilization does not work at all. People focus by far more on getting >> new >> features for dev done, sometimes backporting syntax changes without >> spending >> the time on branches/1.12 to ensure that it is as stable as it should be. >> >> I can understand that getting new things done is a lot more interesting >> than >> just working on finding and fixing bugs. But if this is how things work we >> should probably not do an approach like this the next time we want to >> work on >> a stable series. >> >> And yes, I agree that more testing for each release would of course be >> great. >> I just know that I am not the one having the time to actually do so. IMO >> this >> is something which should be spread among all devs by focusing on >> stabilizing >> branches/1.12 (aka finding and fixing bugs). >> >> Cheers, >> Nils Kneuper aka Ivanovic >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.0.22 (GNU/Linux) >> >> iEYEARECAAYFAlOJ82oACgkQfFda9thizwWA6gCdGiHLVWcO7uqbzvjEb5Iu1fug >> j3sAoJT/GuW2b45x/IOpAOXIcpDPvZoi >> =2+Tj >> -----END PGP SIGNATURE----- >> >> _______________________________________________ >> Wesnoth-dev mailing list >> Wesnoth-dev@gna.org >> https://mail.gna.org/listinfo/wesnoth-dev >> > >
_______________________________________________ Wesnoth-dev mailing list Wesnoth-dev@gna.org https://mail.gna.org/listinfo/wesnoth-dev
