+1 to Ian's thoughts. Kind Regards, Chris
2011/1/27 Ian Walls <[email protected]> > Paul, > > > I hear you. There are lots of really great features that are in limbo, > waiting to be tested. The unfortunate situation of life is that we're all > incredibly busy, and it's hard to find time to test things. This is > compounded when testing certain features requires coding or systems > experience, since that limits the pool of potential testers. As a result, > lots of good stuff sits, waits, and diverges. > > So what can be done? I don't think that changing the signoff procedure > will have the desired effect. That'll just let more unreviewed code slip > in, potentially introducing bugs we won't find for weeks/months/years. > > I think a better solution is make it easier to test and signoff on work. > There are several components to this: > > - Testing plans for larger developments/bugfixes > - A robust testing data set made readily available > - Teaching people how to test and signoff on code > > By including testing plans with developments or complex bugfixes, the > developer is communicating to everyone how they can prove their code works. > It lays out the intention of the development (it should do x, y and z), and > a series of tests to show how to get x, y and z without losing a, b and c. > > Combine this testing plan (written in language librarians can understand, > not coder jargon) with the necessary data set to do the testing (an SQL dump > you just load into a DB), and you've lowered the barrier for testing so that > anyone who can afford a little time to run through a series of listed > procedures can answer the question "does this work?". > > The third step to this is to lay out the procedure for running through the > test plan in a clear, simple manner, and distribute that information far and > wide. Make it something that librarians can do by following a list of > steps. Lowering the threshold of experience required to test things will > allow us to harness the Long Tail. > > To this end, I'm throwing my hat in the ring for Quality Assurance Manager > for Koha 3.6. My proposal can be found on the wiki ( > http://wiki.koha-community.org/wiki/QA_Manager_for_3.6_Proposal), and much > of it is explained above. In addition to this, I would also serve as a > coordinator for testing work submitted, and provide regular reports to the > community on the status of these developments. Branches that are not > receiving active testing feedback would receive attention towards creating a > simpler, easier to follow testing plan. > > I would very much like to discuss this at the next IRC meeting. It'll be > pretty early for me (5am, I believe), but I'll caffeinate heavily beforehand > :) > > Cheers, > > > -Ian > > > > > On Thu, Jan 27, 2011 at 11:03 AM, Paul Poulain > <[email protected]>wrote: > >> Hello koha-devel, >> >> The more time passes, the more I think we have a workflow for Koha >> development that does not work correctly/efficiently/fluently. >> The rule we have is: >> * create a branch for the feature >> * file a bug for it >> * submit your branch to koha-patches ML >> * the RM checks that everything apply smoothly on master & ask for QA >> * someone signs-off the branch/patch >> * the RM merge the branch. >> >> For the maintenance branch, this workflow is perfect and must be secured >> more and more to avoid any regression. As I see it, there is always >> someone that can find time to sign-off a patch or 2. As ppl have >> system live with it i'm sure it's cricital for many ppl to check & >> double check ! >> >> But for the next release/unstable branch this workflow does not work well. >> >> Here is a single line of something owen said on the chat today: >> <owen> paul_p: I would much prefer to be testing your stuff today, but >> other stuff is taking precedence. >> >> God knows i'll be forever thankfull to owen for the work he does. But this >> sentence reveals something: for most of us, there is always "other stuff >> taking precedence" :\ >> >> BibLibre has submitted many new features 3 months ago. I tried to >> motivate ppl to get feedback & sign-off. I even have organised an IRC >> meeting ... where no one came. Those patches includes >> new features, and requires a lot of work to be tested. >> 2 weeks ago, I did the same for a bunch of small improvements (and I >> have many many more pending). Once again, no feedback, except for one >> very easy-to-test (new images) : signed-off by Nicole & pushed by chris >> in less than 24H! >> Those 2 minor events (with owen & nicole) made me realize that it's >> technically and >> functionnaly hard & long to test: rebase, drop database, install, file >> test cases, check the new features,... At KohaCon, liz tested some of >> those features and said "wow, great, I want this feature for my library >> !" (I don't remember what it was exactly). The problem is that liz is >> not a techie woman, and can't test patches by her own. For our images, >> it was easy for Nicole to test, no need to updatedatabase or anything >> complex, just reach the itemtypes.pl page check the images are here. >> Easy for a librarian ! >> >> Other point: dependancies between new features. Let me take an example: >> Templates on those devs relies on html::template. Chris is working on >> moving everything to Template::Toolkit. >> Once he consider it's ready, he should submit for QA, as for every new >> features. >> Suppose no-one sign-off. >> And suppose someone sign-off, there are 2 options: >> * move master to T::T when it's signed-off. What happends with other >> branches >> not merged ? they can't be merged anymore, they have to be rewritten. >> wow, I can't imagine that something we've submitted for QA 3 months ago >> has to be rewritten because no-one took care of it ! >> * wait until those branches are merged. Two major problems here: the >> release date won't be respected, and we have switched again to feature >> based release. Or 3.4 will be released on time, but won't contain many >> things all those great features that are live in all our libraries since >> months. The second problem is that other devs are on the way. They still >> rely on H::T as master is based on that. Switching to T::T will cause >> problems for those branches later. >> well, I think that none of those solutions is a correct one. >> >> I can take an other example: the analytics record feature may/will >> probably overlap functionnaly with the new features we have submitted >> for cataloguing (not sure, but i think so). Suppose our cataloguing >> branch is integrated in 2 weeks. it has been submitted 3 months ago. It >> means our indian colleagues will have to rebase & face problems to get >> their analytical record branch working. If our branch had been merged >> quickly, they would have had 3 more months to deal with it. >> Our actual workflow, causes a lot of overhead ! And -worst- I don't >> think it improves the stability & security: there are merge problems >> that are difficults to detect. It means that any test done on analytical >> records would have to be done again if our cataloguing branch is merged >> : more work, more pain. And the later a branch is merged, the smaller >> the time to find a problem. >> >> My main conclusion is that we are not a large enough community to deal >> with testing/validating/merging new features in a short timeline. So I >> think we must change the way we integrate new features. The general idea >> being: remove the sign-off lock on the workflow. >> >> Here is a proposal: >> Let's go back to the timeline: we plan to release a version every 6 >> months. >> We could have a window of, say 4 months. During those 4 months a feature >> can be included if : >> 1- the submitter provides something that applies on master (ie: it's >> technically valid) >> 2- no-one object in 2 weeks (or 3, or 4, or defined by the RM when the >> branch is submitted) >> >> It put more pressure on others to test quickly, and don't put pain on >> parallels/parallelizing developments. Once a branch is merged if there >> is a problem, then everybody should see it, and it will be fixed ! >> >> Then, for 1 month, any new feature can be applied only if a newly >> submitted branch is approved/signed-off (less than 2 months to detect a >> problem in a new feature is a must-have) >> Then, feature freeze, 1 month debugging and translating. >> >> I know it is a big change in the workflow, but the actual one is not >> working well, so, we have to find another one. >> I humbly request to have this question being put on the next irc agenda >> (which does not mean you should not also start a thread here, of course) >> >> PS: pls don't say i'm wrong and the workflow works well. Having branches >> submitted 3 months ago, getting no feedback, and planning to have a release >> in less than 3 months, can't be considered as something working well. >> >> Friendly. >> >> -- >> Paul POULAIN >> http://www.biblibre.com >> Expert en Logiciels Libres pour l'info-doc >> Tel : (33) 4 91 81 35 08 >> >> _______________________________________________ >> Koha-devel mailing list >> [email protected] >> http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >> website : http://www.koha-community.org/ >> git : http://git.koha-community.org/ >> bugs : http://bugs.koha-community.org/ >> > > > > -- > Ian Walls > Lead Development Specialist > ByWater Solutions > Phone # (888) 900-8944 > http://bywatersolutions.com > [email protected] > Twitter: @sekjal > > _______________________________________________ > Koha-devel mailing list > [email protected] > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel > website : http://www.koha-community.org/ > git : http://git.koha-community.org/ > bugs : http://bugs.koha-community.org/ >
_______________________________________________ Koha-devel mailing list [email protected] http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel website : http://www.koha-community.org/ git : http://git.koha-community.org/ bugs : http://bugs.koha-community.org/
