Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Hi all, Pedro wrote (10-01-12 18:57) Michael Meeks-2 wrote OTH more releases means more features but also more bugs. And because new bugs occur, old bugs are left behind. Oh ! so - this is an argument for doing a build every decade ;-) Not really. It's just an argument that if releases are too close, developers will only have time to fix blockers :) So not so critical bugs tend to accumulate. This could mean that it will loose quality as it goes along if it there are no major obstacles ;) This has been part of our discussion in Paris. http://wiki.documentfoundation.org/QA/Improving_QA-Release-3.5 more specific http://wiki.documentfoundation.org/File:Liboconf2011_improving_qa_release_cycle-cornouws_Slides19-28.pdf See #4 on slide 23, and slide 27 explaining that point. That discussion (of course) carried no final conclusion - but the 'agreement' (consent with Petr that it is a good idea) that the developers try to remember/summarise their experience with this, so that that can be part of an evaluation. The idea/hope is, that faster/smarter fixing of bugs, leads to a shift in the time spending: less weeks on bug fixing and more on features. However, there is inevitably a strong tangling with the QA work. And - despite my optimistic nature and the many hours spent on both QA and getting that process at a higher level/notice - I have some serious concerns on how secure our over all process is in this regard. This results in bugs that should have been handled/getting floated much earlier. Fix may be easy / ready in master / needed, but do not find their way to the bug fix releases. Examples: - 45068 Update from 3.4.4 on Win not possible without ... initial bug 2011-04-29 3.4.0 beta3, workaround published 2012-01-22 - 41054 Saving problem ods with new sheets initial report 2011-09-20 fixed for 3.5 2012-01-10 fixed for 3.4 still pending - 39118 Charts do not update initial report 2011-07-10 fixed for master/3.5 2011-12-13 fixed for 3.4.6 2012-01-16 (pls don't ask me to provide more examples) Because I know some people are quite sensitive for the impression of being accused personally: this is not the case. I do not accuse someone. I just show where our process, our mutual activity, falls short. Maybe our process at the moment is even better then 6 months ago (good change). Still, it's not good enough. This is caused by the amount if bugs filed and the lack of time to handle those properly. So important issues do not always get the attention they deserve. It is not that bugs are not important enough, so that people ignore spending enough time on them. Another reason: lack of triage / bundling of issues, which is both important for developers (fixing) ans users (simple how-to's for work around). (Will do some kick off on that point later). HTH, -- - Cor - http://nl.libreoffice.org ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Cor Nouws wrote (12-01-12 23:22) Cor Nouws wrote (09-01-12 23:10) Find mentors officially and don't rely on people to show up for the second one. It did work out on the first session, but esp. for the follow-up event it should be sure that mentors are around. I agree with that. Are there people that can help with this? I've been invited - I realised lately - for a meeting on January 22 in the afternoon. So that makes my presence for both day's rather weak. No problem for me - though I would have liked better if I could join the full two days. In any case, as it looks now, we are rather unsure about people available for mentoring. Nevertheless I suggest just to continue with the plan. Cheers, -- - Cor - http://nl.libreoffice.org ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Hi, please keep in mind that I'm by no means a QA expert, but sometimes I'm good in expressing thoughts and fears of ordinary people ;-) On Tuesday, January 10, 2012 09:58:48 AM Michael Meeks wrote: On Mon, 2012-01-09 at 16:20 +0100, Nino Novak wrote: ... So - I'd love to understand this desire for less frequent releases better :-) After all, we have tinderboxes churning out at least daily releases (in theory), perhaps several a day if we are lucky. I think, people simply need enough time as daily spare time window might be small: imagine about 2-3x weekly 1-2 hours, but often there's much less. So in good times they can install one release per week and test it for one or two 1-2 hours periods in the same week. That's it. As for the frequency: I for my part prefer to have a most-recent build for testing, so no - the release frequency should IMHO *not* decrease. But somehow I'd also like to have the feeling of having enough time to test in depth. Here a clearer prioriritization might be helpful. I don't know if it's important, but I just wanted to mention that I very rarely take the time to test a release according to a fixed testing plan (Litmus etc) but most often just try to do my usual office work on copies of my original documents in a sandbox (and if nothing suspicious happens, after 3-4 weeks those sandbox document copies become masters again and replace the original documents). And my impression is, that many people do this en passant testing and thereby discover problems or bugs. What is the concern about having new RC's ? is it that you think developers will not care about and/or test any bugs that appear in something one release-candidate old ? [ that seems unlikely if it is a serious bug ], or ? ... For /serious/ bugs, well, ok, but what if they are not-so-serious? Where's the threshold? And, to raise a different issue: People might well feel overwhelmed by the release frequency. Lost in release fusillades, so to speak. I personally have decided to concentrate on testing the most recent code line whenever possible. But many people still do not understand the release plan, and in addition do not know, how they can be (or make) sure that their test install will not interfere with their productive version. The QA-FAQ does not address this issue, you have to search for infos in the wiki... So in summary, it may be a little bit the Mohammed - mountain problem. Cor's activities are a good starting point and most appreciated :-) In the end, we have the common goal to make the software working as smoothly as possible. Fourth, which is more an open question, how the success of Release QA could be monitored intelligently. My (naive) wish would be to have usage numbers, let's say - how often a Release has been launched on which OS platform without failure We have some download statistics of those that can be extracted (I suspect), and we have the on-line update statistics too which may give some yard-stick for successful launch ;-) usually the app has to stay alive for a little while to do that request. (I'd appreciate if something like that could be implemented, but the effort should be kept low) - how often which module has been started - how many documents have been created/edited/viewed successfully - which particular functions have been called how often successfully These other phone-home things are more tricky, needing coding support, but it's of course a good idea to ensure good code coverage. Ideally - I'd like to reduce the burden on human QA though, so we're investing and encouraging (where we can) fast automated test that run during the compile: so you should never get a build that has pathaological failures [ assuming our test are complete enough ;-]. Hopefully that makes the process of QA more difficult rewarding ;-) but of course there is always room for lots of improvement, and some things are hard to test. All the above written does not relate to machine tests, only to manual tests. We should keep these two different approches well separated in discussion as they have different needs each one. For automated tests, you need skilled people. Manual testing can be done by Joe Average, at least in theory. One thing that is really nasty to test is the new header/footer/page-break stuff. I get intermittent leakage of page-breaks in documents (with several rendered on the screen); -but- while (after editing a document) I can reproduce them nicely, if I save re-load in another instance - I cannot ;-) so - there is a real need for some from a clean document reproduction steps for those issues - some of which may be races too ;-) help there much appreciated. (others have to step in here, as I didn't test header/footer much yet - except that I wondered that deleting header/footer cannot be undone :-( ) Nino ___ List Name:
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Hi Michael, all Michael Meeks-2 wrote Actually, there is an updated patch in bugzilla's bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=294608 Unclear how to help focus minds there though :-) please do watch that bug (requires a login ;-) and report back as/when it's merged. Aaarghhh! Another Login! See what I mean? This is insane! :) I will keep an eye on it. But given the negative feedback the patch submitter got and that it has been neglected since November 17th by both parts, I would say it's another dead duck :( Regards, Pedro -- View this message in context: http://nabble.documentfoundation.org/Libreoffice-qa-3-5-0-QA-from-BHS-1-to-BHS-2-tp3643039p3650302.html Sent from the QA mailing list archive at Nabble.com. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Hi Kohei The truth is that different people have different pet peeve bugs they want backported to 3.4.x, and we can't respond to all of them because it's extra work. Backporting a change is not free, someone has to review the change and make sure that change won't introduce regressions. And that's not as easy as you may think, since a lot of things are different between 3.4 and 3.5, and 3.4 being marked stable, there is additional effort required to ensure no regressions. I'm aware of the work involved in backporting fixes even if I'm not a developer ;) As for the bug you mentioned, you just need to prod someone to review, sign off, and backport that change. I can't do it since I'm the one you made the change; it needs to be reviewed by another developer. I was quoting that particular problem as an example. Maybe someone less unpopular than myself can do that :) To be honest I'm puzzled that a program which reportedly is used by 25 *million* people worldwide has half a dozen people in QA... I guess this shows a lot about human nature :( Could you clarify on this? I'm not sure how to interpret this. I meant that there are (reportedly) so many people downloading and using LO that it is absurd that so few are willing to give something back... And we aren't even talking about money... just a few minutes of their time... ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
On Tue, 2012-01-10 at 16:22 +, Pedro Lino wrote: To be honest I'm puzzled that a program which reportedly is used by 25 *million* people worldwide has half a dozen people in QA... I guess this shows a lot about human nature :( Could you clarify on this? I'm not sure how to interpret this. I meant that there are (reportedly) so many people downloading and using LO that it is absurd that so few are willing to give something back... And we aren't even talking about money... just a few minutes of their time... Gotcha. Yes, I agree; it's a shame indeed. Although in my observation those same users aren't too shy about reporting problems in forums and other communication mediums. Perhaps there is something about bugzilla and its environment that hold them back. Kohei -- Kohei Yoshida, LibreOffice hacker, Calc ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Hi Pedro, On Tue, 2012-01-10 at 08:48 -0800, Pedro wrote: But the main obstacle IMO is the requirement to have to subscribe to yet another account. I have suggested elsewhere that OpenID should be adopted as the default identification method. I agree. Most people already have an OpenID (e.g. a Gmail account) so that obstacle would be reduced It'd be great; there was even a patch created for this some time ago, that has sadly bit-rotted: https://wiki.mozilla.org/Bugzilla:OpenID_Auth_Plugin Should email verification process still occur? * There doesn't appear to be any way around it, as there's no way to query an OpenID server for an email address. Doesn't fill me with deep-joy I must say :-) Actually, there is an updated patch in bugzilla's bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=294608 Unclear how to help focus minds there though :-) please do watch that bug (requires a login ;-) and report back as/when it's merged. ATB, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
On Tue, 2012-01-10 at 07:12 -0800, Pedro wrote: I know this wasn't addressed to me, but here are my thoughts... I always like to hear your thoughts :-) First of all RCs: RC releases replace the tester's stable release. I know it can't be otherwise. Ok - so this might be a good argument for keeping parallel-installability until later, perhaps for RC1 itself ? I'd really prefer to have two releases to test the real release code though :-) So my opinion is that there should be more Betas than RCs even if the total testing period is the same. Fair comment; though it's perhaps better to call it an RC - since we start beefing up our code review / checkin criteria then - which (hopefully) helps make the final result more predictable. OTH more releases means more features but also more bugs. And because new bugs occur, old bugs are left behind. Oh ! so - this is an argument for doing a build every decade ;-) IMHO the six monthly cadence seems to work reasonably well, and it fits the Linux distributions too ... Here is an example of what I'm talking about (and the reason why I insisted on giving more weight to 3.4.5 than to 3.5.0...) http://nabble.documentfoundation.org/Blood-pressure-chart-doesn-t-work-tt3646489.html#a3647580 Sure - but given a choice between getting a fix for this in a stable release next month - and getting a fix in another six months time (a yearly release schedule) - which would you choose ? ;-) It's not clear to me that releasing less frequently creates more resources for back-porting and reviewing patches. Anyhow - thanks for the feedback also for the great work on QA :-) All the best, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] 3.5.0 QA ... from BHS 1 to BHS 2
Hi Michael, all Michael Meeks-2 wrote Ok - so this might be a good argument for keeping parallel-installability until later, perhaps for RC1 itself ? I'd really prefer to have two releases to test the real release code though :-) This is a Catch 22... If it is installed in parallel, then it can't be named a Release Candidate because it doesn't behave like one :) I agree totally on two releases for the real release code. Michael Meeks-2 wrote OTH more releases means more features but also more bugs. And because new bugs occur, old bugs are left behind. Oh ! so - this is an argument for doing a build every decade ;-) Not really. It's just an argument that if releases are too close, developers will only have time to fix blockers :) So not so critical bugs tend to accumulate. This could mean that it will loose quality as it goes along if it there are no major obstacles ;) Michael Meeks-2 wrote It's not clear to me that releasing less frequently creates more resources for back-porting and reviewing patches. See previous answer :) It doesn't create more resources but it provides more time to clean up the slate until the next batch of bugs is introduced :) Michael Meeks-2 wrote Anyhow - thanks for the feedback also for the great work on QA :-) This is my way to say thanks and to give something back in return for the program that was offered to me. I wish more people would do the same... even if they can't code ;) Best regards, Pedro -- View this message in context: http://nabble.documentfoundation.org/Libreoffice-qa-3-5-0-QA-from-BHS-1-to-BHS-2-tp3643039p3648320.html Sent from the QA mailing list archive at Nabble.com. ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
