Re: Cleaning bug list
On 06/21/2012 10:51 PM, Michael Stahl wrote: On 21/06/12 21:32, bfo wrote: Disagree. It is a page for bug reporters. I really like the gerrit migration and would like to see it integrated with Bugzilla. Also always precommit hooks can be implemented if developers tend to forget to put a bug number into a commit message. I already stumbled upon UNCONFIRMED bugs which have been fixed already and suddenly changed into RESOLVED FIXED. I would like to know what is going on with the bugs at any moment. such a precommit hook doesn't work so well. there are lots of commits that do cleanups, refactorings, fix build breakers, etc. none of which have or need a bug id. in the OOo project there was a policy that every commit include a bug id, and the result was that all of these build breaker fixes used the same notorious #i1# number, which doesn't help anybody. ...plus, it (together with the CWS process, but the bug ID requirement alone was often reason enough to just not bother to do it) prevented small drive-by fixes and enhancements from going into the code base. The much lower overhead (esp. for accepted committers) in LO is IMO one of the big benefits over the old OOo. i've sometimes found bugs that were already fixed on master, but in the vast majority of the cases the author of the commit was not aware of the bug, i.e. the bug was a almost-but-not-exactly duplicate of the bug the author mentioned, or the author had a bug in a non-fdo tracker, or it was just something the author found themselves or whatever. This happens to me regularly. Even though I try to keep an overview of bugzilla, I often only learn later that there actually was a bugzilla bug for something I just fixed because I had stumbled upon it independently. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Hi bfo, this are interesting questions. I put back QA mailing list into CC because there people there are interested. bfo píše v Út 19. 06. 2012 v 11:24 -0700: This is a very nice workflow, but I have some questions: - how you define Bug prevent users from making professional quality work? By professional quality work, I understand that everything works as expected. It means that all menu entries, buttons, and dialogs does something reasonable or something that is described in the documentation ;-) These bugs are about all functions that are used even by power-users. If they do not work at all in reasonable scenarios, they should have higher severity. If they basically works but they do not have ideal results, they should have lover severity. Interoperability issues are a big obstacle. If this package is supposed to read/write MSOffice files, then every new bug in this area should have high priority by default. Sure. On the other hand, it would be ugly if the application is able to perfectly import documents but you could not modify them because of poor or buggy functionality. We need to ballance it with functional bugs. I have reported two annoying: https://bugs.freedesktop.org/show_bug.cgi?id=47138 47138 and https://bugs.freedesktop.org/show_bug.cgi?id=49102 49102 . Are those prevent users from making professional quality work enough? Sure. They are perfect candidates. Home users can live with a workaround, but it is impossible to expect that xx(x) users will do it xx times a day. Everyday. That is not professional (even worse - it is very unprofessional for those users migrated from MSO, where it just worked). No it is not professional. On the other hand, we need to make difference between broken and non-optimal functionality and also with the level of annoyance. If would be ugly if most bugs ends here and the minor and trivial categories are empty ;-) I'd change the workflow a little bit by putting the obvious things at the top: - feature requests aka wishlist I do not have any strong opinion for this. I think that it is is good to be able to discuss features, so enhancement bugs in bugzilla might be usable. - add target milestone if a feature is planned already This must be done by developer, anyway :-) - bugs reported against not supported branches - exclude those at reporting level by disabling old versions in select fields; but what to do when they appear anyway - mark them as INVALID at triaging? ask the reporter to check in new version? Any comments? We need to be more careful here. The current rule is to do NOT modify the version picker if you reproduce the problem with newer version. It is because it is very valuable to know how the bug is old. It helps developers to locate it. So, maybe, we should do it the other way and set the field to the oldest version where it was found. - is it Most Frequently Reported - then just dupe it and now, if a bug is still there, triage it by the proposed workflow. What do you mean by Most Frequently Reported? Note that Joel's flowchart was about prioritizing. It is only one part of the bugtriage process. Yes, looking for duplicates should be done before. - how you define most and many - I reported those two bugs on behalf of ~xx people - is my bug better than other reports? Should this be controlled by number of dupes? At what levels? Maybe enabling voting feature of Bugzilla would help here... Good question and hard to answer. The number of dupes is a good helper. I would personally enable voting as well but some other people thing that it is not objective. Otherwise, I use a common sense. I try to guess how each function is hard to use and how many people would do such action. Most people do opening documents, do simple editation (elements from toolbar), saving, printing. Many people does more professional things, like using styles, adding footnotes, inserting table of content, do computation in Calc. Only experienced users use macros, work with databases, send mails using addressbook... Another good symptom is how is the function hidden in menu ;-) - regressions - this is a major problem and I think it deserves its own place in the workflow. It is not a big problem for home users to switch the version back and forth. But when we are talking in terms of bigger deployment, when you have xx computers (and growing) it becomes a real blocker sometimes. What good are new and fixed versions for, where there is a regression which disqualifies the software for daily use? That is a big problem. You are right, regression are bad and we need to fix them with high priority. On the other hand, you still need to compare them with other reported bugs and decide what is more annoying for the daily work. We could not blindly set the highest priority for a tiny functional problem just because it is a regression. As someone said, you could view almost any bug as regression, so we need to
Re: Cleaning bug list
Petr Mladek wrote I'd change the workflow a little bit by putting the obvious things at the top: - feature requests aka wishlist I do not have any strong opinion for this. I think that it is is good to be able to discuss features, so enhancement bugs in bugzilla might be usable. - add target milestone if a feature is planned already This must be done by developer, anyway :-) Well, to be clear, I see all this more like how to triage guide, that is why I started with enhancements. Target milestone can be set by QA member if feature is available. https://bugs.freedesktop.org/show_bug.cgi?id=33773#c11 for instance. - bugs reported against not supported branches - exclude those at reporting level by disabling old versions in select fields; but what to do when they appear anyway - mark them as INVALID at triaging? ask the reporter to check in new version? Any comments? We need to be more careful here. The current rule is to do NOT modify the version picker if you reproduce the problem with newer version. It is because it is very valuable to know how the bug is old. It helps developers to locate it. So, maybe, we should do it the other way and set the field to the oldest version where it was found. I was thinking about New bug reports, which should be reported for supported branches only (proposed in https://bugs.freedesktop.org/show_bug.cgi?id=51070). Unfortunately it depends on https://bugs.freedesktop.org/show_bug.cgi?id=50198. - is it Most Frequently Reported - then just dupe it What do you mean by Most Frequently Reported? Is it at https://bugs.freedesktop.org/duplicates.cgi for LibreOffice product. Then it can be handled very quickly, but I see that searching for dupes is already listed on http://wiki.documentfoundation.org/BugTriage I would personally enable voting as well but some other people thing that it is not objective. Unfortunately I discovered https://bugs.freedesktop.org/show_bug.cgi?id=39739. WONTFIXed by Mr Bielefeld who authored http://wiki.documentfoundation.org/Vote_for_Enhancement, where it begins with [...] Bugzilla bug tracking system does not support Votes for requests as it is available for OpenOffice.org. Here you find an experimental Voting (or better: statistical online survey), currently with only very few rules and only for Enhancement requests. I am perplexed. How manual wiki voting, adding special keywords to bugs can be better than automated, build-in, out of the box, customizable voting within Bugzilla? Strange. I know that voting is not a recipe to find best bugs to fix/features to implement, but I think that giving an ability to vote (even only for QA team members or LO developers) would help going forward. I do vote for features in other Bugzillas myself. You are right, regression are bad and we need to fix them with high priority. On the other hand, you still need to compare them with other reported bugs and decide what is more annoying for the daily work. We could not blindly set the highest priority for a tiny functional problem just because it is a regression. As someone said, you could view almost any bug as regression, so we need to prioritize them as well. I would keep the current approach. Add regression into Whiteboard and set one level higher priority that you would set for non-regression. I disagree. Many people just won't upgrade because of them. Fast query shows 180 opened bugs with regression keyword. Very worrying... It actually helps to prioritize bugs as well. If reporter is not willing to provide more information, the problem is probably not that important for him. Disagree, as providing proper bt is complicated. Also note that some crashers are reproducible only in very strange circumstances. Also some scenarios are very hard to prepare. We just need help from reporters in these cases. Sure, but IMHO only if bug is not reproducible. If someone change the priority a wrong way, I would ask them not to do it, explain that the level is not that important, and change it back. If they change it once again, I would leave it as is. Developers would filter it themselves :-) I found https://bugs.freedesktop.org/show_bug.cgi?id=49168. IMHO instead of flooding the Bugzilla, one can just disable it. I think that it is not worth spending resources on migrating bugs to a single bugzilla. IMHO, it is better to spend time on fixing other bugs, improving the functionality, testing, ... Already some bugs are migrated (from Symphony for instance). Some of them are fixed already. I just can't imagine proper bug management in a project, when you have to follow many bugtrackers. The ideal process is described at http://wiki.documentfoundation.org/BugReport_Details#Initial_state Of course, developers are just humans, sometimes quite overloaded and they do not follow the process. [...] developers to close bugs by asking about the state. I wonder, how often do you see
Re: Cleaning bug list
On 21/06/12 21:32, bfo wrote: Petr Mladek wrote I'd change the workflow a little bit by putting the obvious things at the top: - feature requests aka wishlist I do not have any strong opinion for this. I think that it is is good to be able to discuss features, so enhancement bugs in bugzilla might be usable. - add target milestone if a feature is planned already This must be done by developer, anyway :-) Well, to be clear, I see all this more like how to triage guide, that is why I started with enhancements. Target milestone can be set by QA member if feature is available. https://bugs.freedesktop.org/show_bug.cgi?id=33773#c11 for instance. currently the target is set only after the bug is fixed, and this is how it ought to work. there general pattern is to add one target:x.y.z value per branch where the fix is applied to the whiteboard. in case you want the target to be set before the bug is fixed, as some kind of planning mechanism, please be aware that this was actually done in the OOo project, and that this worked so badly (i.e. the main result was that every developer had some 100 bugs assigned to him that he moved from target 3.0 to target 3.1 to target 3.2 etc. without ever having time to fix them) that it was actually decided (shortly before the end) to modify the process to set the target only after a bug is fixed, just like it is in LO. You are right, regression are bad and we need to fix them with high priority. On the other hand, you still need to compare them with other reported bugs and decide what is more annoying for the daily work. We could not blindly set the highest priority for a tiny functional problem just because it is a regression. As someone said, you could view almost any bug as regression, so we need to prioritize them as well. I would keep the current approach. Add regression into Whiteboard and set one level higher priority that you would set for non-regression. I disagree. Many people just won't upgrade because of them. Fast query shows 180 opened bugs with regression keyword. Very worrying... agreed, regressions are generally more important than non-regressions, precisely because they prevent users from using the latest version. If someone change the priority a wrong way, I would ask them not to do it, explain that the level is not that important, and change it back. If they change it once again, I would leave it as is. Developers would filter it themselves :-) this developer has already learned to ignore bugzilla priorities :) I think that it is not worth spending resources on migrating bugs to a single bugzilla. IMHO, it is better to spend time on fixing other bugs, improving the functionality, testing, ... Already some bugs are migrated (from Symphony for instance). Some of them are fixed already. I just can't imagine proper bug management in a project, when you have to follow many bugtrackers. this is a general problem of our project due to the convoluted history. Disagree. It is a page for bug reporters. I really like the gerrit migration and would like to see it integrated with Bugzilla. Also always precommit hooks can be implemented if developers tend to forget to put a bug number into a commit message. I already stumbled upon UNCONFIRMED bugs which have been fixed already and suddenly changed into RESOLVED FIXED. I would like to know what is going on with the bugs at any moment. such a precommit hook doesn't work so well. there are lots of commits that do cleanups, refactorings, fix build breakers, etc. none of which have or need a bug id. in the OOo project there was a policy that every commit include a bug id, and the result was that all of these build breaker fixes used the same notorious #i1# number, which doesn't help anybody. i've sometimes found bugs that were already fixed on master, but in the vast majority of the cases the author of the commit was not aware of the bug, i.e. the bug was a almost-but-not-exactly duplicate of the bug the author mentioned, or the author had a bug in a non-fdo tracker, or it was just something the author found themselves or whatever. Well, we do not want to create bug report for each fix. It is too complex process with poor results. If a developer is talkative, she provides a good commit message. If she is not talkative, she would create useless bug report. We could motivate them by asking for details, saying thanks for info, ... Anyway, I think that it is not important now. We do not have enough people who could test all commits, so we do not need perfect commit messages. It is very unfortunate and doesn't help triaging bugs. Another resource to watch by QA member - the commits. I really like Bugzilla developers strict workflow. Every commit with bug number, patch reviewed by a specialist (not just +1 to get it as soon as possible), bug assigned and status changed upon commit. It can be done automatically. we already have quite a bit of that,
Re: Cleaning bug list
On 21.06.2012 22:51, Michael Stahl wrote: we already have quite a bit of that, comments are added to bugzilla when a commit mentions a bug, and with gerrit we'll soon handle reviews better. gerrit bugzilla integration still has to be implemented. feature set of such integration can be seen here: https://github.com/hobbs/jirret) Regards David ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Hi Joel, Joel Madero píše v Út 19. 06. 2012 v 13:36 -0700: I moved this to a new thread because the subject here didn't really accurately portray the direction of the conversation The mail includes many good questions and proposals that might move us forward. I'll try to answer it later this week. Just a hint. In the future, I suggest to do NOT solve too many problems in one mail. Too long mails are discouraging. They are hard to read after few replies. I write such mails from time to time as well and I am always told that they are hard to handle ;-) In addition, I suggest to use formatting using spaces and tabs. Some people use text-only mail clients (pine) and they do not see the bold text :-) Finally, we need to add libreoffice-qa mailing list into CC for replies to the other mail. It is affecting the QA job. but I wanted to say I have uploaded the latest flowchard: https://wiki.documentfoundation.org/images/0/06/Prioritizing_Bugs_Flowchart.jpg I wasn't sure how or if I needed a wiki page or if I should just link to the jpg in the Useful Links section of the Bug Triage wiki page. Any thoughts? If you could respond on the other thread (more accurate subject) that would be great, if not here is fine. Thanks all for the input, I think that this is at least a decent start. I would add it instead of http://wiki.documentfoundation.org/BugReport_Details#Severity and add extra step for this into the BugTriage process. I hope that Rainer is not against. Thanks for the nice flowchart and working on triaging the bugs. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Thanks for the advice. I thought I had included the qa list, my mistake. As for the length, I agree and I almost didn't include it but that was an email in response to mine so I felt a bit obligated to respond despite the length. I had another side question, the response to the thread was made here: http://nabble.documentfoundation.org/Cleaning-bug-list-td3988836i20.html#a3991106 I never got an email about the response, instead it was only part of the digest. Is this the norm because the response was made through nabble? I was just lucky that I read the digest and saw that there was a response, otherwise that person would have never heard back from me about all of his concerns. Thanks again for the feedback, I think this topic is about exhausted at least for now. I'll get something up on the site where you suggested as well as a little blurb about it unless Rainer suggests otherwise. Joel On Wed, Jun 20, 2012 at 2:17 AM, Petr Mladek pmla...@suse.cz wrote: Hi Joel, Joel Madero píše v Út 19. 06. 2012 v 13:36 -0700: I moved this to a new thread because the subject here didn't really accurately portray the direction of the conversation The mail includes many good questions and proposals that might move us forward. I'll try to answer it later this week. Just a hint. In the future, I suggest to do NOT solve too many problems in one mail. Too long mails are discouraging. They are hard to read after few replies. I write such mails from time to time as well and I am always told that they are hard to handle ;-) In addition, I suggest to use formatting using spaces and tabs. Some people use text-only mail clients (pine) and they do not see the bold text :-) Finally, we need to add libreoffice-qa mailing list into CC for replies to the other mail. It is affecting the QA job. but I wanted to say I have uploaded the latest flowchard: https://wiki.documentfoundation.org/images/0/06/Prioritizing_Bugs_Flowchart.jpg I wasn't sure how or if I needed a wiki page or if I should just link to the jpg in the Useful Links section of the Bug Triage wiki page. Any thoughts? If you could respond on the other thread (more accurate subject) that would be great, if not here is fine. Thanks all for the input, I think that this is at least a decent start. I would add it instead of http://wiki.documentfoundation.org/BugReport_Details#Severity and add extra step for this into the BugTriage process. I hope that Rainer is not against. Thanks for the nice flowchart and working on triaging the bugs. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Joel Madero píše v St 20. 06. 2012 v 08:08 -0700: Thanks for the advice. I thought I had included the qa list, my mistake. As for the length, I agree and I almost didn't include it but that was an email in response to mine so I felt a bit obligated to respond despite the length. If only I had more time to react early in this interesting discussion :-) I had another side question, the response to the thread was made here: http://nabble.documentfoundation.org/Cleaning-bug-list-td3988836i20.html#a3991106 I never got an email about the response, instead it was only part of the digest. Is this the norm because the response was made through nabble? I was just lucky that I read the digest and saw that there was a response, otherwise that person would have never heard back from me about all of his concerns. I see, bfo sent the response only to the mailing list. The problem is that different people has different style in handling mailing lists. I prefer to stay in CC because I monitor mailing lists less frequently. Hence, I keep also other people in CC. I know many other guys that do not want to get mails twice (from mailing list and from CC), so they remove themselves and all other people from CC. I am afraid that we can't do much about it. Both approaches have some advantages :-) Thanks again for the feedback, I think this topic is about exhausted at least for now. I'll get something up on the site where you suggested as well as a little blurb about it unless Rainer suggests otherwise. Yup, I need to do some other work as well. I have spent too much time with mails the previous two days ;-) Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Joel Madero-2 wrote Version 2, changed orientation and tried to take comments into account. Let me know what you all think. Hi. This is a very nice workflow, but I have some questions: - how you define Bug prevent users from making professional quality work? Interoperability issues are a big obstacle. If this package is supposed to read/write MSOffice files, then every new bug in this area should have high priority by default. I have reported two annoying: https://bugs.freedesktop.org/show_bug.cgi?id=47138 47138 and https://bugs.freedesktop.org/show_bug.cgi?id=49102 49102 . Are those prevent users from making professional quality work enough? Home users can live with a workaround, but it is impossible to expect that xx(x) users will do it xx times a day. Everyday. That is not professional (even worse - it is very unprofessional for those users migrated from MSO, where it just worked). I'd change the workflow a little bit by putting the obvious things at the top: - feature requests aka wishlist - add target milestone if a feature is planned already - bugs reported against not supported branches - exclude those at reporting level by disabling old versions in select fields; but what to do when they appear anyway - mark them as INVALID at triaging? ask the reporter to check in new version? Any comments? - is it Most Frequently Reported - then just dupe it and now, if a bug is still there, triage it by the proposed workflow. - how you define most and many - I reported those two bugs on behalf of ~xx people - is my bug better than other reports? Should this be controlled by number of dupes? At what levels? Maybe enabling voting feature of Bugzilla would help here... - regressions - this is a major problem and I think it deserves its own place in the workflow. It is not a big problem for home users to switch the version back and forth. But when we are talking in terms of bigger deployment, when you have xx computers (and growing) it becomes a real blocker sometimes. What good are new and fixed versions for, where there is a regression which disqualifies the software for daily use? That is a big problem. - backtraces - not all people can do proper backtrace, especially on Windows, so maybe while triaging crash bugs one could ask for one a QA member? BTW: Having own dev build of LO 3.5.4.2 I can be QA contact for missing bt for all Windows crash bugs (except Base) on that branch. - permissions - seems like every user has canconfirm (Can confirm a bug) and editbugs (Can edit all aspects of any bug). Did you think about changing that, so one could not interfere with QA actions or vandalize the bug reports? Bugs triaged. But those are only b.f.o. bugs. Checking release info for 3.5.5.1 I can see such acronyms like bnc, i, rhbz and there are bugs reported on b.f.o. coming from aoo (where they may be fixed already). Not mentioning ubuntu, gentoo, opensuse and other bugtrackers. This is a very serious problem, because not all bugs are in one place. Any comments about that? Does the developers have their workflow in Bugzilla? I compared few commits (libreoffice/core libreoffice-3-5-5) with b.f.o reports: - 50603 - not assigned, UNCONFIRMEDRESOLVED FIXED - 43967 - assigned, UNCONFIRMEDNEWRESOLVED FIXEDREOPENEDASSIGNEDRESOLVED FIXED - 48601 - not assigned, UNCONFIRMEDNEW - 50988 - assigned, UNCONFIRMEDASSIGNEDRESOLVED FIXED - 43249 - assigned, UNCONFIRMEDNEWRESOLVED FIXED So, not bad but if there is more not assigned RESOLVED FIXED or NEW bugs with commits in the repo, then I see a problem here. QA should know what is happening with the bugs without the need to follow the comments. How does it look for commits? Also I see a lot of commits without bug report number in it. What is fixed then? I see it as another problem. I see you do not use QA field in Bugzilla. Why is that? Every component of a product can have its default QA specialist. Then developers responsible for a component quality can get a notice of new bugs instead of news about all of them. Also QA specialist assigned here could be responsible for bibisecting, bt delivery or verifying the bug. By the way - Browsing through dev/qa mailinglist I can see that you prefer those lists than bugs in Bugzilla. For instance - are those action items from ESC or QA meetings reported as bugs in Bugzilla? b.f.o is not used project wide or am I missing something? Do you VERIFY bugs? I see just 62 bugs as VERIFIED FIXED. I do not see such section in Getting Involved. Surely there are users willing to participate this way. I see you do not use flags and prefer Whiteboard. Why is that? Maybe a better idea than a MAB nightmare (IMHO meta bugs with 300 comments are useless) would be a release tracking flag, where QA would like to see a fix. Mozilla projects use them a lot for instance... Well, sorry for such a pack of off-topic questions, but I'd like to understand QA in this project better. Best regards. -- View this message in context:
Re: Cleaning bug list
Hi Joel, On 2012-06-15 at 23:09 -0700, Joel Madero wrote: I brainstormed a bit today and I came up with this flowchart. Looking for input. I read through email threads and see that prioritizing bugs has been an interesting discussion but as of now looks to be pretty unsettled. I'm going to make a similar chart for myself for setting bugs status. These kinds of things help me, not sure if they help others, but if they do, feel free to comment :) I like it - thanks for that! :-) This is meant as a general guideline, possibly to put up on the wiki for future and current QA/Devs to have a bit of focus. It's NOT meant to be me forcing everyone to conform as I know that QA requires quite a bit of discretion. Yes, it would be great to have that in the Wiki. Regards, Kendy ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Joel Madero píše v Pá 15. 06. 2012 v 23:09 -0700: I brainstormed a bit today and I came up with this flowchart. Looking for input. I read through email threads and see that prioritizing bugs has been an interesting discussion but as of now looks to be pretty unsettled. I'm going to make a similar chart for myself for setting bugs status. These kinds of things help me, not sure if they help others, but if they do, feel free to comment :) Cool! I like the logic. Also the flowchart is easier to understand than any text. I only miss regressions handling. There were long discussions that regressions should have higher priority than other bugs. Also we add the regression flag into Whiteboard. I think how to best handle it in the flow chart. I would add a subprocess at the end of each way. The question is what to do in this subprocess. We need to set regression in whiteboard. We also would want to increase the priority or even severity by one level. This is meant as a general guideline, possibly to put up on the wiki for future and current QA/Devs to have a bit of focus. It's NOT meant to be me forcing everyone to conform as I know that QA requires quite a bit of discretion. Sounds great! It is even impossible to use it as a strict rule because different people might have different opinion about how each function is important and how many users are affected. We need to add there a text that people should not fight about the severities and priorities. They are just helper tools for developers. Each volunteer developers has its own opinion anyway ;-) By the way. I wonder how complicated would be to rotate the chart to have the main questions from top to bottom. You know, it is much easier to scroll the page up and down using the mouse wheel. I am looking forward to see it on the wiki. Thanks a lot for working on this. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Joel Madero schrieb: I brainstormed a bit today and I came up with this flowchart. Hi Joel, great to see that all in a chart, your conclusions and definitions seem plausible. But the chart also shows the limitations of that concept: It's really sophisticated, and no developer will sit at his's desk to decide whether he should fix a Bug with Severity=major and Priority=normal before or after a bug with Severity=normal and Priority=high ;-) But as you stated, such a chart can give some orientation, and so I think it should be shown in the Wiki. Not on one of the current existing Pages like BugTriage or similar, but on a page where should gather detailed background information what would blow up those pages too much, but nevertheless are interesting or important to know for all who want to gain more knowledge concerning th bugfixing, QA or whatever else process. The question is whether the Priority entry is evaluated by anyone, IMHO most here see that as a personal statement (of reporter?) concerning importance of the bug - and ignore it. Best regards Rainer ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
I'll modify the orientation today or tomorrow and try to see where regression should fit. I think that it has to go in Priority and not in Severity. As for how devs use it, I agree completely that right now it's almost useless but maybe if it becomes more uniform and it actually provides some information as to what the bug is doingwho knows. For instance for me, at this point my abilities are pretty low so trivial bugs seem to be the go to...hopefully in the future this changes. As for users prioritizing themselves, I've (mentioned/suggested/got irritated with) in comments a couple users setting Severity to Critical for something as minor as Conditional Formatting, it's almost to the point that it is useless to let the end user categorize their own bugs like this. Maybe regression should automatically set Priority to Highest regardless of how many people are affected. Ok off to work. Glad it at least seems a little useful. I have such limited experience in programming/programming projects that I just felt a bit overwhelmed so this helped me quite a bit last night when I was going through bugs. Joel On 06/18/2012 04:21 AM, Rainer Bielefeld wrote: Joel Madero schrieb: I brainstormed a bit today and I came up with this flowchart. Hi Joel, great to see that all in a chart, your conclusions and definitions seem plausible. But the chart also shows the limitations of that concept: It's really sophisticated, and no developer will sit at his's desk to decide whether he should fix a Bug with Severity=major and Priority=normal before or after a bug with Severity=normal and Priority=high ;-) But as you stated, such a chart can give some orientation, and so I think it should be shown in the Wiki. Not on one of the current existing Pages like BugTriage or similar, but on a page where should gather detailed background information what would blow up those pages too much, but nevertheless are interesting or important to know for all who want to gain more knowledge concerning th bugfixing, QA or whatever else process. The question is whether the Priority entry is evaluated by anyone, IMHO most here see that as a personal statement (of reporter?) concerning importance of the bug - and ignore it. Best regards Rainer ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Rainer Bielefeld píše v Po 18. 06. 2012 v 13:21 +0200: Joel Madero schrieb: I brainstormed a bit today and I came up with this flowchart. Hi Joel, great to see that all in a chart, your conclusions and definitions seem plausible. But the chart also shows the limitations of that concept: It's really sophisticated, and no developer will sit at his's desk to decide whether he should fix a Bug with Severity=major and Priority=normal before or after a bug with Severity=normal and Priority=high ;-) You are right, developers will not strictly follow the severity and priority. On the other hand, they care of the usability of the application. They are actively working on the most annoying bugs. They prioritize bugs themselves, so why not help them. Note that MAB is only a workaround because all the other bugs are not prioritized. I think that it is hard job to prioritize all bugs. It will cause some troubles because different people might have different opinions. Though, I think that we should try it. But as you stated, such a chart can give some orientation, and so I think it should be shown in the Wiki. Not on one of the current existing Pages like BugTriage or similar, but on a page where should gather detailed background information what would blow up those pages too much, but nevertheless are interesting or important to know for all who want to gain more knowledge concerning th bugfixing, QA or whatever else process. You are right that we need to keep the bugtriage page easy to do not scare people. It would make sense to describe the prioritization on extra page and just link it from the BugTriage page. BTW: Recently someone said that the Bugtriage page was not easy to understand (state CONFIRMED). I think that some people might prefer the graphics flow chart over the text. It would be nice to have similar flow chart also for the overall bugtriage process :-) The question is whether the Priority entry is evaluated by anyone, IMHO most here see that as a personal statement (of reporter?) concerning importance of the bug - and ignore it. Good question. My opinion is: + severity defines how much a bug affect the functionality + priority defines order in which it should be fixed It means that, some feature might have higher priority because of marketing reasons. Also some less severe bug might have higher priority because it affects many users. Hmm, when I think about it, I would propose another prioritization: 1. Define severity by symptoms. + this is well handled in the flow chart. + alternative proposal is at at http://wiki.documentfoundation.org/BugReport_Details#Severity is reasonable. + I think that they both describe the same things different way. I do not have any strong opinion what formulation is easier to understand. We would need feedback from more people ;-) 2. Define priority more clever way: + the default priority should basically follow the severity; it means that critical bugs should have high priority, ... + the number of affected users and visibility should shift it higher or lower, though + it is already somehow handled in the flowchart. Well, I would, allow to skip even more levels. For example, typo in the main menu is minor bug but it might have even highest priority. Such bug in final release would look amateurish and would bring bad feeling from users. Hmm, I am not sure how to effectively describe it in the flowchart. + with this logic, the bugs with the high and highest priority should be mentioned in MAB. + anyway, developers should have the final word about priorities (for assigned bugs); they might want to organize their daily work by it Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Joel Madero píše v Po 18. 06. 2012 v 09:32 -0700: Version 2, changed orientation and tried to take comments into account. Let me know what you all think. It is much better readable. I finally got a better picture :-) Well, I think that it still need some thinking. You set inability to safe as major. I think that it should be blocker. The application would be almost useless with this bug ;-) I suggest the following changes in the top levels: Q: Does bug cause crash, loss of data[*], inability to install, broken core function, e.g. inability to safe any writer documents, print? + A: yes: = Q: Does it affect almost all users within everyday usage and is it a regression? + A: yes = blocker, highest, MAB + A: no = critical, high, MAB I would leave the + A: no = Q: Does bug involve a serious glitch such as tediously slow, inability to open particular documents, install some extensions, print on some printers? + A: yes: = major, high IMHO, the rest might stay as is. Well, it would be great to create several examples also for the other categories. I do not have power to invent them now, though ;-) [*] Note that data loss might be caused by incomplete import. If it is not a regression, it might be marked by developers as an enhancement. Note that it needs many developer/years to support some file formats. OOXML format is described on several thousands of pages... I am very happy with the progress and the state. Thanks a lot for working on it. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
I agree with the save comment, I'll change that right now. Also realized I didn't put a note in for regressions so I added to the bottom notes: **Regressions** Special attention should be paid to regressions. In most cases a regression calls for an increase in priority but in some cases it will not. If the regression is not elevated to a higher priority, a comment may be useful to explain why this decision was made Joel On Mon, Jun 18, 2012 at 10:54 AM, Petr Mladek pmla...@suse.cz wrote: Joel Madero píše v Po 18. 06. 2012 v 09:32 -0700: Version 2, changed orientation and tried to take comments into account. Let me know what you all think. It is much better readable. I finally got a better picture :-) Well, I think that it still need some thinking. You set inability to safe as major. I think that it should be blocker. The application would be almost useless with this bug ;-) I suggest the following changes in the top levels: Q: Does bug cause crash, loss of data[*], inability to install, broken core function, e.g. inability to safe any writer documents, print? + A: yes: = Q: Does it affect almost all users within everyday usage and is it a regression? + A: yes = blocker, highest, MAB + A: no = critical, high, MAB I would leave the + A: no = Q: Does bug involve a serious glitch such as tediously slow, inability to open particular documents, install some extensions, print on some printers? + A: yes: = major, high IMHO, the rest might stay as is. Well, it would be great to create several examples also for the other categories. I do not have power to invent them now, though ;-) [*] Note that data loss might be caused by incomplete import. If it is not a regression, it might be marked by developers as an enhancement. Note that it needs many developer/years to support some file formats. OOXML format is described on several thousands of pages... I am very happy with the progress and the state. Thanks a lot for working on it. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Hi Joel, your plans are great. On Thu, 2012-06-07 at 23:49 -0700, Joel Madero wrote: With thousands of unknown bugs I think that this will help us divvy up the work and prioritize a bit. After I do this phase I'll try to get consensus on how to prioritize. Obviously security bugs and resource leaks get highest, next I'm not sure but we can discuss this after. Just a side note. Some proposals of bug priorities were discussed in another thread, see: http://lists.freedesktop.org/archives/libreoffice/2012-June/032776.html http://lists.freedesktop.org/archives/libreoffice/2012-June/033011.html My opinion is that some prioritization would be highly appreciated. In each case, we should not fight about each bug. The severity and priority is just a helper information. It could motivate but it can't force volunteers to work on the bugs. Too many comments that does not bring any new information makes bugs less readable and worse to handle :-) Thanks for helping with sorting the bugs. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Sure thing. As soon as I do what I'm considering Phase 1 (just changing from UNCONFIRMED to NEEDINFO or NOTOURBUG) I'll be moving to the second phase which is to prioritize. I'll bookmark those links, appreciate the heads up. It would be nice if we could get 2-3 people to just pound through them and focus on nothing else until they are done but I know that's wishful thinking. My goal is to do 200 a week, that's still 5 months worth of sorting just for phase 1 :( But, progress is progress and once it's done we'll all be able to tackle these bugs a bit easier. Joel P.S. I still really think we need a CONFIRMED status so that devs don't have to look at UNCONFIRMED bugs or at least don't have to always look at them. I've voiced this a few times but I think I may be the only one On Wed, Jun 13, 2012 at 8:42 AM, Petr Mladek pmla...@suse.cz wrote: Hi Joel, your plans are great. On Thu, 2012-06-07 at 23:49 -0700, Joel Madero wrote: With thousands of unknown bugs I think that this will help us divvy up the work and prioritize a bit. After I do this phase I'll try to get consensus on how to prioritize. Obviously security bugs and resource leaks get highest, next I'm not sure but we can discuss this after. Just a side note. Some proposals of bug priorities were discussed in another thread, see: http://lists.freedesktop.org/archives/libreoffice/2012-June/032776.html http://lists.freedesktop.org/archives/libreoffice/2012-June/033011.html My opinion is that some prioritization would be highly appreciated. In each case, we should not fight about each bug. The severity and priority is just a helper information. It could motivate but it can't force volunteers to work on the bugs. Too many comments that does not bring any new information makes bugs less readable and worse to handle :-) Thanks for helping with sorting the bugs. Best Regards, Petr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Sure thing, I'll include it here and add a link as soon as I post over at freedesktop bugs 1. If there has been a request for information and there has been no response for 30+ days I'm putting NEEDINFO 2. If two or more people have said that they do not have the bug I'm doing the following if there hasn't been action for 30+ days: a. If it's stated that the bug was fixed in a recent release, I'm putting RESOLVED with a comment that if it's not for the author or someone else to open it back up b. If it's stated that it's not our bug I'm changing status to NOTOURBUG c. If it's stated that it never was a bug I'm putting NOTABUG with a comment saying to open it back up with more information if it is a bug 3. If it's confirmed by other people I'm changing it to confirmed 4. Of course I'm taking a glance at them to see if I can take them on, I've assigned two to myself. 5. If someone appears to be working on the bug and has implicitly or explicitly said they are doing it (ie. it's in progress, almost done, I'll take this one, etc..) I'm changing to assigned and adding a name With thousands of unknown bugs I think that this will help us divvy up the work and prioritize a bit. After I do this phase I'll try to get consensus on how to prioritize. Obviously security bugs and resource leaks get highest, next I'm not sure but we can discuss this after. I hope I'm not overstepping, just trying to help as much as possible as it seems like there is a bit of a back log. If this isn't wanted just let me know and I'll cease immediately. Thanks to everyone contributing to this great project. Joel On Thu, Jun 7, 2012 at 10:49 PM, Rainer Bielefeld libreoff...@bielefeldundbuss.de wrote: Joel Madero schrieb: Just a heads up to everyone. I'm doing a serious cleaning of bug reports. Changing a lot to NEEDINFO, RESOLVED, etc...We have way too many bugs listed as unknown that haven't been touched in months and have been confirmed to not be an issue or needs updated without updates from original authors of the bugs. Hope no one minds. Hi Joel, thank you for your initiative. Can you please present your ideas with some more details on libreoffice-qa@lists.**freedesktop.orglibreoffice...@lists.freedesktop.org ? Best regards Rainer Bielefeld ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Hi Joel, On Thu, 2012-06-07 at 23:49 -0700, Joel Madero wrote: Sure thing, I'll include it here and add a link as soon as I post over at freedesktop bugs This is prolly best on the libreoffice-qa list (I just CC'd it) - but it's interesting on the hackers list too. Your cleanup sounds most welcome [ to a non QA expert such as myself at least ] :-) With thousands of unknown bugs I think that this will help us divvy up the work and prioritize a bit. After I do this phase I'll try to get consensus on how to prioritize. Obviously security bugs and resource leaks get highest, next I'm not sure but we can discuss this after. Wonderful :-) I hope I'm not overstepping, just trying to help as much as possible as it seems like there is a bit of a back log. If this isn't wanted just let me know and I'll cease immediately. It's good to have more people getting stuck into the bug cleanup - of course, worth getting advice / input from Rainer the -qa list, but it sounds like you're on a good track to me :-) Thanks to everyone contributing to this great project. And thank-you ! it's really helpful to have more QA people scanning the bug lists, confirming bugs closing old duplicates. Having said that, there are thousands of open bugs - my report shows: 2.5k in the 'NEW' state, and 1.5k in the UNCONFIRMED state. Fixing 6bugs per day on average, that's a good pipeline of a couple of years work of fixing ;-) so the prioritisation work is really very greatly appreciated to ensure we fix the right bugs. Thanks ! Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Hi Joel, On 2012-06-07 at 23:49 -0700, Joel Madero wrote: 1. If there has been a request for information and there has been no response for 30+ days I'm putting NEEDINFO 2. If two or more people have said that they do not have the bug I'm doing the following if there hasn't been action for 30+ days: a. If it's stated that the bug was fixed in a recent release, I'm putting RESOLVED with a comment that if it's not for the author or someone else to open it back up b. If it's stated that it's not our bug I'm changing status to NOTOURBUG c. If it's stated that it never was a bug I'm putting NOTABUG with a comment saying to open it back up with more information if it is a bug 3. If it's confirmed by other people I'm changing it to confirmed 4. Of course I'm taking a glance at them to see if I can take them on, I've assigned two to myself. 5. If someone appears to be working on the bug and has implicitly or explicitly said they are doing it (ie. it's in progress, almost done, I'll take this one, etc..) I'm changing to assigned and adding a name Thanks so much for this - this is greatly appreciated! I like this approach, and I'd like to ask you for some additional points that would help a lot (if that fits your workflow): 6. If the bug talks about a misbehavior in a document, but the document is missing, NEEDINFO the reporter to provide the document. Similarly, if the bug says something like create document, do this, do that, do another thing, and then when you choose XY, it does AB instead of CD, NEEDINFO the reporter to create such a document, so that the developer can focus only on when you choose XY, it does AB instead of CD. 7. If the bug is a crash on Linux, ask the reporter for a backtrace, if it is not provided yet (unfortunately it is still way too hard to get the backtrace on Windows now) - ideally by pointing to: http://wiki.documentfoundation.org/BugReport#How_to_get_backtrace_.28on_Linux.29 8. If the bug is a crash, it is a probable candidate to become one of the Most Annoying Bugs; depending on the impact, consider making it dependant on https://bugs.freedesktop.org/show_bug.cgi?id=6 I hope I'm not overstepping, just trying to help as much as possible as it seems like there is a bit of a back log. If this isn't wanted just let me know and I'll cease immediately. The opposite - the more people join this effort, the better! :-) For more co-ordination, I am sure people on libreoffice-qa@ mailing list (CC'd) will help you. Thank you a lot, Kendy ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
Hi Joel, On Thu, Jun 07, 2012 at 11:49:52PM -0700, Joel Madero wrote: Sure thing, I'll include it here and add a link as soon as I post over at freedesktop bugs [...] I hope I'm not overstepping, just trying to help as much as possible as it seems like there is a bit of a back log. If this isn't wanted just let me know and I'll cease immediately. Thanks to everyone contributing to this great project. /me is overfilled with joy hearing this great news! Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: Cleaning bug list
One more thing to add to this. Last night when I did some (I think I did about 25-50 so it wasn't too many) I was doing the following: If someone asked is this reproducible in the latest release, but didn't say anything else as to if they themselves had tried to reproduce it. I would mark as NEEDINFO. I think that this is a bad policy as we can't expect users to constantly come back when a new release is done and update their bug saying still an issue. For these ones I'm going to leave them as UNKNOWN for now, I'll come back to them once some policy is decided on to handle them. Maybe what I'll do is at least with the Linux ones I'll try to confirm that the bug still exists and change status to CONFIRMED, if I am unable to confirm them I'll post a comment and change them to NEEDINFO. Joel On Fri, Jun 8, 2012 at 3:27 AM, Jan Holesovsky ke...@suse.cz wrote: Hi Joel, On 2012-06-07 at 23:49 -0700, Joel Madero wrote: 1. If there has been a request for information and there has been no response for 30+ days I'm putting NEEDINFO 2. If two or more people have said that they do not have the bug I'm doing the following if there hasn't been action for 30+ days: a. If it's stated that the bug was fixed in a recent release, I'm putting RESOLVED with a comment that if it's not for the author or someone else to open it back up b. If it's stated that it's not our bug I'm changing status to NOTOURBUG c. If it's stated that it never was a bug I'm putting NOTABUG with a comment saying to open it back up with more information if it is a bug 3. If it's confirmed by other people I'm changing it to confirmed 4. Of course I'm taking a glance at them to see if I can take them on, I've assigned two to myself. 5. If someone appears to be working on the bug and has implicitly or explicitly said they are doing it (ie. it's in progress, almost done, I'll take this one, etc..) I'm changing to assigned and adding a name Thanks so much for this - this is greatly appreciated! I like this approach, and I'd like to ask you for some additional points that would help a lot (if that fits your workflow): 6. If the bug talks about a misbehavior in a document, but the document is missing, NEEDINFO the reporter to provide the document. Similarly, if the bug says something like create document, do this, do that, do another thing, and then when you choose XY, it does AB instead of CD, NEEDINFO the reporter to create such a document, so that the developer can focus only on when you choose XY, it does AB instead of CD. 7. If the bug is a crash on Linux, ask the reporter for a backtrace, if it is not provided yet (unfortunately it is still way too hard to get the backtrace on Windows now) - ideally by pointing to: http://wiki.documentfoundation.org/BugReport#How_to_get_backtrace_.28on_Linux.29 8. If the bug is a crash, it is a probable candidate to become one of the Most Annoying Bugs; depending on the impact, consider making it dependant on https://bugs.freedesktop.org/show_bug.cgi?id=6 I hope I'm not overstepping, just trying to help as much as possible as it seems like there is a bit of a back log. If this isn't wanted just let me know and I'll cease immediately. The opposite - the more people join this effort, the better! :-) For more co-ordination, I am sure people on libreoffice-qa@ mailing list (CC'd) will help you. Thank you a lot, Kendy ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
[Libreoffice-qa] [Fwd: Re: Cleaning bug list]
I guess this should have been CC'd here ... Forwarded Message From: Joel Madero jmadero@gmail.com To: Rainer Bielefeld libreoff...@bielefeldundbuss.de Cc: libreoff...@lists.freedesktop.org Subject: Re: Cleaning bug list Date: Thu, 7 Jun 2012 23:49:52 -0700 Sure thing, I'll include it here and add a link as soon as I post over at freedesktop bugs 1. If there has been a request for information and there has been no response for 30+ days I'm putting NEEDINFO 2. If two or more people have said that they do not have the bug I'm doing the following if there hasn't been action for 30+ days: a. If it's stated that the bug was fixed in a recent release, I'm putting RESOLVED with a comment that if it's not for the author or someone else to open it back up b. If it's stated that it's not our bug I'm changing status to NOTOURBUG c. If it's stated that it never was a bug I'm putting NOTABUG with a comment saying to open it back up with more information if it is a bug 3. If it's confirmed by other people I'm changing it to confirmed 4. Of course I'm taking a glance at them to see if I can take them on, I've assigned two to myself. 5. If someone appears to be working on the bug and has implicitly or explicitly said they are doing it (ie. it's in progress, almost done, I'll take this one, etc..) I'm changing to assigned and adding a name With thousands of unknown bugs I think that this will help us divvy up the work and prioritize a bit. After I do this phase I'll try to get consensus on how to prioritize. Obviously security bugs and resource leaks get highest, next I'm not sure but we can discuss this after. I hope I'm not overstepping, just trying to help as much as possible as it seems like there is a bit of a back log. If this isn't wanted just let me know and I'll cease immediately. Thanks to everyone contributing to this great project. Joel On Thu, Jun 7, 2012 at 10:49 PM, Rainer Bielefeld libreoff...@bielefeldundbuss.de wrote: Joel Madero schrieb: Just a heads up to everyone. I'm doing a serious cleaning of bug reports. Changing a lot to NEEDINFO, RESOLVED, etc...We have way too many bugs listed as unknown that haven't been touched in months and have been confirmed to not be an issue or needs updated without updates from original authors of the bugs. Hope no one minds. Hi Joel, thank you for your initiative. Can you please present your ideas with some more details on libreoffice-qa@lists.freedesktop.org? Best regards Rainer Bielefeld ___ LibreOffice mailing list libreoff...@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: Cleaning bug list
Joel Madero schrieb: Just a heads up to everyone. I'm doing a serious cleaning of bug reports. Changing a lot to NEEDINFO, RESOLVED, etc...We have way too many bugs listed as unknown that haven't been touched in months and have been confirmed to not be an issue or needs updated without updates from original authors of the bugs. Hope no one minds. Hi Joel, thank you for your initiative. Can you please present your ideas with some more details on libreoffice...@lists.freedesktop.org? Best regards Rainer Bielefeld ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
