Re: [Libreoffice] Litmus testcase structure improvement.
Hi Cor, Thanks for spreading it! It is quite important we have a common sense on the working flow. I think your thinking is absolutely right and it is worth of taking time :) Thank you again! Best wishes, Yifan On Fri, Jul 01, 2011 at 07:36:12PM +0200, Cor Nouws wrote: Hi Yifan Yifan Jiang wrote (30-06-11 12:51) That would be great :) Thanks! I commited some words (untested) here today for Litmus admin: http://wiki.documentfoundation.org/Litmus#Litmus_for_Libreoffice_Admin_Guide Please also let me know if something is confusing. For the moment - and the few spare minutes I found, I start to understand it. I will - again - mail to the Dutch list to see if others step in. btw. would you do the changes in 'Master Regression TC' branches, where all the latest test cases should be tracked:) Yes, that is clear and makes much sense. But - noticing my current 'activity' here - you can imagine that much work is not likely to be done by me. Sorry, but hey, don't let that be discouraging. Anyone can stand up at any moment to help too :-) ATM for me most important is to understand it, so that I can point others (Dutch NL group) to it, and give some guidance. (For the rest, I try to keep on track with different releases, rc's, sometimes nightly builds, etc and work in that and try to find/file bugs when I come across something strange.) In addition, maybe the step by step working practice section 'Use case - Adding a new regression test case' would be helpful: http://wiki.documentfoundation.org/Litmus#Working_practices http://wiki.documentfoundation.org/Litmus#Step_by_Step_Working_practices Will look to that next days! Regards, ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice -- Yifan Jiang Libreoffice Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Hi Yifan Yifan Jiang wrote (30-06-11 12:51) That would be great :) Thanks! I commited some words (untested) here today for Litmus admin: http://wiki.documentfoundation.org/Litmus#Litmus_for_Libreoffice_Admin_Guide Please also let me know if something is confusing. For the moment - and the few spare minutes I found, I start to understand it. I will - again - mail to the Dutch list to see if others step in. btw. would you do the changes in 'Master Regression TC' branches, where all the latest test cases should be tracked:) Yes, that is clear and makes much sense. But - noticing my current 'activity' here - you can imagine that much work is not likely to be done by me. Sorry, but hey, don't let that be discouraging. Anyone can stand up at any moment to help too :-) ATM for me most important is to understand it, so that I can point others (Dutch NL group) to it, and give some guidance. (For the rest, I try to keep on track with different releases, rc's, sometimes nightly builds, etc and work in that and try to find/file bugs when I come across something strange.) In addition, maybe the step by step working practice section 'Use case - Adding a new regression test case' would be helpful: http://wiki.documentfoundation.org/Litmus#Working_practices http://wiki.documentfoundation.org/Litmus#Step_by_Step_Working_practices Will look to that next days! Regards, -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Yifan Jiang wrote (30-06-11 06:44) Hi Sophie / all, [...] (just to let you know that I see the mails etc, and hope to be able to pick up the next days - if not, it will be somewhere next week ..) -- - Cor - http://nl.libreoffice.org ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Hi Cor, That would be great :) Thanks! I commited some words (untested) here today for Litmus admin: http://wiki.documentfoundation.org/Litmus#Litmus_for_Libreoffice_Admin_Guide Please also let me know if something is confusing. btw. would you do the changes in 'Master Regression TC' branches, where all the latest test cases should be tracked:) In addition, maybe the step by step working practice section 'Use case - Adding a new regression test case' would be helpful: http://wiki.documentfoundation.org/Litmus#Working_practices Best wishes, Yifan On Thu, Jun 30, 2011 at 08:53:23AM +0200, Cor Nouws wrote: Yifan Jiang wrote (30-06-11 06:44) Hi Sophie / all, [...] (just to let you know that I see the mails etc, and hope to be able to pick up the next days - if not, it will be somewhere next week ..) ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice -- Yifan Jiang Libreoffice Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Hi all, A bit update as follows: On Wed, Jun 22, 2011 at 02:41:57PM +0800, Yifan Jiang wrote: TODO List I could think: 1. Litmus issue of test case manager access control (Thanks for Rimas is working on that) 2. We need to find some place to store test samples attachment of test cases (Need help, didn't follow up yet, maybe in git testing repo)? Thanks for petr's help, we can upload test cases to our git repo now: http://cgit.freedesktop.org/libreoffice/contrib/test-files/tree I'll follow up details later on. Any questions, feel free to ping. If you have any test samples for Litmus test cases to be upload and do not know how to handle it, please send it to me at this moment. Please make sure to tell which test cases are covered by the test samples. 4. Update wiki I updated several sections for admin stuff: http://wiki.documentfoundation.org/Litmus It would be appreciated if you can have a review to see how clear it is. We may need more practical use cases: http://wiki.documentfoundation.org/Litmus#Step_by_Step_Working_practices Best regards, Yifan -- Yifan Jiang Libreoffice Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
+1 Busy in a big show here... Kind regards Olivier Em 30-06-2011 03:53, Cor Nouws escreveu: Yifan Jiang wrote (30-06-11 06:44) Hi Sophie / all, [...] (just to let you know that I see the mails etc, and hope to be able to pick up the next days - if not, it will be somewhere next week ..) -- Olivier Hallot Founder, Steering Commitee Member - The Document Foundation Voicing the enterprise needs LibreOffice translation leader for Brazilian Portuguese +55-21-8822-8812 ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Hi Yifan, On Thu, Jun 30, 2011 at 2:05 PM, Yifan Jiang yfji...@novell.com wrote: Hi all, A bit update as follows: On Wed, Jun 22, 2011 at 02:41:57PM +0800, Yifan Jiang wrote: TODO List I could think: 1. Litmus issue of test case manager access control (Thanks for Rimas is working on that) 2. We need to find some place to store test samples attachment of test cases (Need help, didn't follow up yet, maybe in git testing repo)? Thanks for petr's help, we can upload test cases to our git repo now: http://cgit.freedesktop.org/libreoffice/contrib/test-files/tree I'll follow up details later on. Any questions, feel free to ping. If you have any test samples for Litmus test cases to be upload and do not know how to handle it, please send it to me at this moment. Please make sure to tell which test cases are covered by the test samples. 4. Update wiki I updated several sections for admin stuff: http://wiki.documentfoundation.org/Litmus It would be appreciated if you can have a review to see how clear it is. We may need more practical use cases: http://wiki.documentfoundation.org/Litmus#Step_by_Step_Working_practices Ok, thanks a lot for the updates and taking care of this. I'll review it later today. Kind regards Sophie -- Founding member of The Document Foundation ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Hi Yifan, On 30/06/2011 14:05, Yifan Jiang wrote: Hi all, A bit update as follows: On Wed, Jun 22, 2011 at 02:41:57PM +0800, Yifan Jiang wrote: TODO List I could think: 1. Litmus issue of test case manager access control (Thanks for Rimas is working on that) 2. We need to find some place to store test samples attachment of test cases (Need help, didn't follow up yet, maybe in git testing repo)? Thanks for petr's help, we can upload test cases to our git repo now: http://cgit.freedesktop.org/libreoffice/contrib/test-files/tree I'll follow up details later on. Any questions, feel free to ping. oh I forgot in my other, it's great, thanks a lot to Petr :) If you have any test samples for Litmus test cases to be upload and do not know how to handle it, please send it to me at this moment. Please make sure to tell which test cases are covered by the test samples. 4. Update wiki I updated several sections for admin stuff: http://wiki.documentfoundation.org/Litmus It would be appreciated if you can have a review to see how clear it is. We may need more practical use cases: For me it's ok, but I didn't follow the steps, only used my memory. Thanks a lot for your work! May be Cor or Oliver, you can follow the step by step to make sure that nothing in missing in the steps. http://wiki.documentfoundation.org/Litmus#Step_by_Step_Working_practices I'll add a translation part tomorrow for those who only want to make the translation and no admin work. Kind regards Sophie -- Founding member of The Document Foundation ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
On Wed, Jun 22, 2011 at 02:41:57PM +0800, Yifan Jiang wrote: 3. In DE testgroup, the impress and draw are separated and they got more cases than other languages. (Yifan will follow up this) Hi all, The difference is obvious to see in run report : https://tcm.documentfoundation.org/test_run_report.cgi?test_run_id=6 Subgroup: #DE - Impress Subgroup: #DE - Draw Subgroup: #EN - Impress Draw Subgroup: #FR - Impress Draw ... IMHO, The DE version test cases (which separate Impress and Draw test case groups) look more reasonable and easy to handle against combining Impress and Draw cases. Not sure if we had particular concerns to put the 2 set of cases together in English veresion (probably other languages version are derived from that)? So what's your opinion to: 1. divide Impress/Draw cases finally for all languages. 2. translate the incremental part of DE cases to English, then from English to all other languages :) Thanks! Best wishes, Yifan -- Yifan Jiang Libreoffice Contact: yifan - irc.freenode.net/libreoffice = http://www.libreoffice.org/ http://www.documentfoundation.org/ ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
Hi Yifan, all On 29/06/2011 11:28, Yifan Jiang wrote: On Wed, Jun 22, 2011 at 02:41:57PM +0800, Yifan Jiang wrote: 3. In DE testgroup, the impress and draw are separated and they got more cases than other languages. (Yifan will follow up this) Hi all, The difference is obvious to see in run report : https://tcm.documentfoundation.org/test_run_report.cgi?test_run_id=6 Subgroup: #DE - Impress Subgroup: #DE - Draw Subgroup: #EN - Impress Draw Subgroup: #FR - Impress Draw ... IMHO, The DE version test cases (which separate Impress and Draw test case groups) look more reasonable and easy to handle against combining Impress and Draw cases. Not sure if we had particular concerns to put the 2 set of cases together in English veresion (probably other languages version are derived from that)? So what's your opinion to: 1. divide Impress/Draw cases finally for all languages. 2. translate the incremental part of DE cases to English, then from English to all other languages :) Yes, and we should really get the same tests in every languages. @the German group, please, if you add/modify tests, let us know so that we can discuss and report the changes to get them for all the testers. Being able to compare or check the same sets in all languages is part of the QA and the overall quality of the process. Thanks in advance :) Kind regards Sophie -- Founding member of The Document Foundation ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Litmus testcase structure improvement.
TODO List I could think: 1. Litmus issue of test case manager access control (Thanks for Rimas is working on that) 2. We need to find some place to store test samples attachment of test cases (Need help, didn't follow up yet, maybe in git testing repo)? 3. In DE testgroup, the impress and draw are separated and they got more cases than other languages. (Yifan will follow up this) 4. Update wiki (Not done/assigned yet). 5. We need feature test writing. I suggest it could be as simple as a link to somewhere else, or a brief sentence to tell clearly what the feature functions like and where to find it in user space :) Until the feature is as important as to be merged into Master Regression branch, we fulfil the case in detail. (need to start) Best wishes, Yifan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
[Libreoffice] Litmus testcase structure improvement.
Hi all, Referring to the attached discussion thread initialed by Jean-Baptiste (great thanks!), we are trying to make testcase management tool Litmus (https://tcm.documentfoundation.org/run_tests.cgi) better managed and organized. The following explained all current states. As usual, comments are welcome! Thanks for all the support! [For QA people running cases] The run created by new method for 3.4.1: https://tcm.documentfoundation.org/run_tests.cgi?test_run_id=13 Basically no practical changes from before. Meanwhile please notice: a. make sure the build version you are using is suitable for the corresponding test run. For example, for the test run - LibreOffice 3.4.0 RC Regression test libreoffice build 3.4.0.x is needed. b. Please make sure the platform, operating system and build id information, which should be filled before the testing, exactly reflecting the testing environment. c. Attaching a bug id to a failed case result would be very nice! [For Litmus admins] We are trying to make every thing much more clear and easy to update. The current test case organization changes: a. Divide test case set to Regression test and New Feature test ([1]) b. Regression Test A branch *Master Regression TC* was created specifically. So that we can all work on a consistent test case base, from which new branches can be always *cloned*. Keep all new/updated cases in this branch would be necessary to make everything inside reusable. That is to say, we use the Master branch to keep a reusable and continuously maintanable test cases storage, but NOT for creating test runs. When a new build (big version as x.x) will come, we 1. clone (Recursively) a new regression test branch for the build from Master branch, like 3.4 Regresstion Test (branch) 3.5 Regresstion Test (branch) ... 2. through the branches, create a test run for the corresponding builds: libreoffice 3.4.0 rc regresstion test (run created from 3.4 branch) libreoffice 3.4.1 rc regresstion test (run created from 3.4 branch) libreoffice 3.4.2 rc regresstion test (run created from 3.4 branch) libreoffice 3.5.0 rc regresstion test (run created from 3.5 branch) libreoffice 3.5.1 rc regresstion test (run created from 3.5 branch) libreoffice 3.5.2 rc regresstion test (run created from 3.5 branch) ... *NOTICE* If some new cases needed, please create them in the Master Testcase branch. Thus we can use the new regression test case set next time creating branches([2]). I have noticed a new set of pt-BR test cases created in '3.4 Regression test' branch today, I will 'sync' them to the Master branch later. c. New Feature Test A branch *Master Feature TC* was created specifically. This is more like an outline without cases (but with group subgroup info) filled in. So for each new big version (x.x), 1. clone from the Master Feature branch: 3.4 Feature Test (branch) 3.5 Feature Test (branch) 3.6 Feature Test (branch) 2. we create a single run for each big version: libreoffice 3.4 feature test (run created from 3.4 branch) libreoffice 3.5 feature test (run created from 3.5 branch) libreoffice 3.6 feature test (run created from 3.6 branch) ... 3. we 'seek' and write new feature cases in the corresponding subgroup according to git log, mailing list, release notes or so. 4. we update important feature case to Master Regression TC branch [Footnotes] [1] Regression and Feature test are there for quite different purpose: The Regression test would hold relatively stable set of test cases, which should be ran in each of the release build. The New Feature test would be dynamically changed based on a big version update (3.3-3.4). Meanwhile New feature test might be run from even early development build phase. [2] As you may notice, by this process we didn't really change the running branch. However I didn't find an easy way to sync test cases between branches :( A workaround method to 'link' new test cases is: 1. Add a new test cases in the Master branch, and assign the cases to a proper subgroup 2. Edit current running branch's subgroup to link a new
