Re: [Libreoffice-qa] Minutes of ESC call: 2015-07-30
Hi all, Jan Holesovsky wrote on 01-08-15 02:43: + !!!please file easy hacks if you have ideas!!! (Bjoern) For ideas, query with needsDevEval in Whiteboard. At least part of those could be good candidates. Thanks, -- Cor Nouws GPD key ID: 0xB13480A6 - 591A 30A7 36A0 CE3C 3D28 A038 E49D 7365 B134 80A6 - vrijwilliger http://nl.libreoffice.org - volunteer http://www.libreoffice.org - The Document Foundation Membership Committee Member ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Minutes of ESC call: 2015-07-30
On Sun, Aug 2, 2015 at 11:25 AM, Markus Mohrhard markus.mohrh...@googlemail.com wrote: AI: + will have a look at the CppUnit to implement 'expected failure' (Jan-Marek) + Cpp logs are e.g. in workdir/CppunitTest/sal_rtl_math.test.log That is already possible with cppunit. Instead of using CPPUNIT_TEST use CPPUNIT_TEST_FAIL which tells cppunit that the test is expected to fail with a cppunit exception being thrown (it is extensively used in the cppunit internal unit tests). The test will start to fail when none of the asserts fail anymore. Keep in mind that it might be dangerous to use this with more than one assertion as an unexpected one might fail. That is not exactly what tha aim is... the aim is to have a failure be repport as such and not stop everything.. when a test is tagged as _can fail_ or something this is when a test is added before the fix also the minutes claim that cppunit log exist.. it is true they do, but they are not exploitable. I would like to be able to repport at the end of the build a nice recap module section nb_of_test #sucess #skipped #failed for each failed spacetestname : FAILED (optionally one line reason) the minute point to workdir/CppunitTest/sal_rtl_math.test.log which contains: OK (3) but writerperfect_writer.test.log fro instance contain 12027 lines.. most of it random trace messages, and I could not find any way to parse (at all, even less reliably) that thing to extract any useful information. maybe it is just a matter of using http://cppunit.sourceforge.net/doc/1.8.0/class_cpp_unit_1_1_xml_outputter.html and then adding a (optional) post processing step in the build to generate a nice summary Norbert ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Minutes of ESC call: 2015-07-30
* Areas that require unit tests with each bugfix (Markus/???)
+ https://wiki.documentfoundation.org/Development/Unit_Tests
+ updated the page with the areas where tests for fixes are
mandatory (Kendy)
+ so far: chart2, writer filters (import + export)
+ do we have more areas where we can have this rule? (Kendy)
+ important to have the maintainer who is checking the code,
and the test itself too (Kendy)
+ happened in the past that the test did not fail when the fix
was reverted (Kendy)
So just to make it clear as it seems that I gave the wrong impression. I
don't insist on all chart2 commits having a test (it would be nice if
everyone would write a test), however if I manage to review a patch and ask
for a test to a chart2 patch I expect that the patch is not pushed without
the test. In general I will ask for a test in all chart2 patches where it
makes sense. Of course I would be even more happy if everyone would just
write the test and I would only need to make sure that the test passes and
fails without the fix.
+ would be good to be able to make tests even for bugs that are not
fixed yet (Bjoern)
+ happened in the past too, much appreciated, helps fixing (Miklos)
+ do we have a way to mark such a case? (Jan-Marek)
+ not really (Norbert, Miklos)
+ 'make -k stagingcheck' - for tests that do not have a fix yet
(Bjoern)
+ isn't it enough to have them in gerrit? (Miklos)
+ can prototype that (Bjoern)
+ that's easy, harder is the infra to show them to the
people (Bjoern)
+ might be better to have 'expected failures' tests (Jan-Marek)
+ with easy way to move them to 'expected passes'
(Jan-Marek)
+ writing a test needs some knowledge, couldn't we actually mentor
the author to do a fix too? (Kendy)
+ can be much harder (Norbert, Bjoern)
+ problem is how to promote the existing tests that don't have the
fixes yet (Norbert)
+ more concerned to have some xxx tests run, yyy failed, zzz
passed (Norbert)
+ have the special rule could work, would like not to see that
some tests may fail
in the normal build, so that it is not perceived as 'normal
that some tests fail (Kendy)
AI: + will have a look at the CppUnit to implement 'expected failure'
(Jan-Marek)
+ Cpp logs are e.g. in
workdir/CppunitTest/sal_rtl_math.test.log
That is already possible with cppunit. Instead of using CPPUNIT_TEST use
CPPUNIT_TEST_FAIL which tells cppunit that the test is expected to fail
with a cppunit exception being thrown (it is extensively used in the
cppunit internal unit tests). The test will start to fail when none of the
asserts fail anymore. Keep in mind that it might be dangerous to use this
with more than one assertion as an unexpected one might fail.
Additionally there is CPPUNIT_TEST_EXCEPTION which expects as second
parameter the expected exception. So this is a replacement for the
following pattern that can sometimes be found in our code:
void Test::someTestMethod()
{
bool bExceptionThrown = false;
try
{
callMethodThatShouldThrow();
}
catch (ExpectedExceptionType)
{
bExceptionThrown = true;
}
CPPUNIT_ASSERT(bExceptionThrown);
}
+ people should look if there has been a -1 (Norbert)
+ -1 is not sticky, it is bound to the version of the patch
(Norbert)
+ please look if somebody gave -1 before you are reviewing and
pushing, to avoid too
many people using -2 which is sticky (Norbert)
+ if reverting a patch, please don't forget to let the original author
know (Kendy)
+ should we do a git hook to mail when a revert happened? (Norbert)
+ mechanic message maybe could make problems - human interaction
probably better? (Bjoern)
+ but letting know is important; usually there is a good reason
for a revent (Norbert)
+ it is not personal, it just happens (Norbert)
AI: + can have a look (Norbert)
+ mail to the author + committer + the person who reverted
(Bjoern)
+ best to get more reviewers in gerrit, so that reverts are seldom
(Bjoern)
+ how to handle when something is pushed to these areas anyway?
+ the same rules as when breaking the build (Miklos)
+ when the author is responsive, and working on the test, let
it in (Miklos)
+ when not, revert (Miklos)
+ timing is up to the maintainer (Miklos, Norbert)
+ it is always a trade-off (Bjoern)
+ best to talk to the maintainer, not to overrule him/her
(Norbert)
+ important to talk to people, don't push without talking when
overruling (Eike)
___
List Name: Libreoffice-qa mailing list
Mail address:
Re: [Libreoffice-qa] Minutes of ESC call: 2015-07-30
Hi, On Sun, Aug 02, 2015 at 06:25:05PM +0200, Markus Mohrhard wrote: That is already possible with cppunit. Instead of using CPPUNIT_TEST use CPPUNIT_TEST_FAIL which tells cppunit that the test is expected to fail with a cppunit exception being thrown (it is extensively used in the cppunit internal unit tests). The test will start to fail when none of the asserts fail anymore. Keep in mind that it might be dangerous to use this with more than one assertion as an unexpected one might fail. Additionally there is CPPUNIT_TEST_EXCEPTION which expects as second parameter the expected exception. So this is a replacement for the following pattern that can sometimes be found in our code: Yes, but for one we do not only have c++ tests, esp. since one of the aims is to get bugreporters (aka a broad audience) to improve their reports by adding (failing) tests: + writing a test needs some knowledge, couldn't we actually mentor the author to do a fix too? (Kendy) + can be much harder (Norbert, Bjoern) Also there might be unstable tests, for which expecting reliable failure is not what we want (rather we want to ignore unstable tests until they are stable and non-failing). Thus: + 'make -k stagingcheck' - for tests that do not have a fix yet (Bjoern) That doesnt preclude using the cppunit features you describe for really expected failures/exceptions, so thanks for the hints! Best, Bjoern ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Minutes of ESC call: 2015-07-30
Hey, On Sun, Aug 2, 2015 at 9:01 PM, Norbert Thiebaud nthieb...@gmail.com wrote: On Sun, Aug 2, 2015 at 11:25 AM, Markus Mohrhard markus.mohrh...@googlemail.com wrote: AI: + will have a look at the CppUnit to implement 'expected failure' (Jan-Marek) + Cpp logs are e.g. in workdir/CppunitTest/sal_rtl_math.test.log That is already possible with cppunit. Instead of using CPPUNIT_TEST use CPPUNIT_TEST_FAIL which tells cppunit that the test is expected to fail with a cppunit exception being thrown (it is extensively used in the cppunit internal unit tests). The test will start to fail when none of the asserts fail anymore. Keep in mind that it might be dangerous to use this with more than one assertion as an unexpected one might fail. That is not exactly what tha aim is... the aim is to have a failure be repport as such and not stop everything.. when a test is tagged as _can fail_ or something this is when a test is added before the fix So you want to have this as non-fatal tests. Ok, that is currently missing from cppunit but at least with the CPPUNIT_TEST_FAIL you can add tests before the bugfix and just change it from CPPUNIT_TEST_FAIL to CPPUNIT_TEST when the fix has been implemented. So it is a compromise as it is already there and we can have a look how difficult it would be to properly implement non-fatal tests in cppunit. also the minutes claim that cppunit log exist.. it is true they do, but they are not exploitable. I would like to be able to repport at the end of the build a nice recap module section nb_of_test #sucess #skipped #failed for each failed spacetestname : FAILED (optionally one line reason) the minute point to workdir/CppunitTest/sal_rtl_math.test.log which contains: OK (3) but writerperfect_writer.test.log fro instance contain 12027 lines.. most of it random trace messages, and I could not find any way to parse (at all, even less reliably) that thing to extract any useful information. maybe it is just a matter of using http://cppunit.sourceforge.net/doc/1.8.0/class_cpp_unit_1_1_xml_outputter.html and then adding a (optional) post processing step in the build to generate a nice summary It is even easier. You can add another TestListener in sal/cppunittester/cppunittester.cxx that logs all failures, and executed tests. Of course as there is no support for non-fatal tests right now you can't log them. It would at least give you an information how many tests are executed. I think there is no post-processing necessary as you get all the information the the xml outputter has already in the listener and process is directly there. Regards, Markus Norbert ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Minutes of ESC call: 2015-07-30
On Sat, 01 Aug 2015 02:43:41 +0200, Jan Holesovsky ke...@collabora.com wrote: * Release Engineering update (Christian) + 4.4.5 - RC2 status + released as final earlier today, done + will update the online update soon would you please take a look at this: https://bugs.documentfoundation.org/show_bug.cgi?id=92791#c8 is the update mechanism not yet correctly aligned to the latest release? -- Using Opera's revolutionary email client: http://www.opera.com/mail/ ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
[Libreoffice-qa] Minutes of ESC call: 2015-07-30
* Present: + Muthu, Thorsten, Eike, Kendy, Miklos, Norbert, Jan-Marek, Christian, David T., Sophie, Giuseppe C., Lionel, Robinson, Bjoern * Completed Action Items: + create presenter console cherry-pick request (Kendy) [ there already was one by Michael, but abandoned, due to the need of more testing ] + check EMF+ rendering on Windows with the presenter console bug fixed (Robinson) [ let the presenter console bugfix for 5.0.] + get a non-TDF branded 5-0 set into git / master (Cloph) [ have the new files, but OS X dnd - but in the works will push the new 5.0 branding to the libreoffice-5-0 branch ] + move existing git images into a private TDF branded pack (Cloph) + increase availble disk for crash-test VM136 (Cloph/Caolan) [ no need for a change ] + file conference papers [!] (All) + please submit abstracts: + http://conference.libreoffice.org/2015/call-for-papers/ + Ace of Base (Lionel) + poke Jay / Kendy do we need to revert UI / menu changes ? (Sophie) [ nothing necessary to revert in 5.0 (Sophie) ] + review ongoing pootle maintenance funding arrangements (Floeff) [ done (Sophie) ] * Pending Action Items: + review Giuseppe's webdav work in gerrit (Kendy/?) + https://gerrit.libreoffice.org/#/c/17189/ [ have a VM for testing against, code looks good in general, need to verify push (Kendy) ] + come up with a fun new name for 'blocker' (Robinson) [ still accepting new ideas ] + register at the conference: http://conference.libreoffice.org/2015/registration/ (all) [ mandatory from security reasons (Sophie) ] + re-arrange the help XML for the menu changes (Jay) [ Jay was away for vacation, promised to have a look at HelpAuthoring.oxt over the weekend (Kendy) ] + track support help XML status (Kendy) [ new HelpAuthoring.oxt released, more reports from Regina (Kendy) ] + UserAgent - drop bundled-languages (Michael) + UserAgent - produce a patch for review (Michael) + check 'make check' on a Windows 32bit box instead (Norbert) [ still don't have the machine ] + promote Development Mentoring Lead Job Offer through all channels: Universities, Companies, Clubs, ex-GSoCers etc. (all) https://blog.documentfoundation.org/2015/07/15/tdf-freelance-job-opening-201507-01-development-mentoring-lead/ [ please spread the word (Bjoern) ] [ print some leaflets too distribute (Bjoern) ] * Release Engineering update (Christian) + 4.4.5 - RC2 status + released as final earlier today, done + will update the online update soon + 5.0.0 - status / branch + heavy-duty triple-review process for libreoffice-5-0-0 branch + 1x review for 5-0 and +2 more for 5-0-0 + will tag the final later today + Major bugs + presenter console - not rendering on Windows (Michael) + https://bugs.documentfoundation.org/show_bug.cgi?id=91574 [ deferred to 5.0.1, invasive fix ] + was a problem with dictionaries on OS X (Cloph) + fixed now, and Norbert will double-check before building + anything else (Robinson) + 3 pending patches, please review! (Cloph) AI: + will poke people on the IRC (Cloph) + Late Feature Status (Michael) + LibreOfficeKit / Online tweaks (Kendy) + no changes - mostly changes going into master anyway. + OpenGL / double-buffer RenderContext (Kendy) + OpenGL by default not achievable for 5.0.0 + propose re-visit @ ESC for 5.0.1 for some H/W on Windows. + won't affect anything TDF ships for 5.0 + ongoing, but nothing for 5.0.0 + 5.0 splash / startcenter graphics / about dialog (Kendy?) + https://bugs.documentfoundation.org/show_bug.cgi?id=90452#c45 + did everything get in done. + https://bugs.documentfoundation.org/show_bug.cgi?id=93004 AI: + Cloph will fix the graphics accordingly to the report (Cloph) + no time for a full round-trip via the designer (Kendy) + Android iOS Remote * UX Update (Kendy) + call canceled, everybody on vacation * Documentation (Olivier) + new HelpAuthoring-3.1.0.oxt (Kendy) + and updated https://wiki.documentfoundation.org/Documentation/Help + Regina reported some problems, will have a look (Kendy) * Crashtest update (Caolán) + still close to 0 (Miklos) + 1 crash in DOCX import + 1 crash in ODT export (may be related to the import problem) * GSoC (Cedric) + Next important dates: + Aug. 21: pencil down start of evaluations + Aug. 28: final evaluations deadline + conference travel - poking the
