[Libreoffice] Contributing test cases
On Fri, 2010-11-19 at 16:03 +, Caolán McNamara wrote: On Fri, 2010-11-19 at 01:32 +0100, Miklos Vajna wrote: I'm asking because I guess the more complex to run it, the fewer people will actually try it out at all. True. I was sort of wondering about potential colossal size. But sure I guess a separate repo for megatest, optional like l10n in the normal course of things, and a make check would do the trick for me. Oh - incidentally; the KDE guys already have quite a nice test suite here: svn co svn://anonsvn.kde.org/home/kde/trunk/tests/kofficetests/ Warning: quite big ;-) I suggest we work with them to grow that - they seemed open to that. We could also use git-svn to move / mirror and add to it on freedesktop.org. Let me know if I should create a new repository for that. It'd would also be great to have some people to look through the tests to see what (if anything) breaks us, and/or we are missing, and/or do the shell instrumentation to simply load, save, and close all of those files :-) In fact - this is a really useful test; I ran an soffice under gdb, and then: for a in `find /opt/OpenOffice/kofficetests/interoperability -type f | grep -v '\.svn/'`; do ./soffice $a; sleep 1; done It failed after only a handful of documents on the attached with: #0 0xabd83b95 in writerfilter::dmapper::DomainMapper_Impl::finishParagraph(boost::shared_ptrwriterfilter::dmapper::PropertyMap) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #1 0xabd6e446 in writerfilter::dmapper::DomainMapper::utext(unsigned char const*, unsigned int) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #2 0xabcc17c6 in writerfilter::ooxml::OOXMLFastContextHandler::endOfParagraph() () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #3 0xabc1fefd in writerfilter::ooxml::OOXMLFactory_wml::endAction(writerfilter::ooxml::OOXMLFastContextHandler*) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #4 0xabca302d in writerfilter::ooxml::OOXMLFactory::endAction(writerfilter::ooxml::OOXMLFastContextHandler*, long) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #5 0xabcbda5a in writerfilter::ooxml::OOXMLFastContextHandler::lcl_endFastElement(long) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #6 0xabcbd45f in writerfilter::ooxml::OOXMLFastContextHandler::endFastElement(long) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #7 0xabf86c65 in sax_fastparser::FastSaxParser::callbackEndElement(char const*) () from /data/opt/OOInstall/program/../basis-link/program/fastsax.uno.so So ... I am certain, that simply building a large collection of documents exercising our various features, and then loading / saving / closing them all in a big long churning run every now and then would catch a ton of bugs for us :-) Any volunteers to play with it ? Thanks, Michael. -- michael.me...@novell.com , Pseudo Engineer, itinerant idiot mw07_template_urban_merge_fax.docx Description: application/vnd.openxmlformats-officedocument.wordprocessingml.document ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Contributing test cases
On Fri, 2010-12-03 at 11:57 +, Michael Meeks wrote: closing them all in a big long churning run every now and then would catch a ton of bugs for us :-) Any volunteers to play with it ? Hmm; perhaps this is just all .docx stuff since these are the first files in the test suite; but another 20 files (successfully loaded) later, I get a hang as a child of sax_parser - which is rather odd. So - this seems very fruitful, might help us catch our memory bloatage problems with lots of documents opened closed; and of course the prospect of leaving LibreOffice running inside valgrind around on some idle beast of a machine churning through the test-suite is quite a pleasant one I think. Fun, Michael. -- michael.me...@novell.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Contributing test cases
Hi Michael, On Fri, 2010-12-03 at 11:57 +, Michael Meeks wrote: It failed after only a handful of documents on the attached with: #0 0xabd83b95 in writerfilter::dmapper::DomainMapper_Impl::finishParagraph(boost::shared_ptrwriterfilter::dmapper::PropertyMap) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #1 0xabd6e446 in writerfilter::dmapper::DomainMapper::utext(unsigned char const*, unsigned int) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #2 0xabcc17c6 in writerfilter::ooxml::OOXMLFastContextHandler::endOfParagraph() () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #3 0xabc1fefd in writerfilter::ooxml::OOXMLFactory_wml::endAction(writerfilter::ooxml::OOXMLFastContextHandler*) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #4 0xabca302d in writerfilter::ooxml::OOXMLFactory::endAction(writerfilter::ooxml::OOXMLFastContextHandler*, long) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #5 0xabcbda5a in writerfilter::ooxml::OOXMLFastContextHandler::lcl_endFastElement(long) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #6 0xabcbd45f in writerfilter::ooxml::OOXMLFastContextHandler::endFastElement(long) () from /data/opt/OOInstall/program/../basis-link/program/libwriterfilterli.so #7 0xabf86c65 in sax_fastparser::FastSaxParser::callbackEndElement(char const*) () from /data/opt/OOInstall/program/../basis-link/program/fastsax.uno.so Do you have any docx in particular raising this? This is clearly a docx import problem ;) So ... I am certain, that simply building a large collection of documents exercising our various features, and then loading / saving / closing them all in a big long churning run every now and then would catch a ton of bugs for us :-) Any volunteers to play with it ? We already have some effort started in that direction in the build repository. Have a look at the bin/run-tests.sh script and the test folder... Those tests could be improved as only the ooxml tests are automated ATM and they are not even complete. Regards, -- Cédric Bosdonnat LibreOffice hacker http://documentfoundation.org OOo Eclipse Integration developer http://cedric.bosdonnat.free.fr ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Contributing test cases
On Fri, 2010-11-19 at 01:32 +0100, Miklos Vajna wrote: Is 3) about cppunit as well? If yes, what about just having 3) in the tree as well, and then it could be invoked with a 'make check' or similar that would run all those tests. I'm asking because I guess the more complex to run it, the fewer people will actually try it out at all. True. I was sort of wondering about potential colossal size. But sure I guess a separate repo for megatest, optional like l10n in the normal course of things, and a make check would do the trick for me. C. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
[Libreoffice] Contributing test cases
Hello, in order to report a subtle bug [1], I had to create a minimal failing test case document. Is there a way to contribute this kind of testcases, and assertions on which results one should expect, to a testsuite so that future changes to the LibreOffice code can be tested against these well-known bugs? In my case I'd need to describe something like «type this, type that, apply this style; save; in the resulting XML file there should not be a space between this element and this other element». Regards, [1] https://bugs.freedesktop.org/show_bug.cgi?id=31707 -- Gioele Barabucci gio...@svario.it ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Contributing test cases
On Thu, 2010-11-18 at 12:16 +0100, Gioele Barabucci wrote: Hello, in order to report a subtle bug [1], I had to create a minimal failing test case document. Is there a way to contribute this kind of testcases, and assertions on which results one should expect, to a testsuite so that future changes to the LibreOffice code can be tested against these well-known bugs? We really should try and organize this alright. I know there are some adhoc collections of documents here and there, e.g.scratch/sc-vba, but we need to get something organized and better automated IMO. What I'd like to see, and I'm still working on a few pieces, is to get basic cppunit build-time tests for the major apps together, e.g. a basic calc one is working under Linux and I've added another super-basic one I haven't enabled yet for starmath. As part of those tests I think I'd like to have a least one load/save for each file format, maybe, depends on the practicability of that. First though I need to get the smoketest reliable on windows at least. In my case I'd need to describe something like «type this, type that, apply this style; save; in the resulting XML file there should not be a space between this element and this other element». Something like that might be suitable for a build-time cppunit test. But either way we definitely should have slower automated tests that run separately. I feel the testtool thing is a bit of a disaster really, needs too much continual poking and adding timeouts etc to keep it going. There other random grabbags of stuff in qadevOOO, then there's the subsequent tests etc. What I think I'd like to see is 1) basic cppunit tests take place during the build, keep us honest and basic functionality working at all times. Have a few of these at the moment. 2) smoketest run at the end of the build, at least by the buildbots to make sure that works. This needs a little bit more love, at the moment I've some enthusiasm that I may have found some of problems that makes it die horribly at exit time for some people. 3) a bit optional module/repo filled with masses of test cases with a simple make target that gets run at release time and/or whenever qa folk want to test the build. C. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: [Libreoffice] Contributing test cases
On Thu, Nov 18, 2010 at 11:58:29AM +, Caolán McNamara caol...@redhat.com wrote: 1) basic cppunit tests take place during the build, keep us honest and basic functionality working at all times. Have a few of these at the moment. 2) smoketest run at the end of the build, at least by the buildbots to make sure that works. This needs a little bit more love, at the moment I've some enthusiasm that I may have found some of problems that makes it die horribly at exit time for some people. 3) a bit optional module/repo filled with masses of test cases with a simple make target that gets run at release time and/or whenever qa folk want to test the build. Hi, Is 3) about cppunit as well? If yes, what about just having 3) in the tree as well, and then it could be invoked with a 'make check' or similar that would run all those tests. I'm asking because I guess the more complex to run it, the fewer people will actually try it out at all. pgppBqGNMeWzF.pgp Description: PGP signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
