Re: fdo#55814: unit test is missing
On 01/05/13 23:33, David Ostrovsky wrote: On 30.04.2013 23:46, Stephan Bergmann wrote: On 04/29/2013 11:42 PM, David Ostrovsky wrote: cli_ure is failing to compile here atm. How's it failing for you exactly? I'd hope it does work now, across various Windows build configurations. yes, after you've fixed it ;-) so, with this patch https://gerrit.libreoffice.org/#/c/3728/1 unit tests are invoked now on windows, but python is failing to locate the tests: ImportError: No module named 'get_expression' that's because the PYTHONPATH was not extended: PYTHONPATH=$S/instdir/wntmsci13.pro/program/python-core-3.3.0/lib;$S/instdir/wntmsci13.pro/program/python-core-3.3.0/lib/lib-dynload:$S/instdir/wntmsci13.pro/program as expected with this function (PythonTest.mk): # gb_PythonTest_add_modules directory module(s) define gb_PythonTest_add_modules $(call gb_PythonTest_get_target,$(1)) : PYPATH := $$(PYPATH):$(2) [...] And i don't see why ;-( problem is the :, on Windows it should be ;... the CLASSPATH stuff in the java makefiles has the same problem, there is some variable there for the separator that should fix that. by the way if the test fails to create a document the first thing you should check in the debugger is if the process has 2 tklo.dll loaded (that's what happened last time i tried and it definitely doesn't work). ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On 30.04.2013 23:46, Stephan Bergmann wrote: On 04/29/2013 11:42 PM, David Ostrovsky wrote: cli_ure is failing to compile here atm. How's it failing for you exactly? I'd hope it does work now, across various Windows build configurations. yes, after you've fixed it ;-) so, with this patch https://gerrit.libreoffice.org/#/c/3728/1 unit tests are invoked now on windows, but python is failing to locate the tests: ImportError: No module named 'get_expression' that's because the PYTHONPATH was not extended: PYTHONPATH=$S/instdir/wntmsci13.pro/program/python-core-3.3.0/lib;$S/instdir/wntmsci13.pro/program/python-core-3.3.0/lib/lib-dynload:$S/instdir/wntmsci13.pro/program as expected with this function (PythonTest.mk): # gb_PythonTest_add_modules directory module(s) define gb_PythonTest_add_modules $(call gb_PythonTest_get_target,$(1)) : PYPATH := $$(PYPATH):$(2) [...] And i don't see why ;-( David ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On 04/29/2013 11:42 PM, David Ostrovsky wrote: cli_ure is failing to compile here atm. How's it failing for you exactly? I'd hope it does work now, across various Windows build configurations. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On 04/27/2013 12:40 PM, David Ostrovsky wrote: Python test machinery (in process) is up and running on system and internal python and on all platforms. Many thanks to Michael S. and Stephan for helping out with this! Is it really working on all platforms, incl. Windows? If yes, why is sw/Module_sw.mk still ifneq ($(DISABLE_PYTHON),TRUE) ifneq ($(OS),WNT) $(eval $(call gb_Module_add_subsequentcheck_targets,sw,\ PythonTest_sw_python \ )) endif endif Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On 04/27/2013 12:40 PM, David Ostrovsky wrote: so I migrated the fdo#55814 java unit test to python: sw/qa/complex/writer/VarFields.java sw/qa/python/var_fields.py http://cgit.freedesktop.org/libreoffice/core/commit/?id=9e0502f27d4a3069540eaf6090ed0b5936889fb2 comment out storeToURL() as it crashes with debug build is fixed with http://cgit.freedesktop.org/libreoffice/core/commit/?id=b993c5591e965de3b9e95e32007ea4231b2997d7 RuntimeException missing from exception specification for UNO method now. Stephan ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On 29.04.2013 09:41, Stephan Bergmann wrote: Is it really working on all platforms, incl. Windows? If yes, why is sw/Module_sw.mk still ifneq ($(DISABLE_PYTHON),TRUE) ifneq ($(OS),WNT) $(eval $(call gb_Module_add_subsequentcheck_targets,sw,\ PythonTest_sw_python \ )) endif endif good catch, can not verify it: cli_ure is failing to compile here atm. David ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
Hi, On Fri, 2013-03-22 at 10:58 +0100, Miklos Vajna wrote: Hi David, On Thu, Mar 21, 2013 at 10:42:27AM +0100, David Ostrovsky wrote: I wonder if it would make sense to establish Python_test machinery in our build system to be able to write such and many other tests in python instead and just say make Pytest_sw_complex instead of make JunitTest_sw_complex (python have got unittest module that we can start with)? Hmm, yes, I think that would be an improvement. done :-) make PythonTest_sw_python Python test machinery (in process) is up and running on system and internal python and on all platforms. Many thanks to Michael S. and Stephan for helping out with this! so I migrated the fdo#55814 java unit test to python: sw/qa/complex/writer/VarFields.java sw/qa/python/var_fields.py Problem (as i described in the python test file): while java unit test is failing as expected, python unit test doesn't fail... Unlike in java, field's test condition is not get overridden in python. Apparently xDoc.refresh() is somehow broken (from python), because doing the update manually fix this: 1. run the python test: make PythonTest_sw_python 2. open created document workdir/unxlngx6.pro/PythonTest/sw_python/VarFields.odt 3. check that TextSection's condition still has the right value: foo EQ 1 4. update field with Tool=Update=Fields (or [F9]) 5. check that TextSection's condition was overridden: 0 Another UNO bug? Thanks David ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
yet another unit test framework? -- was fdo#55814: unit test is missing
[Moving this discussion to ML to have better visibility] with https://gerrit.libreoffice.org/#/c/3128/ we have support for unit tests written in python. (We have even found two bugs with it already ... and fixed). I am not going to provide the huge advantages of dynamic type languages in general here, but while python is very impressive it *is* truly read-write language compare to number of write-only languages, that used in LO ecosystem. Yes, it is probably true that you can not easily debug these unit tests. But is the debuggability the only argument here? I doubt it. We have logging framework and in the end one can still migrate python unit test to C++ (if needed) to debug it. Thanks David On Fri, 2013-03-22 at 10:58 +0100, Miklos Vajna wrote: Hi David, On Thu, Mar 21, 2013 at 10:42:27AM +0100, David Ostrovsky d.ostrov...@idaia.de wrote: And yes i am going to migrate it to C++ as you argued that it would be easier to debug then java. I wonder if it would make sense to establish Python_test machinery in our build system to be able to write such and many other tests in python instead and just say make Pytest_sw_complex instead of make JunitTest_sw_complex (python have got unittest module that we can start with)? Hmm, yes, I think that would be an improvement. As far as I understand, most of the Java tests have two difficulties: 1) A separate soffice process is started, then Java code connects to this, and executes tests. This is a bit slow, compred to the C++ unit tests, where we bootstap UNO ourselves + a bit painful to debug, as you have to run one process in gdb (listening on a socket) and an other process to trigger the problem. If do a 1:1 conversion of Java tests to Python, this will be still an issue. IMHO doing the C++ way for Python (running the tests in a single process) makes more sense. (Don't confuse these C++ tests with the various uwriter/ucalc/etc tests, which even have access to private library symbols.) 2) When a unit test fails, it's handy to step the unit test line-by-line in gdb to see exactly which line triggers an exception, etc. I imagine this only works if you write the test in C++, but even with basic or python, it should not be *that* bad, as we can have the interpreter with debug symbols, etc. I think in this second case even a 1:1 conversion from java to python would help a lot. And after all, be sure to talk to Markus, he's the testing expert, not me. :-) HTH, Miklos ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
while python is very impressive it *is* truly read-write language compare to number of write-only languages, Could we please stop this childish language bigotry? --tml ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
I'm extremely grateful to anyone willing to work on improving our unit test infrastructure. However... On 2013-04-04 10:24, David Ostrovsky wrote: I am not going to provide the huge advantages of dynamic type languages in general here, but while python is very impressive it *is* truly read-write language compare to number of write-only languages, that used in LO ecosystem. Yes, it is probably true that you can not easily debug these unit tests. But is the debuggability the only argument here? I doubt it. We have Yes it very much is. I'm currently struggling with visibility into a failing unit test, and the dual Java/C++ nature of the unit test makes it incredibly hard for me to find the source of the problem. I am, with great joy, looking forward to the day when almost all of our unit tests are written in C++ so I don't have to jump through hoops when debugging. Besides, the C++ unit tests, thanks to the excellent frameworks we use, have very little extra verbiage compared to their python counterparts. Disclaimer: http://www.peralex.com/disclaimer.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
David Ostrovsky wrote: with https://gerrit.libreoffice.org/#/c/3128/ we have support for unit tests written in python. (We have even found two bugs with it already ... and fixed). I like it, because it probably increases the number of people able to write unit tests. I dislike it, since as you admit yourself it makes debugging a failing test a possibly frustrating experience. Then again, turning a working python test into a working C++ one is probably a rather mechanical EasyHack. Therefore, a net benefit in my mind. Out of curiosity, was that sparked by the Python test discussion at the Impress Sprint? Cheers, -- Thorsten signature.asc Description: Digital signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
On Thu, Apr 04, 2013 at 09:37:51AM +0200, Noel Grandin wrote: Yes, it is probably true that you can not easily debug these unit tests. But is the debuggability the only argument here? I doubt it. We have Yes it very much is. I'm currently struggling with visibility into a failing unit test, and the dual Java/C++ nature of the unit test makes it incredibly hard for me to find the source of the problem. Well, IMHO the main problem with the unoapi tests wrt this is that they 'centralized' a lot of the expectations on the UNO-Api -- which made them hard to quickly rewrite in C++. THe complex tests should not have this issue. Other than that, IMHO this is mostly a false dilemma -- not everyone who might write Python tests would write C++ tests. If this leads to more reliable tests of any kind, which will turn into tests of the C++ kind when they first fail, Im all for it. A side issue: As I wrote in: http://skyfromme.wordpress.com/2013/03/19/autopkgtests-for-adults/ I enabled running the (Java) tests against a version installed into the system -- which is better than testing against a version not into the system, so I would love to keep that ability when we get more C++ tests (for Python tests I assume that to be trivial). @moggi: As I havent looked into that yet -- would you see any blockers there? Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
2013/4/4 David Ostrovsky d.ostrov...@gmx.de [Moving this discussion to ML to have better visibility] with https://gerrit.libreoffice.org/#/c/3128/ we have support for unit tests written in python. (We have even found two bugs with it already ... and fixed). I am not going to provide the huge advantages of dynamic type languages in general here, but while python is very impressive it *is* truly read-write language compare to number of write-only languages, that used in LO ecosystem. Do we really want to start a discussion on this level? Yes, it is probably true that you can not easily debug these unit tests. But is the debuggability the only argument here? I doubt it. We have logging framework and in the end one can still migrate python unit test to C++ (if needed) to debug it. Yes debugging is the main argument for maintainability of our tests. While it has not the nicest syntax it is at least easy for the person debugging a failing test. And often (just according to math) the person debugging a failing unit test is not the one who wrote it so the argument rewrite it when you need to debug it is a bit lame. IMO even debugging the c++ should be easier as we can see with random people running into test failures that the common advice is to disable the test instead of debugging it. I fear that we see this effect much more in the python tests as more people will follow that path when a test randomly fails (and yes every test will fail randomly at some point on a strange platform). To some degree we have the same problem in c++ but until now we were able to limit this behavior mainly to disabled test cases for BSD. Also I'm not the biggest fan of the argumentation that it allows more people to write unit tests. I still believe that tests are mainly written after a bug has been fixed which means that the developer knows at least a bit of C++ and with the existing testing infrastructure adding a test case to one of the existing tests is hopefully easy enough. If it is not we should work on making it easier to write the C++ based test. Additionally an example out of Calc/Impress that the argumentation make writing test easier and magically people will show up writing them is not true: We have for Calc and Impress existing test frameworks that require no coding from the person writing the test and I had exactly two persons after long advocating at conferences who contributed one test case to Calc. I'll stop here because I think I made my point but I have a few other arguments. Personally I'm against python based tests especially if they are out-of-process but as long as someone agrees to maintain them (that also means that this person must feel responsible when a test fails in some rare circumstances and nobody else cares) I can live with them. Regards, Markus ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
On 2013-04-04 10:53, Bjoern Michaelsen wrote: Well, IMHO the main problem with the unoapi tests wrt this is that they 'centralized' a lot of the expectations on the UNO-Api -- which made them hard to quickly rewrite in C++. If this leads to more reliable tests of any kind, which will turn into tests of the C++ kind when they first fail, Im all for it. These two statements are mutually contradictory. Either Java/Python unit tests are easy to convert to C++, or they are not, you can't have it both ways :-) Besides, LO is a C++ program - if you can already code in C++, why would you want to switch to a different language to write a unit test? Disclaimer: http://www.peralex.com/disclaimer.html ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
Hi, On Thu, Apr 04, 2013 at 11:00:17AM +0200, Noel Grandin wrote: On 2013-04-04 10:53, Bjoern Michaelsen wrote: Well, IMHO the main problem with the unoapi tests wrt this is that they 'centralized' a lot of the expectations on the UNO-Api -- which made them hard to quickly rewrite in C++. If this leads to more reliable tests of any kind, which will turn into tests of the C++ kind when they first fail, Im all for it. These two statements are mutually contradictory. Either Java/Python unit tests are easy to convert to C++, or they are not, you can't have it both ways :-) No: The unoapi tests are hard to convert (not because of them being in Java, but because of the qadevOOo framework needing a C++ equivalent), the complex tests are not. Besides, LO is a C++ program - if you can already code in C++, why would you want to switch to a different language to write a unit test? Because there are people willing to write Python tests but unable (or unwilling to invest the time) to write C++ tests? I dont see that as a problem. A reliable test should succeed almost always -- and in that case it doesnt matter in what language it is written in. If it fails, its probably well worth the time to reimplement it in C++ proper. But I see little reason to dogmatically write all tests in C++ as: - more people can write tests in Python - there are tests that might never fail (and hopefully they are the majority) and the language they are written in does not matter at all in that case Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
On Thu, Apr 04, 2013 at 10:57:59AM +0200, Markus Mohrhard wrote: 2013/4/4 David Ostrovsky d.ostrov...@gmx.de I am not going to provide the huge advantages of dynamic type languages in general here, but while python is very impressive it *is* truly read-write language compare to number of write-only languages, that used in LO ecosystem. Do we really want to start a discussion on this level? /me hands Markus a certificate confirming him as the 'trollfeeder of the month' -- with Tor a close second in the category. ;) Best, Bjoern ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: yet another unit test framework? -- was fdo#55814: unit test is missing
Markus Mohrhard wrote: Also I'm not the biggest fan of the argumentation that it allows more people to write unit tests. There were at least two people at the Impress Sprint willing to write tests, but only able to do so in Python. I think this project has the general policy of welcoming any contribution of good quality, provided it does not totally mess with the architecture. As you say below, if people are willing to maintain it, let's have it in please. :) Cheers, -- Thorsten signature.asc Description: Digital signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
Hi David, On Thu, Mar 21, 2013 at 10:42:27AM +0100, David Ostrovsky d.ostrov...@idaia.de wrote: And yes i am going to migrate it to C++ as you argued that it would be easier to debug then java. I wonder if it would make sense to establish Python_test machinery in our build system to be able to write such and many other tests in python instead and just say make Pytest_sw_complex instead of make JunitTest_sw_complex (python have got unittest module that we can start with)? Hmm, yes, I think that would be an improvement. As far as I understand, most of the Java tests have two difficulties: 1) A separate soffice process is started, then Java code connects to this, and executes tests. This is a bit slow, compred to the C++ unit tests, where we bootstap UNO ourselves + a bit painful to debug, as you have to run one process in gdb (listening on a socket) and an other process to trigger the problem. If do a 1:1 conversion of Java tests to Python, this will be still an issue. IMHO doing the C++ way for Python (running the tests in a single process) makes more sense. (Don't confuse these C++ tests with the various uwriter/ucalc/etc tests, which even have access to private library symbols.) 2) When a unit test fails, it's handy to step the unit test line-by-line in gdb to see exactly which line triggers an exception, etc. I imagine this only works if you write the test in C++, but even with basic or python, it should not be *that* bad, as we can have the interpreter with debug symbols, etc. I think in this second case even a 1:1 conversion from java to python would help a lot. And after all, be sure to talk to Markus, he's the testing expert, not me. :-) HTH, Miklos signature.asc Description: Digital signature ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
Hi Miklos, On Mon, 2013-03-18 at 11:18 +0100, Miklos Vajna wrote: Hi David, On Sat, Mar 09, 2013 at 02:29:54PM +0100, David Ostrovsky david.ostrov...@gmx.de wrote: I wonder why we don't have any unit test for that part of code? We do have one: http://cgit.freedesktop.org/libreoffice/core/commit/?id=10e02dfdffb5ef3a02a40b52c6cda176f7f4447c It's just not testing the case we have in fdo#55814. Thank you for that pointer, the question was about programmatic creation of such a document and not loading an existing one. Guys was laughing at me that i am trying to create one ;-) But it turns out, that only 21 steps are needed (well at least in java) and here we go: http://cgit.freedesktop.org/libreoffice/core/commit/?id=bb59e2e22161fcf66e3c71f58f5b160a24c9 And yes i am going to migrate it to C++ as you argued that it would be easier to debug then java. I wonder if it would make sense to establish Python_test machinery in our build system to be able to write such and many other tests in python instead and just say make Pytest_sw_complex instead of make JunitTest_sw_complex (python have got unittest module that we can start with)? Last but not least, thank you, Michael Stahl and Stephan for helping me to set it all up ;-) David ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On Sat, 2013-03-09 at 14:29 +0100, David Ostrovsky wrote: i've been told yesterday that the usage of writer is severly restricted. All versions since 3.5 are affected. There is a bug for it: https://bugs.freedesktop.org/show_bug.cgi?id=55814 Nasty. I wonder why we don't have any unit test for that part of code? No-one wrote it yet ? :-) it's hardish to implemnt writer unit-tests even now ? ( at least my attempt at auto-testing the red-lining also ran into the ground inconclusively before I could write code to test the specific issue that was reported ;-) Another question is bug prioritization. That one is clearly a blocker for me, and belongs on ESC list of most critical bugs we have and should take care of. So - it should be a MAB - mark it a dependency of: https://bugs.freedesktop.org/showdependencytree.cgi?id=6hide_resolved=1 How can i create a unit test for that? Could you point me to some similar writer tests ? So - of course, fiddling with the raw core API is one way to construct such a document checkout: sw/qa/core/uwriter.cxx But apparently simple stuff such as inserting text into a document there is far less than obvious and clean (in my experience at least). I would expect that programmatic construction of such a document shouldn't be hard. ;-) personally I'd load the document to remove that problem; Miklos has a load of tests vs. the RTF filter here: sw/qa/extras/rtfimport.cxx that do UNO driven testing of document structure that are prolly well worth pillaging there :-) ATB, Michael. -- michael.me...@suse.com , Pseudo Engineer, itinerant idiot ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
Re: fdo#55814: unit test is missing
On Mon, 2013-03-11 at 13:28 +, Michael Meeks wrote: No-one wrote it yet ? :-) it's hardish to implemnt writer unit-tests even now ? ( at least my attempt at auto-testing the red-lining also ran into the ground inconclusively before I could write code to test the specific issue that was reported ;-) Thank you for that, Michael. It makes me feel less bad about my failed attempts. ;-) personally I'd load the document to remove that problem; Miklos has a load of tests vs. the RTF filter here: sw/qa/extras/rtfimport.cxx For the archive, I think that should be sw/qa/extras/rtfimport/rtfimport.cxx Terry. ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
fdo#55814: unit test is missing
Hi, i've been told yesterday that the usage of writer is severly restricted. All versions since 3.5 are affected. There is a bug for it: https://bugs.freedesktop.org/show_bug.cgi?id=55814 I wonder why we don't have any unit test for that part of code? Another question is bug prioritization. That one is clearly a blocker for me, and belongs on ESC list of most critical bugs we have and should take care of. May be we need a new category? How about Most servere impact on customers? Anyway, to reproduce that one a trivial document with only two character is needed: 0 a where 0 is a variable test with the initial value 0 a is a section that should be hidden if the condition is met: test EQ 1 Changing the value of variable from 0 to 1 overrides currently the condition to 0, leaving the section unhidden. How can i create a unit test for that? Could you point me to some similar writer tests? I would expect that programmatic construction of such a document shouldn't be hard. Then change the variable from 0 to 1 and check that the section is removed from the model, preserving the original condition. Thanks David ___ LibreOffice mailing list LibreOffice@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/libreoffice
