Re: [Libreoffice-qa] Issues with 3.5.0 - ready for release ...
2012/2/8 Caolán McNamara caol...@redhat.com: On Tue, 2012-02-07 at 22:05 +0100, Markus Mohrhard wrote: But keep in mind that this is nothing that will be fast. I needed around one week to check a bit more than 4000 files My times are dramatically faster than this for .doc files in writer at least. I can load 4000 .doc files *under valgrind* overnight in about 10 hours. So apparently mileage varies quite a bit depending on hardware and file format and debugging level. That only works if you have no crashs or looping documents. Especially looping is a big problem in calc. Then if we want to be fast using a debug/dbgutil build is the wrong way but then we loose the advantages of gcc's safe iterators. So I think 4000 known good documents can be easily tested in one day or even faster with a decent machine but taking 4000 random documents from bugzilla needs some manual interaction and therefore will need more time. As mentionend in the last mail, I think we could speed that up by copying the test code and the makefile several times and run the test in parallel. That way we could use more cores and would be more reliable against crashs. ( we should of course not commit this stuff then) Markus ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Issues with 3.5.0 - ready for release ...
Cor Nouws píše v Út 07. 02. 2012 v 01:08 +0100: Hi, As promised to Thorsten on Sunday, I've been looking at 3.5.0 and issues. I looked at issues that I've seen, that are at 37361 and at the 44 issues filed during our second BugHuntSession (little delay :-\ ) Added some comments, added some to bug 37361 etc. Great work. I hope that we will fix most of them soon. So... as for 3.5.0rc3: In the issues I know, I do not see issues that should be blocker (maybe Bug 45584 - presentation view defective, but I know others looked at that one.) We have workaround for bug 45584. I have mentioned it at https://wiki.documentfoundation.org/ReleaseNotes/3.5#Most_annoying_bugs. So, it need not block the release. However, I have no idea if the issues from the last weeks have been spotted / judged consequently and with enough detail (looking at what happened with the issues from January 21 and 22, I have to doubt seriously alas.) So, though the overall impression is OK (as is with builds from the master) as long that checking issues from the last weeks is not clear, I would not bet on it. I have the same feeling. Well, we have had lack of bug-triage people since the beginning and we were newer able to review all bug reports in time. I am afraid that there is no short-time solution, so we need to live with what we have now and believe that the most critical bugs were correctly propagated. Thanks a lot for your work and nice summary. Best Regards, Petr ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Issues with 3.5.0 - ready for release ...
On Tue, 7 Feb 2012, Cor Nouws wrote: But the quick look I had at the rough list in the train this morning, showed crashes and such but as far as I could judge with special documents. Great issues to solve for the next release(s) ... I can demonstrate issues with some filters during conversions, and when generating (incorrect) Flat ODF have had crashes too (reported). Do we have tests doing automated QA (conversions) to test export filters and are we using fuzzing techniques to test our import filters ? The conversion problems have not been reported yet, but I am sure if we throw more example documents (maybe from Google ?) at it we'll find a lot more problems in an automated fashion. I remember at the LibreConference that Gabriel Monnerat of cloudooo fame was reporting the same problems and reported 10% conversion errors (LibO crashes and filter errors) on a set of 100.000 internet documents. Gabriel implemented various techniques to make his setup more resillient (killing run-away and memory-leaked instances, restarting on crashes and filter instabilities) bringing this number down to 1.1%. Which proves that the majority of these issues are not necessarily reproducable. I am interested in this, both because it helps making unoconv a better tool, but also because it will learn me how to report such issues and who to address them to. -- -- dag wieers, d...@wieers.com, http://dag.wieers.com/ -- dagit linux solutions, i...@dagit.net, http://dagit.net/ [Any errors in spelling, tact or fact are transmission errors] ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
Re: [Libreoffice-qa] Issues with 3.5.0 - ready for release ...
Hey, 2012/2/7 Caolán McNamara caol...@redhat.com: On Tue, 2012-02-07 at 20:32 +0100, Dag Wieers wrote: Do we have tests doing automated QA (conversions) to test export filters and are we using fuzzing techniques to test our import filters ? I've personally been to-date more interested in automated import filter tests, typically various binary file formats. You can find the existing tests in e.g. sw/qa/core/filters-test.cxx where the minimal always-tested test-cases are in e.g. sw/qa/core/data/*/ The idea of these is that you use something like bin/get-bugzilla-attachments-by-mimetype to slurp down a pile of extra documents and shove them into sw/qa/core/data/*/indeterminate and just run make in sw to see if any of them triggers a crash (or a valgrind warning when export VALGRIND=memcheck make is used). Same basic see if file format parser crashes idea implemented for other modules spread around the place, e.g. sc, sd, svtools and stuff like .xls, .ppt .wmf and so forth. With some modifications this will also work for odt. Odt is just a special case because our internal code distinguishs odt and all other filters and will fail if we don't provide all necessary information for odt ( see sc/qa/unit/filters-test.cxx for the needed adjustments ). The more difficult part is then to find all necessary component files and add them otherwise around 80% of the complex documents will crash. I can port the sc adjustments to sw but someone else would need to add the component files. calc has extended things a bit further for some extra tests to ensure that selected test documents have the expected calculation results and Markus demoed at FOSDEM the in-progress stuff for sd to ensure that selected test documents render as expected. We also used Caolan's great script and concept to test all ods, xls and xlsx files from OOo, fdo and redhat bugzilla for crashs. Testing these files can only search for crashs not that documents are imported because only around 60 to 70% of them can be imported at all. A lot of them are either totally broken or contain passwords. I added a bit of code to sc's filters-tesst and planned to improve that code for the next release but if you plan to use this concept for writer or impress I can integrate the remaining points from todo list. But keep in mind that this is nothing that will be fast. I needed around one week to check a bit more than 4000 files. Anyway it would be great if you could help out doing the same for writer. This would mainly involve finding the component files and running the test, I would port all my changes to sw if needed. Regards, Markus ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
[Libreoffice-qa] Issues with 3.5.0 - ready for release ...
Hi, As promised to Thorsten on Sunday, I've been looking at 3.5.0 and issues. I looked at issues that I've seen, that are at 37361 and at the 44 issues filed during our second BugHuntSession (little delay :-\ ) Added some comments, added some to bug 37361 etc. Quite some have already been picked up. Thanks a lot! = = = Some remarks about issues that, as far as I know, don't have attention as for now: - the print issues from: https://bugs.freedesktop.org/show_bug.cgi?id=37361#c173 - captions/cross refs https://bugs.freedesktop.org/show_bug.cgi?id=37361#c172 - There there are some issues around documents with field: (attention to at least explain / value these, would be great.) - Bug 45341 - Saving document with a input list field as a .doc file makes the input list field read-only - Bug 41798 - FORMATTING: Opening a specific .doc file (frames, fields) gives wrong formatting Regression in Libreoffice 3.3/4 = = = So... as for 3.5.0rc3: In the issues I know, I do not see issues that should be blocker (maybe Bug 45584 - presentation view defective, but I know others looked at that one.) However, I have no idea if the issues from the last weeks have been spotted / judged consequently and with enough detail (looking at what happened with the issues from January 21 and 22, I have to doubt seriously alas.) So, though the overall impression is OK (as is with builds from the master) as long that checking issues from the last weeks is not clear, I would not bet on it. = = = Of course there still are quite some issues, and more container problems (as the paper size issue that I've added to 37361). (Extra hands to create as simple as possible overviews welcome.) Interested in the findings and opinions of others! Cheers, -- - Cor - http://nl.libreoffice.org ___ List Name: Libreoffice-qa mailing list Mail address: Libreoffice-qa@lists.freedesktop.org Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/ Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
