On Sat, Oct 24, 2020 at 8:14 PM Carl Marcum <cmar...@apache.org> wrote:
> Hi All, > > I've been testing builds with the automated BVT and FVT tests lately. > I have a few questions: > > 1. Is there anything documented about how much coverage these tests > provide vs.functionality? > > 2. Is there yet a place to list new cases it would good to add test for? > > I think there is a lot that could be done in this area to attract new > contributors if we had a place to work from. > Both in documenting and work on some flaky tests that I've run into. > > I'm willing to put some effort into this, both organizing and developing. > > Hi Carl Thank you for helping with the tests. I wrote a good email summarizing the different tests we have some years ago, please see https://lists.apache.org/thread.html/bb1851b82ba009d2cefdf5af9997099b6fdfb04bddac3753172f2698%401459253891%40%3Cdev.openoffice.apache.org%3E Some things changed since then, eg. the smoketest location moved. There are a few other emails too, search for them. I also did some test fixes to bvt/fvt and others at various times. This year I learned quite a lot about them. For example many tests fail because they expect features that the .DOC file format cannot provide (such as strikeout styles unique to .ODT), there were timeouts, registry modifications had to be enabled to fix a test, confirmation dialogs that hang the tests, a 2048 byte limit in FreeBSD's "ps" was causing a pid lookup to fail, java.util.Calendar was being used incorrectly, etc. I fixed some of these, but others remain. Understanding why the .DOC tests fail requires understanding the .DOC file format, so fixing those tests isn't easy. Look through the Git log, I wrote pretty descriptive commit messages. One thing I didn't mention in that email is the thousands of unused tests we have in main/qadevOOo, but they seem difficult to set up, and use a custom test framework, not JUnit. They have a complex architecture, with some code implementing UNO components and some code testing them. There might even be code missing for some of them. There's a mixture of Java and StarBasic tests there, with much duplication - were they first written in one language then semi-ported to another? Anyway I can't help much at the moment, but good luck and let us know how it goes. Damjan
