+1 for test data and test script in a separated tree. Test documents should never be distributed together with product code. Only the sample documents in tutorials should. Another advantage for a separated QE tree is that a volunteer can download any AOO build and run the same test suite from the QE tree again and again, a easy way of regression and even automation. A complex situation maybe the UT by developer that calls internal functions. Sometimes developers like to write UT code together with the product code. But will a sample document be used in UT?
- Simon 2012/8/30 Dave Fisher <[email protected]> > > On Aug 29, 2012, at 11:24 AM, Kay Schenk wrote: > > > > > > > On 08/29/2012 10:51 AM, Rob Weir wrote: > >> On Wed, Aug 29, 2012 at 11:29 AM, Andre Fischer <[email protected]> > wrote: > >>> On 29.08.2012 16:02, Rob Weir wrote: > >>>> > >>>> On Wed, Aug 29, 2012 at 9:52 AM, Andre Fischer <[email protected]> > wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> I just saw that we have now two new binary files in the test/ module. > >>>>> > >>>>> main/test/testgui/data/svt/complex.ods has a size of 9 424 385 Bytes > and > >>>>> main/test/testgui/data/svt/complex.odt has a size of 27 175 936 > Bytes. > >>>>> > >>>>> I wonder if SVN is really the best place for files that large. > >>>>> > >>>>> I also don't think that these files should be part of the source > release. > >>>>> But what else would have to be removed that depends on these files? > >>>>> > >>>>> Any thoughts? > >>>>> > >>>> > >>>> Something to keep in mind is that we'll probably end up with a large > >>>> number of test documents, 200+ MB. Not all of them will be large. > >>>> But if we want to have good test coverage then we'll need test > >>>> documents to cover all areas, for ODF, MS Binaries and OOXML. So this > >>>> will grow, over time, to a large test set. > >>>> > >>>> This leads to four questions: > >>>> > >>>> 1) Should we be testing large/complex documents? > >>>> > >>>> I think the answer is "yes". > >>> > >>> > >>> Agreed. > >>> > >>> > >>>> > >>>> 2) Should such test documents be in SVN? > >>>> > >>>> I think they should. > >>> > >>> > >>> Agreed. > >>> > >>> > >>>> > >>>> 3) Should these documents be in the same source tree with the rest of > >>>> the code that is downloaded by default for a build? > >>>> > >>>> Maybe not. Unless they are needed for a smoke test that should be run > >>>> by every developer. But if not, maybe they should be stored in its > >>>> own tree, like ooo/test/trunk or something like that. > >>>> > >>>> 4) Should these documents be included in the source distribution? > >>>> > >>>> Probably depends on the answer to question 3. Maybe, maybe not. Or > >>>> maybe we have a separate source distribution artifact only for > >>>> test-related files? > >>> > >>> > >>> > >>> My personal opinion is no. I believe that the use case for > downloading and > >>> building the source release is different from the use case for cloning > the > >>> SVN repository. I would expect the source release to be used for > building > >>> AOO, maybe do a simple test to verify that building was successful, > and then > >>> delete the source code. > >>> > >> > >> OK. That is a useful distinction: building versus developing. > > I think Building versus QA - both are developing. > > >> > >>> If I want to start developing then I would choose SVN. Complex tests > would > >>> help me to avoid new errors. > >>> > >>> I don't see the need for complex tests when my goal is not developing. > Lack > >>> of trust that we did not run the tests on the released code? > >>> > >>> But, of course I can be wrong (and often are :-). > >>> > >> > >> If we follow that logic, then we might still store the test data and > >> test code in SVN, but in its own tree, e.g., /ooo/test/trunk. > >> > >> This also preserves the option of us having a "test source" artifact > >> in a future release, if we wanted. > >> > >> -Rob > > > > +1, this seems like a good compromise > > > > I don' think the "test" cases should be in the same tree as source. > > Agreed. > > > > > No use overloading developers who simply want to build and make > modifications. > > Source is required as an Open Source release. (we should all understand > that.) > > QA / test is "optional" but quite important. It should be separate and we > can include a "QA" package as one of our convenience binaries during a > release. > > Regards, > Dave > > > > > > >> > >>> -Andre > >>> > >>>> > >>>> -Rob > >>>> > >>>> > >>>>> Regards, > >>>>> Andre > >>> > >>> > > > > -- > > ------------------------------------------------------------------------ > > MzK > > > > "As a child my family's menu consisted of two choices: > > take it or leave it. " > > -- Buddy Hackett > >
