Alexander,
let me join Alexey Varlamov to discussion since it was him who asked
about presence of regression test for H2335, I believe he controls
regression tests state and progress and can resolved mentioned issues
pretty fast.
>> 1. there should be opportunity to build regression test w/o rebuilding
>> VM.
>
> It could be a usefull feature. Should go to TODO list.
I believe, it worth just get done because it is minor.
AFAIK one just need to define the ant target in
<trunk>/working_vm/build/make/build.xml
which will call to "pre-reg-test" target in
<trunk>/working_vm/build/make/targets/reg.test.xml
>> 2. README should say what way the regression test can be built.
>
> Sorry, it is not clear.
I mean something like:
BUILDING TEST #section name in README
To build your regression test you need to execute the
following commands:
cd <trunk>/working_vm/build
./build.bat pre-reg-test # or better name for target
> I tried to collect notes on infrastructure in [1] mail thread. You can
> separate your notes on "features desired" and "readme notes" and post
> them to the thread.
I believe, both notes are of the same kind. If regression test has a
"development feature" then it just should be described in README.
Should I post set of my simple notes to [1]?
>> For example, H2335 regression test requires classes root dir and
>> native library location are different. What way can I control this?
>
> It is done in such a way. README states it:
Right! I've missed this...
One more question to get answer from README:
6. what way the regression test may be excluded?
There are also set of questions for current status (state) of
regressions tests I got after I run these tests with command:
cd <trunk>/working_vm/build
./build.bat reg.test # builds & runs all regression tests
7. Regression test H1694 does not answer naming conversion. There is no
java source ending with Test in this directory. (I saw the comments says
this test does not pass anyway). Let's get this test fixed to answer
naming convention.
8. sometimes the tests run reports are "coupled" (no empty string and
string with "run-test" between tests run reports) - difficult to see in
log whether or not test is passed. E.g.:
run-test:
[junit] Running org....h0000.DirectByteBufferTest
[junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
[junit] Test org....h0000.DirectByteBufferTest FAILED
[junit] Running org.....h1800.ExecTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.27 sec
run-test:
[junit] Running org.....h2151.VMInit1Test
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.832sec
Can this be controlled? And if "yes", then it would bee nice to get
reports separated, please.
thanks
Vladimir Beliaev
Alexander Kleymenov wrote:
Hi Vladimir,
On 1/18/07, Vladimir Beliaev <[EMAIL PROTECTED]> wrote:
Hello,
While working on http://issues.apache.org/jira/browse/HARMONY-2335 I got
the note - regression test is required. Fine - I've start looking at
<trunk>/working_vm/src/test/regression/README.txt and got the following
points related to regression test development which would be nice to get
reflected:
1. there should be opportunity to build regression test w/o rebuilding
VM.
It could be a usefull feature. Should go to TODO list.
2. README should say what way the regression test can be built.
Sorry, it is not clear.
Do you want the words "java sources are compiled by java's compiller,
c by c's, cpp by cpp's, jasmin by jasmin's" to be there?
3. there should be opportunity to run single regression test.
4. README should say what way a single regression test can be run.
Another TODO for reg testing infrastructure.
I can file JIRA on this if required. If the questions are asked already
somewhere in html (site) documentation, then the link in this README is
just fine.
I tried to collect notes on infrastructure in [1] mail thread. You can
separate your notes on "features desired" and "readme notes" and post
them to the thread.
There is one more specific question:
5. would be helpful to understand (from README) the layout of built
regression tests.
For example, H2335 regression test requires classes root dir and
native library location are different. What way can I control this?
It is done in such a way. README states it:
${reg.test.class.path} - the path to where java and
java assembler sources were compiled
${reg.test.2launch.native.path} - the path to where native
sources were compiled
Still this question is of the low priority. To be honest, the simplest
regression test for H2335 does not require a test native part. And
probably such type of regression test is really rare.
Thanks
Vladimir Beliaev
Thank you,
Alexander
[1] [drlvm][testing] Regression Testing Infrastructure proposal
