Re: [classlib][testing] Showing test failures

2006-11-16 Thread Tim Ellison
Alexey Varlamov wrote: 2006/11/16, Tim Ellison [EMAIL PROTECTED]: One confusing aspect is that the classlib ant build fails if you run the tests 'globally', but passes if you run the tests in a single module. Yes, this was a nasty surprise for me when I saw exactly this sutiation few days

Re: [classlib][testing] Showing test failures (Re: [drlvm] New regression: java.lang.ClassGenericsTest4)

2006-11-15 Thread Alexey Varlamov
2006/11/16, Tim Ellison [EMAIL PROTECTED]: Rana Dasgupta wrote: I think that a problem with the junit tests is that some failures spit out to the console, but show up in the test run results as passed. I find this very confusing. So unless you are watching all the time, you can miss them.

Re: [classlib][testing] need to run classlib tests over DRLVM

2006-11-09 Thread Alexei Zakharov
So the best practice to run classlib tests on DRLVM is now: ant -Dhy.test.mode=perTest [-Dbuild.module=blah-blah] test ? Regards, 2006/11/9, Alexey Petrenko [EMAIL PROTECTED]: I'll take a look. SY, Alexey 2006/11/9, Vladimir Ivanov [EMAIL PROTECTED]: Hello committers, could somebody take

Re: [classlib][testing] need to run classlib tests over DRLVM

2006-11-09 Thread Vladimir Ivanov
On 11/9/06, Alexei Zakharov [EMAIL PROTECTED] wrote: So the best practice to run classlib tests on DRLVM is now: ant -Dhy.test.mode=perTest [-Dbuild.module=blah-blah] test ? Actually, it depends on your choice :) The mode -Dhy.test.forkmode=perTestwill be useful: - to identify crashed/

Re: [classlib][testing] need to run classlib tests over DRLVM

2006-11-09 Thread Alexei Zakharov
Well, my scenario is to make sure that my fixes in beans don't introduce new hangs/failures on DRLVM. This means running AWT and SWING tests on top of DRLVM that was a painful process before. Regards, 2006/11/9, Vladimir Ivanov [EMAIL PROTECTED]: On 11/9/06, Alexei Zakharov [EMAIL PROTECTED]

Re: [classlib][testing] need to run classlib tests over DRLVM

2006-11-08 Thread Alexey Petrenko
I'll take a look. SY, Alexey 2006/11/9, Vladimir Ivanov [EMAIL PROTECTED]: Hello committers, could somebody take care about *HARMONY-2107*http://issues.apache.org/jira/browse/HARMONY-2107? It is trivial to fix issue allows easily identify tests that leads to vm hang up/crash in the process

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-08 Thread Vladimir Ivanov
On 10/6/06, Mark Hindess [EMAIL PROTECTED] wrote: Agreed on all three. Do we have a japi script? I have one but it's a little specific to the wrapper we use for the builds that report to the -commits list. But I can provide it if it will help. If this script requires some updates I am

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-06 Thread Vladimir Ivanov
While nobody objects :) the right place for coverage scripts is 'buildtest' module. Seems, that this module should be a little bit reorder: new top level directories should be created: - 'cc' - for cruise control script (and move current stuff to this dir); - 'coverage' - for coverage scripts

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-06 Thread Geir Magnusson Jr.
Vladimir Ivanov wrote: While nobody objects :) the right place for coverage scripts is 'buildtest' module. Seems, that this module should be a little bit reorder: new top level directories should be created: - 'cc' - for cruise control script (and move current stuff to this dir); -

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-06 Thread Mark Hindess
On 6 October 2006 at 9:41, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: Vladimir Ivanov wrote: While nobody objects :) the right place for coverage scripts is 'buildtest' module. Seems, that this module should be a little bit reorder: new top level directories should be created:

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-06 Thread Geir Magnusson Jr.
Thanks - maybe someone can massage that to fit with what we're building... Mark Hindess wrote: On 6 October 2006 at 9:41, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: Vladimir Ivanov wrote: While nobody objects :) the right place for coverage scripts is 'buildtest' module. Seems, that this

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-03 Thread Vladimir Ivanov
I have one more question about coverage: should it be the part of the BT infrastructure or integrated to the current classlib build system? From my point of view it should be a part of BTI while it is rarely used functionality and no needs to waste regular build by this data (it does not

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-03 Thread Mark Hindess
On 3 October 2006 at 21:08, Vladimir Ivanov [EMAIL PROTECTED] wrote: I have one more question about coverage: should it be the part of the BT infrastructure or integrated to the current classlib build system? From my point of view it should be a part of BTI while it is rarely used

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-10-03 Thread Vladimir Ivanov
On 10/3/06, Mark Hindess [EMAIL PROTECTED] wrote: Regarding the 'run japi' script what are you planning to do here? The IBM Build/test builds also run japi (on linux only since we get enough information using one platform and linux is easier). We might as well share ant code. Also note that

Re: [classlib] [testing] How to file JIRA issues for impl JUnit tests?

2006-10-03 Thread Tim Ellison
Submit a JIRA with the failure and expected result (or am I missing something?) Regards, Tim Anton Luht wrote: Hello, It's clear how to file issues for public API - write a test with differend behaviour on RI and Harmony. It's not clear how to write tests or report problems when JUnit impl

Re: [classlib] [testing] How to file JIRA issues for impl JUnit tests?

2006-10-03 Thread Stepan Mishura
On 10/3/06, Anton Luht wrote: Hello, It's clear how to file issues for public API - write a test with differend behaviour on RI and Harmony. It's not clear how to write tests or report problems when JUnit impl test fail. Hi Anton, At minimum you can just file a JIRA describing a failure

Re: [classlib][testing] give DRLVM a chance!

2006-09-14 Thread Oliver Deakin
Does this mean we could no longer use the IBM VME to run the tests? Regards, Oliver Mikhail Loenko wrote: There are two options supported by Support_Exec: IBM and Sun I'd like to apply this change (remove IBM branch and leave the Sun branch for all VMs): Index:

Re: [classlib][testing] give DRLVM a chance!

2006-09-14 Thread Geir Magnusson Jr.
I don't understand either. (And we'd certainly want to always be able to choose J9 or whatever else shows up too...) geir Oliver Deakin wrote: Does this mean we could no longer use the IBM VME to run the tests? Regards, Oliver Mikhail Loenko wrote: There are two options supported by

Re: [classlib][testing] give DRLVM a chance!

2006-09-14 Thread Mikhail Loenko
2006/9/14, Geir Magnusson Jr. [EMAIL PROTECTED]: I don't understand either. Neither do I :) It's not clear for me why different command lines are generated for Sun and IBM, given that the tests pass on IBM if Sun's command line is passed there What I suggest is to start all VMs with the same

Re: [classlib][testing] give DRLVM a chance!

2006-09-14 Thread Geir Magnusson Jr.
Ah - yes - lets not call it sun then if it's universal. Mikhail Loenko wrote: 2006/9/14, Geir Magnusson Jr. [EMAIL PROTECTED]: I don't understand either. Neither do I :) It's not clear for me why different command lines are generated for Sun and IBM, given that the tests pass on IBM if

Re: [classlib][testing] give DRLVM a chance!

2006-09-14 Thread Mikhail Loenko
No sorry for unclear wording, the tests passed on IBM VME 2006/9/14, Oliver Deakin [EMAIL PROTECTED]: Does this mean we could no longer use the IBM VME to run the tests? Regards, Oliver Mikhail Loenko wrote: There are two options supported by Support_Exec: IBM and Sun I'd like to apply

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-09-05 Thread Vladimir Ivanov
The coverage information was updated on the wiki to current state. thanks, Vladimir

Re: [classlib] [testing] crashes in test execution

2006-08-30 Thread Alexey Petrenko
Yep, I had some popups while run tests on drlvm. But I do not remember exact message. SY, Alexey 2006/8/30, Anton Luht [EMAIL PROTECTED]: Hello, I've tried to run 'ant test' in classlib on a recent build hand-made from SVN and came across several problems. First problem was that a popup

Re: [classlib] [testing] crashes in test execution

2006-08-30 Thread Alexey Varlamov
2006/8/30, Anton Luht [EMAIL PROTECTED]: Hello, I've tried to run 'ant test' in classlib on a recent build hand-made from SVN and came across several problems. First problem was that a popup window with assertion appeared - it was easy to solve it - see HARMONY-1340 . The second one is more

RE: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-08-21 Thread Ivanov, Alexey A
Vladimir, I am working to minimize hang-up of swing tests. Regards, -- Alexey A. Ivanov Intel Middleware Product Division -Original Message- From: Vladimir Ivanov [mailto:[EMAIL PROTECTED] Sent: Friday, August 18, 2006 4:28 PM To: harmony-dev@incubator.apache.org Subject: Re: [classlib

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-08-20 Thread Vladimir Ivanov
Coverage for swing module was added. thanks, Vladimir On 8/18/06, Tim Ellison [EMAIL PROTECTED] wrote: Thanks for updating the numbers Vladimir. Overall I think they look quite respectable. There are a few places where it looks like we could use more test coverage. It's a great way to

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-08-18 Thread Vladimir Ivanov
The coverage information was updated on the wiki. Also, coverage for accessibility, awt, instrument and sound modules was added. I have a problem to calculate coverage for swing module (test run hang up) so issue 564 will be updated later. thanks, Vladimir On 6/20/06, Paulex Yang [EMAIL

Re: [classlib] [testing] Coverage (was Re: 37% of total test execution time is spent in a single test)

2006-08-18 Thread Tim Ellison
Thanks for updating the numbers Vladimir. Overall I think they look quite respectable. There are a few places where it looks like we could use more test coverage. It's a great way to contribute to Harmony (he said, hinting to the lurkers g). Pick a favourite package, kick the wheels of Harmony

Re: [classlib][testing] excluding the failed tests

2006-08-17 Thread Andrew Zhang
Hi folks, I'd like to investigate tests/api/java/net/DatagramSocketTest.java and tests/api/java/net/DatagramSocketTest.java in luni module. I have updated the wiki page(http://wiki.apache.org/harmony/Excluded_tests). I'll also plan to study other excluded tests in luni module when I finish these

Re: [classlib] Testing conventions - a proposal

2006-07-26 Thread George Harley
Alexei Zakharov wrote: Hi George, Sorry for the late reply. Hi Alexei, Not a problem. Especially when my reply to you is even later (sorry). It looks like you are using an os.any group for those test methods (the majority) which may be run anywhere. That's a different approach to what I

Re: [classlib] Testing conventions - a proposal

2006-07-26 Thread Paulex Yang
FYI, I haven't studied it yet, but seems new TestNG 5 support ant task with JVM parameter[1] [1] http://www.theserverside.com/news/thread.tss?thread_id=41479 Richard Liang wrote: Just thinking about using TestNG to execute Harmony test cases. :-) Look at our build.xml (e.g.,

Re: [classlib] Testing conventions - a proposal

2006-07-25 Thread Stepan Mishura
On 7/20/06, George Harley wrote: SNIP! Anyway, the point I guess that I am trying to make here is that it is possible in TestNG to select the methods to test dynamically using a little bit of scripting that (a) gives us a lot more power than the include/exclude technique and (b) will work the

Re: [classlib] Testing conventions - a proposal

2006-07-24 Thread Alexei Zakharov
Hi George, Sorry for the late reply. It looks like you are using an os.any group for those test methods (the majority) which may be run anywhere. That's a different approach to what I have been doing. I have been thinking more along the lines of avoiding the creation of groups that cover the

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread Alexei Zakharov
Hi George, Wow, they are fast guys! Thanks for the link. Do you know when do they plan to release 5.0 officially? Regards, 2006/7/19, George Harley [EMAIL PROTECTED]: Hi Alexei, I just downloaded the latest working build of TestNG 5.0 [1] and support for the jvm attribute is in there. This

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread George Harley
Richard Liang wrote: George Harley wrote: Richard Liang wrote: George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread George Harley
Alexei Zakharov wrote: Hi George, Wow, they are fast guys! Thanks for the link. Do you know when do they plan to release 5.0 officially? Regards, Hi Alexei, Actually, I just saw this announcement in my news reader about 15 minutes ago ... http://beust.com/weblog/archives/000400.html

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread Alexei Zakharov
George, I remember my past experience with BeanShell - I was trying to create the custom BeanShell task for ant 1.6.1. I can't say I haven't succeeded. But I remember this as a rather unpleasant experience. At that time BeanShell appeared to me as a not very well tested framework. Please don't

Re: [classlib] Testing conventions - a proposal

2006-07-20 Thread George Harley
Alexei Zakharov wrote: George, I remember my past experience with BeanShell - I was trying to create the custom BeanShell task for ant 1.6.1. I can't say I haven't succeeded. But I remember this as a rather unpleasant experience. At that time BeanShell appeared to me as a not very well tested

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the possibilities that the

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
According to TestNG Ant Task [1], it seems that the TestNG Ant task does not support to fork a new JVM, that is, we must launch ant using Harmony itself. Any comments? Thanks a lot. [1]http://testng.org/doc/ant.html Best regards, Richard George Harley wrote: Andrew Zhang wrote: On 7/18/06,

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Alexei Zakharov
Hmm, do we have problems with launching ant? I thought we have problems with launching TestNG. Just checked - running tests for beans on j9+fresh classlib works fine. I.e. ant -Dbuild.module=beans -Dbuild.compiler=org.eclipse.jdt.core.JDTCompilerAdapter test 2006/7/19, Richard Liang [EMAIL

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
Just thinking about using TestNG to execute Harmony test cases. :-) Look at our build.xml (e.g., modules/luni/build.xml), you will see something like: .. junit fork=yes forkmode=once printsummary=withOutAndErr

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Richard Liang wrote: George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Hi Richard, Actually the Ant task always runs the tests in a forked VM. At present, however, the task does not support specifying the forked VM (i.e. there is no equivalent to the JUnit Ant task's jvm attribute). This matter has already been raised with the TestNG folks who seem happy to

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Alexei Zakharov
Probably my previous message was not clear enough. Why can't we just invoke everything including ant on top of Harmony for now? At least I was able to build and run test-14 examples from TestNG 4.7 distribution solely on top of j9 + our classlib today. C:\Java\testng-4.7\test-14set

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Hi Alexei, It's encouraging to hear that (Ant + TestNG + sample tests) all worked fine together on Harmony. In answer to your question I suppose that the ability to fork the tests in a separate VM means that we do not run the risk of possible bugs in Harmony affecting the test harness and

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Alexei Zakharov
Hi George, Agree, we may experience problems in case of VM hang or crash. I suggest this only as a temporary solution. BTW, the fact that TestNG ant task still doesn't have such attributes looks like a sign for me - TestNG can be still immature in some aspects. Still comparing TestNG and JUnit.

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread George Harley
Hi Alexei, I just downloaded the latest working build of TestNG 5.0 [1] and support for the jvm attribute is in there. This is not the official release build. Best regards, George [1] http://testng.org/testng-5.0.zip Alexei Zakharov wrote: Hi George, Agree, we may experience problems in

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
George Harley wrote: Hi Alexei, It's encouraging to hear that (Ant + TestNG + sample tests) all worked fine together on Harmony. In answer to your question I suppose that the ability to fork the tests in a separate VM means that we do not run the risk of possible bugs in Harmony affecting

Re: [classlib] Testing conventions - a proposal

2006-07-19 Thread Richard Liang
George Harley wrote: Richard Liang wrote: George Harley wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Oliver Deakin
George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: * type.impl -- tests that are specific to Harmony So tests are

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: * type.impl -- tests that are specific to

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Oliver Deakin
George Harley wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: * type.impl -- tests

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Andrew Zhang
On 7/18/06, George Harley [EMAIL PROTECTED] wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Alexei Zakharov
Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are platform-specific. I volunteer to do this trial for beans module. I'm not sure that

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Andrew Zhang
On 7/18/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are platform-specific. I

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Oliver Deakin wrote: George Harley wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of my initial thoughts: *

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Andrew Zhang wrote: On 7/18/06, George Harley [EMAIL PROTECTED] wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread Oliver Deakin
George Harley wrote: Oliver Deakin wrote: George Harley wrote: Oliver Deakin wrote: George Harley wrote: SNIP! Here the annotation on MyTestClass applies to all of its test methods. So what are the well-known TestNG groups that we could define for use inside Harmony ? Here are some of

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Andrew Zhang wrote: On 7/18/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are

Re: [classlib] Testing conventions - a proposal

2006-07-18 Thread George Harley
Alexei Zakharov wrote: Hi, George wrote: Thanks, but I don't see it as final yet really. It would be great to prove the worth of this by doing a trial on one of the existing modules, ideally something that contains tests that are platform-specific. I volunteer to do this trial for beans

Re: [classlib] Testing conventions - a proposal

2006-07-17 Thread George Harley
Andrew Zhang wrote: On 7/14/06, George Harley [EMAIL PROTECTED] wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the

Re: [classlib] Testing conventions - a proposal

2006-07-14 Thread George Harley
Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the possibilities that the TestNG @Test annotation

Re: [classlib][testing] excluding the failed tests

2006-07-14 Thread Geir Magnusson Jr
Vladimir Ivanov wrote: New page http://wiki.apache.org/harmony/Excluded_tests was added to WIKI (refered from http://wiki.apache.org/harmony/ClassLibrary). It would be good if before test investigation one would specify 'in progress, Name' near module name, showing it is under investigation

Re: [classlib][testing] excluding the failed tests

2006-07-14 Thread Richard Liang
Great job. Vladimir ;-) Vladimir Ivanov wrote: New page http://wiki.apache.org/harmony/Excluded_tests was added to WIKI (refered from http://wiki.apache.org/harmony/ClassLibrary). It would be good if before test investigation one would specify 'in progress, Name' near module name, showing it

Re: [classlib][testing] excluding the failed tests

2006-07-14 Thread Alexei Zakharov
Yes Vladimir, nice job! I have updated the data for beans module. Since the reason of failures for the most of excluded test is not known yet I just put their names there without any comment why they were excluded. Thanks, 2006/7/14, Richard Liang [EMAIL PROTECTED]: Great job. Vladimir ;-)

Re: [classlib] Testing conventions - a proposal

2006-07-14 Thread Andrew Zhang
On 7/14/06, George Harley [EMAIL PROTECTED] wrote: Hi, If annotations were to be used to help us categorise tests in order to simplify the definition of test configurations - what's included and excluded etc - then a core set of annotations would need to be agreed by the project. Consider the

Re: [classlib][testing] excluding the failed tests

2006-07-13 Thread Richard Liang
Vladimir Ivanov wrote: On 7/7/06, Tim Ellison [EMAIL PROTECTED] wrote: ... Currently I'm looking on the excluded TestCases and it requires more time than I expected. I'll prepare a report/summary about excluded TestCases at the end of this process. Hello Vladimir, How about the progress

Re: [classlib][testing] excluding the failed tests

2006-07-13 Thread Vladimir Ivanov
New page http://wiki.apache.org/harmony/Excluded_tests was added to WIKI (refered from http://wiki.apache.org/harmony/ClassLibrary). It would be good if before test investigation one would specify 'in progress, Name' near module name, showing it is under investigation being done by Name. Thanks,

Re: [classlib][testing] excluding the failed tests

2006-07-12 Thread Vladimir Ivanov
On 7/7/06, Tim Ellison [EMAIL PROTECTED] wrote: ... Currently I'm looking on the excluded TestCases and it requires more time than I expected. I'll prepare a report/summary about excluded TestCases at the end of this process. Thanks, Vladimir On 7/7/06, Tim Ellison [EMAIL PROTECTED] wrote:

Re: Re: [classlib] Testing conventions - a proposal

2006-07-11 Thread Alexei Zakharov
Hi Alex, It's a pitty what you didn't find common sense in my post. Probably I was not clear enough. My key points are: 1. JUnit is much like a standard of unit testing today 2. We are using JUnit already, have thousands of tests 3. May be I was not correct about bugs in TestNG - I assume that

Re: [classlib][testing] excluding the failed tests

2006-07-10 Thread Alexei Zakharov
Hi, If there are really useful tests that are being unnecessarily excluded by being in the same *Test class, then you may want to consider moving the failing tests into SecureRandom3Test and excluding that -- but by the sound of it all SecureRandom tests will be failing. I think it's a nice

Re: [classlib][testing] excluding the failed tests

2006-07-10 Thread George Harley
Alexei Zakharov wrote: Hi, If there are really useful tests that are being unnecessarily excluded by being in the same *Test class, then you may want to consider moving the failing tests into SecureRandom3Test and excluding that -- but by the sound of it all SecureRandom tests will be failing.

Re: [classlib][testing] excluding the failed tests

2006-07-10 Thread Tim Ellison
Alexei Zakharov wrote: Hi, If there are really useful tests that are being unnecessarily excluded by being in the same *Test class, then you may want to consider moving the failing tests into SecureRandom3Test and excluding that -- but by the sound of it all SecureRandom tests will be

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Oliver Deakin
Geir Magnusson Jr wrote: Oliver Deakin wrote: George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class

Re: Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Alexei Zakharov
Actually, there's a very valid benefit for using TestNG markers (= annotations/JavaDoc) for grouping tests; the directory structure is a tree, whereas the markers can form any slice of tests, and the sets Concerning TestNG vs JUnit. I just like to pay your attention on the fact what it is

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread George Harley
Alexei Zakharov wrote: Actually, there's a very valid benefit for using TestNG markers (= annotations/JavaDoc) for grouping tests; the directory structure is a tree, whereas the markers can form any slice of tests, and the sets Concerning TestNG vs JUnit. I just like to pay your attention on

Re: [classlib][testing] excluding the failed tests

2006-07-10 Thread Alexei Zakharov
Thanks George Tim, I was out during last week and today was reading threads from oldest to the newest. :) I agree, general solution using TestSuites or even TestNG is better than my temporary one. However, defining a general approach can take a long period of time. Anyway, let's move our

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Geir Magnusson Jr
Oliver Deakin wrote: Geir Magnusson Jr wrote: Oliver Deakin wrote: George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Geir Magnusson Jr
George Harley wrote: Alexei Zakharov wrote: Actually, there's a very valid benefit for using TestNG markers (= annotations/JavaDoc) for grouping tests; the directory structure is a tree, whereas the markers can form any slice of tests, and the sets Concerning TestNG vs JUnit. I just like

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Alexei Zakharov
Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact statement was referring to TestNG as probably unstable rather than simply unstable. ;) This statement was

Re: Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread Alex Blewitt
On 10/07/06, Alexei Zakharov [EMAIL PROTECTED] wrote: Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact statement was referring to TestNG as probably

Re: [classlib] Testing conventions - a proposal

2006-07-10 Thread George Harley
Alexei Zakharov wrote: Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact statement was referring to TestNG as probably unstable rather than simply unstable.

[classlib] TestNG v. JUnit (was: RE: [classlib] Testing conventions - a proposal)

2006-07-10 Thread Nathan Beyer
@incubator.apache.org Subject: Re: [classlib] Testing conventions - a proposal Alexei Zakharov wrote: Hi George, For the purposes of this discussion it would be fascinating to find out why you refer to TestNG as being an unstable test harness. What is that statement based on ? My exact

Re: [classlib] Testing conventions - a proposal

2006-07-09 Thread Richard Liang
Richard Liang wrote: Paulex Yang wrote: Richard Liang wrote: Hello All, After read through the document recommended by Alex, I think TestNG can really meet our requirement. It provides much flexibility for test configuration. ;-) If we decide to transfer to TestNG, we shall: 1.

Re: [classlib] Testing conventions - a proposal

2006-07-08 Thread Geir Magnusson Jr
Nathan Beyer wrote: -Original Message- From: Geir Magnusson Jr [mailto:[EMAIL PROTECTED] This is a fun thread. I plan to read it from end to end later today and comment. Initial thoughts are that I've been wanting to use TestNG for months (hence my resistance to any JUnit

Re: Re: [classlib] Testing conventions - a proposal

2006-07-08 Thread Alex Blewitt
On 08/07/06, Geir Magnusson Jr [EMAIL PROTECTED] wrote: So while I like the annotations, and expect we can use them effectively, I have an instinctive skepticism of annotations right now because in general (in general in Java), I'm not convinced we've used them enough to grok good design

Re: [classlib] Testing conventions - a proposal

2006-07-08 Thread Geir Magnusson Jr
Alex Blewitt wrote: On 08/07/06, Geir Magnusson Jr [EMAIL PROTECTED] wrote: So while I like the annotations, and expect we can use them effectively, I have an instinctive skepticism of annotations right now because in general (in general in Java), I'm not convinced we've used them enough

Re: [classlib] Testing conventions - a proposal

2006-07-07 Thread Mark Hindess
On 6 July 2006 at 21:02, Nathan Beyer [EMAIL PROTECTED] wrote: I think Tim has a valid point, or at least the point I'm inferring seems valid: the testing technology is not the real issue. This problem can be solved by either JUnit or TestNG. More specifically, this problem can be solved

Re: [classlib] Testing conventions - a proposal

2006-07-07 Thread Geir Magnusson Jr
Oliver Deakin wrote: George Harley wrote: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library test layouts. The

RE: [classlib] Testing conventions - a proposal

2006-07-07 Thread Nathan Beyer
-Original Message- From: Geir Magnusson Jr [mailto:[EMAIL PROTECTED] Maven layout? We were doing that layout in Jakarta projects long before maven And I would guess the Maven designers would agree. Much of their documentation talks about how the conventions inferred in the

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Mikhail Loenko
2006/7/5, George Harley [EMAIL PROTECTED]: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library test layouts. The current

Re: Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Alex Blewitt
On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: George Harley wrote: A couple of weeks ago I mentioned that the TestNG framework [2] seemed like a reasonably good way of allowing us to both group together different kinds of tests and permit the exclusion of individual tests/groups of

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Richard Liang
Alex Blewitt wrote: On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: George Harley wrote: A couple of weeks ago I mentioned that the TestNG framework [2] seemed like a reasonably good way of allowing us to both group together different kinds of tests and permit the exclusion of

Re: Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Alex Blewitt
On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: It seems that you're very familiar with TestNG. ;-) So would you please identify what we shall do to transfer from junit to TestNG? Thanks a lot. Me? I'm just highly opinionated :-) There's guidelines for migrating from JUnit to TestNG at

Re: [classlib][testing] excluding the failed tests

2006-07-06 Thread Vladimir Ivanov
Message- From: Vladimir Ivanov [mailto:[EMAIL PROTECTED] Sent: Wednesday, July 05, 2006 12:41 AM To: harmony-dev@incubator.apache.org Subject: Re: [classlib][testing] excluding the failed tests Yesterday I tried to add a regression test to existing in security module TestCase, but, found

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Richard Liang
Alex Blewitt wrote: On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: It seems that you're very familiar with TestNG. ;-) So would you please identify what we shall do to transfer from junit to TestNG? Thanks a lot. Me? I'm just highly opinionated :-) There's guidelines for migrating

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread George Harley
Alex Blewitt wrote: On 06/07/06, Richard Liang [EMAIL PROTECTED] wrote: It seems that you're very familiar with TestNG. ;-) So would you please identify what we shall do to transfer from junit to TestNG? Thanks a lot. Me? I'm just highly opinionated :-) Hi Alex, I think we are all

Re: [classlib] Testing conventions - a proposal

2006-07-06 Thread Tim Ellison
Mikhail Loenko wrote: 2006/7/5, George Harley [EMAIL PROTECTED]: Hi, Just seen Tim's note on test support classes and it really caught my attention as I have been mulling over this issue for a little while now. I think that it is a good time for us to return to the topic of class library

Re: [classlib][testing] excluding the failed tests

2006-07-06 Thread Tim Ellison
Vladimir Ivanov wrote: More details: it is org/apache/harmony/security/tests/java/security/SecureRandom2Test.java test. At present time it has 2 failing tests with messages about SHA1PRNG algorithm (no support for SHA1PRNG provider). Looks like it is valid tests for non implemented

  1   2   3   >