IMHO analysis of exception stacktrace output is a bad idea. Tests should not produce uncatched exceptions unless they fail. Test authors should explicitly catch an exception and print some text to stdout. Reasons are:
- exception messages are not standard, different JVMs can produce different output - localized messages can be printed, the test may fail because of locales - if test author expects IOException but subclass of this exception is thrown and printed, the test will fail while if the test was written in the way "catch IOException; print "PASSED" the test would pass On 8/16/06, Stepan Mishura <[EMAIL PROTECTED]> wrote:
On 8/16/06, Anton Luht wrote: > > Cool idea - rethrow exceptions from exec'ed java :) You exec jvm that > dies because some exception (say, NPE) not catched, support_exec > analyzes output, understands that NPE was thrown and rethrows it :) > People who will look at those tests will certainly go mad. You misunderstoond the idea for using support_exec - no exceptions re-throwing should be made. A test will verify only that VM fails in expected way (i.e. with expected error message). Do you have a better idea how to test that? Thanks, Stepan. On 8/16/06, Jimmy, Jing Lv <[EMAIL PROTECTED]> wrote: > > Paulex Yang wrote: > > > Jimmy, Jing Lv wrote: > > >> Hi, > > >> > > >> I'm trying to write some unit tests of instrument. As discussed, > > >> I'm using exec. > > >> Currently I've tried Support_Exec and find it fairly easy to > > >> use(Thanks Stepan :) ). > > >> > > >> For resources (jar files and utility classes), I make a new > > >> directory: > > >> > $Harmony/modules/instrument/src/test/resource/org/apache/harmony/tests/instrument > > >> and put them there. > > >> > > >> And I meet a problem here. Currently the default classpath in > > >> running test is "$Harmony/modules/instrument/bin", so it meets > problem > > >> in loading those utility classes (it can load jar files though). > > >> I remember Mark has plan to change the classpath in running > > >> testcase, it'll be great if he make some progress. > > >> > > >> An another problem is checking, currently the result is checked > by > > >> searching and compare in string(e.g, find "someException" in result). > > >> But the framework seems fails if any exception is thrown out. Can it > > >> make a little change that it passes and returns the error message in > > >> exec? > > > Didn't catch up here, did you mean that you need to check the > exception > > > message as well as exception type? > > > > Not necessary to check messages. However we can not catch those > > exceptions so we have to analysis output. For an example: > > > > run "java -javaagent:test.jar myclass" > > > > so get output: > > Exception in thread "main" java.lang.reflect.InvocationTargetException > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown > Source) > > at java.lang.reflect.Method.invoke(Unknown Source) > > at > > sun.instrument.InstrumentationImpl.loadClassAndCallPremain(Unknown So > > urce) > > Caused by: java.lang.IllegalAccessError > > ... > > > > In this case we can search for "InvocationTargetException" in the output > > to ensure the expected exception. Sometimes we may also check "cause by" > > exceptions. > > > > This approach seems not perfect. Any comments/suggestions? Thanks! > > > > <snip> > > -- > > > > Best Regards! > > > > Jimmy, Jing Lv > > China Software Development Lab, IBM > > > > -- Thanks, Stepan Mishura Intel Middleware Products Division ------------------------------------------------------ Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
-- Regards, Anton Luht, Intel Middleware Products Division --------------------------------------------------------------------- Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]