IMHO analysis of exception stacktrace output is a bad idea. Tests
should not produce uncatched exceptions unless they fail. Test authors
should explicitly catch an exception and print some text to stdout.
Reasons are:

- exception messages are not standard, different JVMs can produce
different output
- localized messages can be printed, the test may fail because of locales
- if test author expects IOException but subclass of this exception is
thrown and printed, the test will fail while if the test was written
in the way "catch IOException; print "PASSED" the test would pass


On 8/16/06, Stepan Mishura <[EMAIL PROTECTED]> wrote:
On 8/16/06, Anton Luht wrote:
>
> Cool idea - rethrow exceptions from exec'ed java :) You exec jvm that
> dies because some exception (say, NPE) not catched, support_exec
> analyzes output, understands that NPE was thrown and rethrows it :)
> People who will look at those tests will certainly go mad.


You misunderstoond the idea for using support_exec - no exceptions
re-throwing should be made. A test will verify only that VM fails in
expected way (i.e. with expected error message).

Do you have a better idea how to test that?

Thanks,
Stepan.

On 8/16/06, Jimmy, Jing Lv <[EMAIL PROTECTED]> wrote:
> > Paulex Yang wrote:
> > > Jimmy, Jing Lv wrote:
> > >> Hi,
> > >>
> > >>     I'm trying to write some unit tests of instrument. As discussed,
> > >> I'm using exec.
> > >>     Currently I've tried Support_Exec and find it fairly easy to
> > >> use(Thanks Stepan :) ).
> > >>
> > >>     For resources (jar files and utility classes), I make a new
> > >> directory:
> > >>
> 
$Harmony/modules/instrument/src/test/resource/org/apache/harmony/tests/instrument
> > >> and put them there.
> > >>
> > >>     And I meet a problem here. Currently the default classpath in
> > >> running test is "$Harmony/modules/instrument/bin", so it meets
> problem
> > >> in loading those utility classes (it can load jar files though).
> > >>     I remember Mark has plan to change the classpath in running
> > >> testcase, it'll be great if he make some progress.
> > >>
> > >>     An another problem is checking, currently the result is checked
> by
> > >> searching and compare in string(e.g, find "someException" in result).
> > >> But the framework seems fails if any exception is thrown out. Can it
> > >> make a little change that it passes and returns the error message in
> > >> exec?
> > > Didn't catch up here, did you mean that you need to check the
> exception
> > > message as well as exception type?
> >
> > Not necessary to check messages. However we can not catch those
> > exceptions so we have to analysis output. For an example:
> >
> > run "java -javaagent:test.jar myclass"
> >
> > so get output:
> > Exception in thread "main" java.lang.reflect.InvocationTargetException
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> >         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
> Source)
> >         at java.lang.reflect.Method.invoke(Unknown Source)
> >         at
> > sun.instrument.InstrumentationImpl.loadClassAndCallPremain(Unknown So
> > urce)
> > Caused by: java.lang.IllegalAccessError
> > ...
> >
> > In this case we can search for "InvocationTargetException" in the output
> > to ensure the expected exception. Sometimes we may also check "cause by"
> > exceptions.
> >
> > This approach seems not perfect. Any comments/suggestions? Thanks!
> >
> > <snip>
> > --
> >
> > Best Regards!
> >
> > Jimmy, Jing Lv
> > China Software Development Lab, IBM
> >
>
>


--
Thanks,
Stepan Mishura
Intel Middleware Products Division

------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




--
Regards,
Anton Luht,
Intel Middleware Products Division

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to