Re: [java] Test result inconsistencies?

[Steve Vinoski]

On Dec 8, 2006, at 7:07 PM, Steve Vinoski wrote:
Kim, an added note: if you run the tests under Eclipse, they seem  
to fail every single time. The common and broker tests are OK, but  
the client tests get about halfway through, fail, and then hang. I  
don't know what the cause is, but I'm starting to look into it.

This is probably a false alarm. We use includes and excludes in maven  
so that only certain tests are executed. Under Eclipse, though, it  
looks like it's trying to run everything, and is tripping over tests  
that we already know don't work.

This points to still more reasons why the tests need an overhaul.  
Maybe we ought to move all tests that currently are not executed by  
maven out from under the project directories and into, say, a  
broken_tests directory at the java level? That way, we won't lose  
them, but they won't get in way under Eclipse or require special  
include and exclude directives under maven.

Unless someone knows of a good reason not to do this, I'll proceed  
with it, but I'll give it a couple day so everyone can speak up if  
they want.

BTW, I get the impression, but may very well be mistaken, that many  
in the qpid-dev group place a high value on added functionality but  
do not value tests as highly. I'm generally a "test first" kind of  
guy, in that if I'm adding functionality, I write the tests for it  
first, and then write the functionality, and commit them all at the  
same time. The low test coverage we currently have makes me wary of  
changing anything, simply because without a test, I don't know  
whether my changes would break anything, and among those of us still  
relatively new to this code base, I doubt I'm alone in that feeling.

When I raise the testing issue in this list, as I've already done on  
numerous occasions, I usually get next to nothing in terms of  
feedback. Do others share my concerns about this whole testing issue?  
Do we as a group agree on the value of overhauling and enhancing our  
tests?

--steve



--steve

On Dec 8, 2006, at 7:25 AM, Kim van der Riet wrote:

I notice that about a quarter to a third of the time the maven tests
hang, even though the first one after a long period seems to work
correctly. Consecutive runs either pass or hang (which I have to
terminate). I make no other changes from run to run - I just run  
"mvn".
Sometimes consecutive runs pass, hang, pass, hang, etc. but not  
always.

Is is possible that there is some sort of left-over
state/condition/file/process from previous tests that may be  
interfering
with the current test? I searched for running brokers, etc., but I  
did
not see anything.

When it works, I get the normal test passed message.  When it hangs I
get (always the same message):

<snip>
Running org.apache.qpid.test.unit.client.forwardall.CombinedTest
Starting 2 services...
Starting client...
Received 1 of 2 responses.
<hang>

Other times, where I got completely normal results on a previous run,
some tests produce error messages, but the test still passes. When  
this
happens (about 1 run in 3 or 4), I get one or several of the  
following:

<snip>
Running  
org.apache.qpid.test.unit.client.channelclose.ChannelCloseOkTest
pool-16-thread-4 2006-12-07 16:27:03,633 ERROR  
[qpid.server.protocol.AMQPFastProtocolHandler] Exception caught  
inAMQProtocolSession(anonymous(7912507)), closing session  
explictly: java.lang.IllegalStateException: Handed undecoded  
ByteBuffer buf = HeapBuffer[pos=0 lim=0 cap=0: empty]
java.lang.IllegalStateException: Handed undecoded ByteBuffer buf =  
HeapBuffer[pos=0 lim=0 cap=0: empty]
        at  
org.apache.qpid.server.protocol.AMQPFastProtocolHandler.messageReceiv 
ed(AMQPFastProtocolHandler.java:198)
        at org.apache.mina.common.support.AbstractIoFilterChain 
$2.messageReceived(AbstractIoFilterChain.java:189)
        at  
org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageR 
eceived(AbstractIoFilterChain.java:502)
        at  
org.apache.mina.common.support.AbstractIoFilterChain.access$1000 
(AbstractIoFilterChain.java:52)
        at org.apache.mina.common.support.AbstractIoFilterChain 
$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:777)
        at org.apache.qpid.pool.Event.process(Event.java:80)
        at org.apache.qpid.pool.Job.processAll(Job.java:81)
        at org.apache.qpid.pool.Job.run(Job.java:103)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask 
(ThreadPoolExecutor.java:650)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)
pool-20-thread-2 2006-12-08 07:03:21,590 ERROR  
[qpid.server.protocol.AMQPFastProtocolHandler] Exception caught  
inAMQProtocolSession(anonymous(1668655)), closing session  
explictly: java.lang.IllegalStateException: Handed undecoded  
ByteBuffer buf = HeapBuffer[pos=0 lim=82 cap=82: 01 00 01 00 00 00  
4A 00 32 00 14 00 00 19 74 6D 70 5F 61 6E 6F 6E 79 6D 6F 75 73 28  
33 32 34 38 36 35 39 30 29 5F 31 0A 61 6D 71 2E 64 69 72 65 63 74  
19 74 6D 70 5F 61 6E 6F 6E 79 6D 6F 75 73 28 33 32 34 38 36 35 39  
30 29 5F 31 01 00 00 00 00 CE]
java.lang.IllegalStateException: Handed undecoded ByteBuffer buf =  
HeapBuffer[pos=0 lim=82 cap=82: 01 00 01 00 00 00 4A 00 32 00 14  
00 00 19 74 6D 70 5F 61 6E 6F 6E 79 6D 6F 75 73 28 33 32 34 38 36  
35 39 30 29 5F 31 0A 61 6D 71 2E 64 69 72 65 63 74 19 74 6D 70 5F  
61 6E 6F 6E 79 6D 6F 75 73 28 33 32 34 38 36 35 39 30 29 5F 31 01  
00 00 00 00 CE]
        at  
org.apache.qpid.server.protocol.AMQPFastProtocolHandler.messageReceiv 
ed(AMQPFastProtocolHandler.java:198)
        at org.apache.mina.common.support.AbstractIoFilterChain 
$2.messageReceived(AbstractIoFilterChain.java:189)
        at  
org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageR 
eceived(AbstractIoFilterChain.java:502)
        at  
org.apache.mina.common.support.AbstractIoFilterChain.access$1000 
(AbstractIoFilterChain.java:52)
        at org.apache.mina.common.support.AbstractIoFilterChain 
$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:777)
        at org.apache.qpid.pool.Event.process(Event.java:80)
        at org.apache.qpid.pool.Job.processAll(Job.java:81)
        at org.apache.qpid.pool.Job.run(Job.java:103)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask 
(ThreadPoolExecutor.java:650)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run 
(ThreadPoolExecutor.java:675)
        at java.lang.Thread.run(Thread.java:595)

<either of the above repeated several times, with possibly  
different buffer contents and lengths>

Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:  
8.194 sec
Running  
org.apache.qpid.test.unit.client.connectionurl.ConnectionURLTest
AnonymousIoService-8 2006-12-08 07:03:29,782 ERROR  
[apache.qpid.pool.Event] Exception event constructed:  
java.util.concurrent.RejectedExecutionException
java.util.concurrent.RejectedExecutionException
        at java.util.concurrent.ThreadPoolExecutor 
$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1477)
        at java.util.concurrent.ThreadPoolExecutor.reject 
(ThreadPoolExecutor.java:384)
        at java.util.concurrent.ThreadPoolExecutor.execute 
(ThreadPoolExecutor.java:856)
        at org.apache.qpid.pool.PoolingFilter.fireEvent 
(PoolingFilter.java:63)
        at org.apache.qpid.pool.PoolingFilter.messageReceived 
(PoolingFilter.java:138)
        at  
org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageR 
eceived(AbstractIoFilterChain.java:502)
        at  
org.apache.mina.common.support.AbstractIoFilterChain.access$1000 
(AbstractIoFilterChain.java:52)
        at org.apache.mina.common.support.AbstractIoFilterChain 
$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:777)
        at  
org.apache.mina.filter.executor.ExecutorFilter.processEvent 
(ExecutorFilter.java:243)
        at org.apache.mina.filter.executor.ExecutorFilter 
$ProcessEventsRunnable.run(ExecutorFilter.java:305)
        at  
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor 
$Worker.runTask(ThreadPoolExecutor.java:665)
        at  
edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor 
$Worker.run(ThreadPoolExecutor.java:690)
        at java.lang.Thread.run(Thread.java:595)
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:  
0.157 sec
<snip>

These errors seem to be independent of the hang problem first  
mentioned,
as they sometimes precede a hang, sometimes continue to a successful
pass message.

Not being active on the java side for a little while, these may be
development-in-progress errors, or they may be tests interacting with
each other in some way. I hope I am not raising a well-known issue  
or a
red herring... However, my "red flag" is based on the idea that
consecutive tests under otherwise identical conditions should produce
identical results.

Ideas?

Kim