On 19/01/12 07:12, Christopher Schultz wrote:
All,I was testing 7.0.25 and "ant test" reports BUILD SUCCESSFUL but I started looking at the TEST-*.txt files that are emitted and I was wondering about a few things. First, I should probably be look at the bottom of the file for the junit summary that looks like this: Testsuite: org.apache.tomcat.util.threads.TestLimitLatch Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 2.545 sec Observing the "Failures: 0, Errors: 0" indicates that the tests have all passed correctly, right? I'm asking because I can also see things like this: TEST-org.apache.catalina.connector.TestMaxConnections.APR.txt:INFO: There were [4] passed requests and [2] connection failures Obviously, that's an INFO line, but it does indicate a "failure" of some kind. There are also some log lines like this: TEST-org.apache.catalina.startup.TestListener.NIO.txt:SEVERE: Context [/] startup failed due to previous errors Does that merely indicate that the test itself caused a failure, and that the failure-to-startup was intentional? Similarly: TEST-org.apache.catalina.tribes.group.interceptors.TestNonBlockingCoordinator.APR.txt:org.apache.catalina.tribes.ChannelException: Send failed, and sender is disconnected. Not retrying.; Faulty members:tcp://{127, 0, 0, 1}:4005; Again, all test files say "Failures: 0, Errors: 0" so I guess everything is okay. It's just tough to see those log lines without asking. Thanks, -chris
1. I noticed this kind of message when I was looking for problems in my own new tomcat tests. Because they didn't originate from my own tests, I didn't follow it up. Sorry...
2. I've seen similar situations in other projects, but I don't know if my explanation is relevant to the tomcat tests... in all previous cases, my tests have been complex enough to require starting another thread so I could have both a client and server active during each individual test method of the class.
3. The tomcat tests (at least the ones I have worked on) subclass TomcatBaseTest, which starts Tomcat in a new thread, which puts them into the category I am talking about.
4. As far as I remember, it is possible for these sub-threads to "fail", or even interfere with each-other (clashes for port numbers or other shared resources), or even interfere with other completely different test classes doing the same kind of thing, while the main thread happily passes all its junit assertions and reports "success".
5. Forgive me if this is too vague, or too simplistic to be helpful. Your comment reminded me of myself (about 10 years ago) when everyone else was happily reading successful junit summaries and I started looking at the log files generated by the tests. I found the messages (like yours) difficult to understand, and even more difficult to diagnose - set a debugger break point in the sub-thread and you will screw up the timing of the tests, thus never even arriving at the troublesome code. I often had to catch the exception when it was thrown, then analyse the call stack - just like the old days of reading core dumps!
6. If you are still reading this, then I'll cheer you up by saying about 5 in 6 of these "problems" came down to artefacts of the test design, rather than bugs in the logic actually under test. Unfortunately, I had to debug and fix all of them before I could stop worrying!
I hope this war story has been helpful. Brian --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
