Well, yeah. The one you're seeing down there does show some extra
command debug info. Only the bottom part of the output is meaningful.
Also, there's a number of ways to customize tests reports and an extra
summary/reports are generated in HTML/XML forms.
Cos
Nigel Daley wrote:
Cos,
Where else does the TestNG output go? Is there a way to have clean
console summary output of pass/fail from each test as it runs (ala
Junit)?
thanks,
Nige
On May 1, 2009, at 11:45 AM, Konstantin Boudnik wrote:
Ok, I have completed the conversion of the unit tests for project
Avro to be executed by TestNG framework. List of tests is built
dynamically and all.
Here's the log of the execution:
--------------- start of log's snippet ---------------
test-java-ng:
[testng] [TestNGAntTask] TESTNG PASSED @/tmp/testng34807 WHICH
CONTAINS:
[testng] [TestNGAntTask] -d
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test-
output
[testng] [TestNGAntTask] -sourcedir
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/src/test/java
[testng] [TestNGAntTask] -testclass
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestDataFile.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestFsData.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestProtocolGeneric.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestProtocolReflect.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestProtocolSpecific.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestReflect.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/TestSchema.class
[testng] [TestNGAntTask] /Users/cos/work/avro/trunk/build/test/
classes/org/apache/avro/io/TestValueReader.class
[testng] [TestNGAntTask] -suitename
[testng] [TestNGAntTask] AvroTestNG
[testng] [TestNGAntTask] -testname
[testng] [TestNGAntTask] Ant test
[testng] [Parser] Running:
[testng] AvroTestNG
[testng]
[testng] 523 [main] INFO org.apache.avro.ipc.SocketServer -
starting on 0.0.0.0/0.0.0.0:0
[testng] 523 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 531 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - connection from /
192.168.102.128:51751
[testng] 531 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 763 [main] INFO org.apache.avro.ipc.SocketServer -
starting on 0.0.0.0/0.0.0.0:0
[testng] 764 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 764 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - connection from /
192.168.102.128:51753
[testng] 764 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 883 [main] INFO org.apache.avro.ipc.SocketServer -
starting on 0.0.0.0/0.0.0.0:0
[testng] 883 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 883 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - connection from /
192.168.102.128:51755
[testng] 884 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 888 [Connection for
java.nio.channels.SocketChannel[connected local=/
192.168.102.128:51754 remote=/192.168.102.128:51755]] INFO
org.apache.avro.TestProtocolGeneric - echo: {name=foo}
[testng] 898 [Connection for
java.nio.channels.SocketChannel[connected local=/
192.168.102.128:51754 remote=/192.168.102.128:51755]] INFO
org.apache.avro.TestProtocolGeneric - echoBytes:
java.nio.HeapByteBuffer[pos=0 lim=13717 cap=13717]
[testng] 909 [Connection for
java.nio.channels.SocketChannel[connected local=/
192.168.102.128:51754 remote=/192.168.102.128:51755]] INFO
org.apache.avro.TestProtocolGeneric - hello: bob
[testng] 1159 [main] INFO org.apache.avro.ipc.SocketServer -
starting on 0.0.0.0/0.0.0.0:0
[testng] 1160 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng] 1162 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - connection from /
192.168.102.128:51757
[testng] 1162 [SocketServer on 0.0.0.0/0.0.0.0:0] INFO
org.apache.avro.ipc.SocketServer - listening on /0.0.0.0
[testng]
[testng] ===============================================
[testng] AvroTestNG
[testng] Total tests run: 42, Failures: 0, Skips: 0
[testng] ===============================================
[testng]
BUILD SUCCESSFUL
--------------- end of log's snippet ---------------
A couple of lessons learned from the exercise:
- JUnitConvertor isn't much help. I've found quite a number of test
methods simply ignored by the application
- a major chunk of conversion have to be done manually, e.g.
sequence of the test method execution is needed to be written before
hand, etc.
- if I knew the product (Avro) and its testing better it would take
much less time, because some of the issues weren't require any
investigation for they become obvious after some initial acquittance
with the code base structure, etc.
I'm creating new JIRA for this improvement and you should see the
patch in a couple of hours.
If anyone has a better idea of how to submit it - I'd appreciate to
hear an opinion from the development community.
Thanks,
Cos
Konstantin Boudnik wrote:
Sharad, Nigel.
I'm trying to finish the conversion of the junit tests into testng
and have accomplished a certain success. A couple of very special
issues I came across:
- a generated interface build/test/src/org/apache/avro/Test.java
was a source of errors when TestNG imports we added into test
files. I had to rename test.js to generated.js and refactor few
tests to work with the new name. Generally, using names like Test
for helper classes might be a source of troubles.
- it seems that some of the tests have to be executed in a
certain order. I probably should be able to derive the order of the
execution by re-engineering build.xml file itself. However,
before I'll jump on it I'd like to know if someone here has any
information about required java test execution sequence.
So, looks like the conversion process goes Ok, but it isn't
seamless and require a significant amount of manual work.
Thanks for any information,
Cos
Sharad Agarwal wrote:
Sounds good. I think you can create a Jira and provide a patch.
- create an XML listing of tests to be executed (simple
selection of
**/*.class doesn't work in this case)
I don't know much about Test NG, but can we workaround this issue?
Maintaining an XML file seems to be an overhead.
- Sharad
