Vikas Mayur wrote:
Well, All the tests supposed to be run successfully. If you see the
logs, note that all the test are not failed.
The only reason that I can think of is that In some cases, the data
for all test cases are not defined properly. I mean few of the test
case does not get the desired data and which is when changed by any
earlier test causes them to fail.
Other reason could be recent changes in services that are being tested
by test case.
Also the whole automated test-suite is supposed to run with fresh data
each time.
Running the tests with fresh data does reduce but not eliminate the
number of failures as shown below.
bash-3.2$ grep "Pass:" log.mysql.clean
2009-02-12 16:37:24,112 (main) [ JunitContainer.java:73 :INFO ]
[JUNIT] Pass: true | # Tests: 4 | # Failed: 0 # Errors: 0
2009-02-12 16:38:34,015 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 22 | # Failed: 0 # Errors: 0
2009-02-12 16:38:34,600 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 20 | # Failed: 1 # Errors: 0
2009-02-12 16:38:36,424 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 1 | # Failed: 0 # Errors: 1
2009-02-12 16:38:36,824 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 5 | # Failed: 0 # Errors: 0
2009-02-12 16:38:38,476 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 10 | # Failed: 2 # Errors: 0
2009-02-12 16:38:39,109 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 2 | # Failed: 0 # Errors: 0
2009-02-12 16:38:43,475 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 7 | # Failed: 1 # Errors: 0
2009-02-12 16:38:43,523 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 1 | # Failed: 1 # Errors: 0
2009-02-12 16:38:44,479 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 12 | # Failed: 13 # Errors: 0
2009-02-12 16:38:45,400 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 2 | # Failed: 0 # Errors: 0
2009-02-12 16:38:45,503 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 1 | # Failed: 0 # Errors: 0
bash-3.2$ grep "Pass:" log/log.mysql.dirty
2009-02-12 16:42:33,133 (main) [ JunitContainer.java:73 :INFO ]
[JUNIT] Pass: true | # Tests: 4 | # Failed: 0 # Errors: 0
2009-02-12 16:43:45,049 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 22 | # Failed: 1 # Errors: 0
2009-02-12 16:43:45,618 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 20 | # Failed: 2 # Errors: 0
2009-02-12 16:43:46,115 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 1 | # Failed: 0 # Errors: 1
2009-02-12 16:43:46,322 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 5 | # Failed: 0 # Errors: 0
2009-02-12 16:43:47,376 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 10 | # Failed: 6 # Errors: 0
2009-02-12 16:43:47,800 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 2 | # Failed: 0 # Errors: 0
2009-02-12 16:43:49,633 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 7 | # Failed: 2 # Errors: 0
2009-02-12 16:43:49,697 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 1 | # Failed: 1 # Errors: 0
2009-02-12 16:43:51,305 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: false | # Tests: 12 | # Failed: 13 # Errors: 0
2009-02-12 16:43:52,333 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 2 | # Failed: 0 # Errors: 0
2009-02-12 16:43:52,386 (main) [ TestRunContainer.java:135:INFO ]
[JUNIT] Pass: true | # Tests: 1 | # Failed: 0 # Errors: 0
bash-3.2$
