|
According to our documentation :
The default behaviour of OnlineTestCase is that, if connect() throws an exception, the test suite is disabled, causing each test to pass without being run. In addition, exceptions thrown by disconnect() are ignored. This behaviour allows tests to be robust against transient outages of online resources, but also means that local software failures in connect() or disconnect() will be silent. To have exceptions thrown by connect() and disconnect() cause tests to fail, set skip.on.failure=false in the fixture property file. This restores the traditional behaviour of unit tests, that is, that exceptions cause unit tests to fail.
After setting up online tests on the build server, using `skip.on.failure=false` I have observed that connection failures to the database still do not cause the test to fail. Looking at the code, I have observed that the connect() method is either not implemented (PostgisNGCreateDatabaseOnlineTest, ...), or does not actually attempt to connect to the database (JDBCTestSupport, OracleOnlineTestCase, ...) for many of the implementations of OnlineTestCase, meaning that connect() often does not throw an exception if a connection is not available.
Additionally, the OnlineTestCase.run() method (which is called before connect()) calls checkAvailiable(), which usually does attempt to connect to the database. A failure here will skip all tests and return success, regardless of `skip.on.failure=false`.
Between these two issues, `skip.on.failure=false` does not work as documented.
|
