[ https://issues.apache.org/jira/browse/JDO-765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239315#comment-16239315 ]
Craig L Russell edited comment on JDO-765 at 11/5/17 12:02 AM: --------------------------------------------------------------- I applied the patch and the behavior did change, but not exactly what I expected. Good: When running with -Djdo.tck.onFailure=failEventually, the tck test ran through all of the tests, reported "Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:runtck (default) on project jdo-tck: There were 2 TCK test failures" and then failed the mvn goal. Good: When running with -Djdo.tck.onFailure default, the tck test ran through all of the tests, reported "Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:runtck (default) on project jdo-tck: There were 2 TCK test failures" and then failed the mvn goal. Bad: When running with -Djdo.tck.onFailure=failFast, the tck test ran through all of the datastoreidentity tests before failing. I expected the test would fail right after the first failure, "Running tests for lifecycle.conf with datastoreidentity on 'derby' mapping= ... FAIL" but the tck test continued with the rest of the datastoreidentity tests. was (Author: clr): I applied the patch and the behavior did change, but not exactly what I expected. Good: When running with -Djdo.tck.onFailure=failEventually, the tck test ran through all of the tests, reported "Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:runtck (default) on project jdo-tck: There were 2 TCK test failures" and then failed the mvn goal. Bad: When running with -Djdo.tck.onFailure default, the tck test ran through all of the tests, reported "Failed to execute goal org.apache.jdo:jdo-exectck:3.2-SNAPSHOT:runtck (default) on project jdo-tck: There were 2 TCK test failures" and then failed the mvn goal. Bad: When running with -Djdo.tck.onFailure=failFast, the tck test ran through all of the datastoreidentity tests before failing. I expected the test would fail right after the first failure, "Running tests for lifecycle.conf with datastoreidentity on 'derby' mapping= ... FAIL" but the tck test continued with the rest of the datastoreidentity tests. > TCK should fail if tests fail > ----------------------------- > > Key: JDO-765 > URL: https://issues.apache.org/jira/browse/JDO-765 > Project: JDO > Issue Type: Bug > Reporter: Tilmann Zäschke > Priority: Trivial > Attachments: JDO-XXX-patch.txt > > > Currently, TCK test failures are reported to the console and in the logs, but > the Maven build still succeeds. > The proposal is to make the behavior configurable, the default being that the > Maven build fails if any TCK tests fail. > The new configuration option would be called "-Djdo.tck.onFailure" and have > the following options: > * "failFast" will immediately abort the test run. > * "failEventually" (default) will execute the whole TCK before failing. > * "logOnly" will report failures to console and logs only but return > 'SUCCESS' to the Maven execution environment. > Attached is a patch with an implementation of the proposed changes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)