stevedlawrence commented on issue #17: URL: https://github.com/apache/daffodil-sbt/issues/17#issuecomment-1908786545
I looked at [sbt's test-interface](https://github.com/sbt/test-interface/tree/master/src/main/java/org/scalatools/testing) to see about what capabilities it provides for custom test infrastructures and see if option 3 is a possibility. The idea is that we would no longer use JUnit and instead us a custom test infrastructure specific for TDML files. The scala test API is actually pretty small, but that also means it's pretty limited. First, it requires that test suites are defined in code, so there's no way to say something like "treat each .tdml file as a test suite". Instead, it only supports classes that either extend a super class or are annotated. But you could imagine having a one or a few scala files ontaining all test suits like this: ```scala import org.apache.daffodil.tdml.TDMLSuite class TestSuiteA extends TDMLSuite("/org/example/a.tdml") class TestSuiteB extends TDMLSuite("/org/example/b.tdml") class TestSuiteB extends TDMLSuite("/org/example/c.tdml") ... ``` This way you wouldn't need to define all the tests, you just associate each suite with a tdml file. Much less verbose with less boilerplate. When running tests via sbt, we would tell sbt how to discover these classes (i.e. anything that extends TDMLSuite) and it would find and the names to a custom test runner class that we implement. This test runner would then create an instance of the class name, read the associated tdml file, and execute the tests, passing back information to SBT about the name of each test we ran in the suite and the result. We would need some way for a TDML file to say "this test is broken, skip it" (right now we just comment out tests), but other than that it would pretty drastically reduce boilerplate. The major downside with this approach is that no IDE's would support it. I was hoping the SBT test API would allow a way to pass information back about the tests so IDE's could list them, know which file/line they are at to enable right-click in a .tdml file to execute, etc. but it just doesn't provide that information. So this approach would also need custom plugins for any IDE's we also want to support, which is probably a deal breaker. So I think idea 3 is out. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
