stevedlawrence commented on issue #17:
URL: https://github.com/apache/daffodil-sbt/issues/17#issuecomment-1910413352
I'm not suggesting we drop the current style, but I think a lot of Mike's
concerns could theoretically be resolved, and I think would make things better:
> I depend on having a line of scala code that I can click on and say
"debug", with breakpoints set in layers, UDFs or other code artifacts of the
schema.
I think it would actually be much more convenient be able to right-click a
parserTestCase in a TDML file and say "run" or "debug", and behind the scenes
it calls runOneTest. I'm not sure if an IDE plugin could support that though or
if they are more code-centric like SBT. But if it could, all the
schemas/inputs/outputs/etc are right there at what you're already looking at.
One of the things I dislike about the scala file separation is having to find
the runner and tdml file associated with a failing scala test--it's a
multi-step process.
> I also really like doing
>
> @Test def test_test01(): Unit = runner.trace.runOneTest("test01")
This could be supported by adding something like a `trace="true"` attribute
to parserTestCase element, which could cause the test interface to enable
tracing before running the test.
> Last point: We frequently find a bug, create TDML tests for it, then do:
>
> // DAFFODIL-XYZY
> // @Test def test_test01(): Unit = runner.runOneTest("test01")
>
> ...
>
> This could be achieved by some technique of editing the TDML file however.
Worst case commenting the whole test out and putting the JIRA ticket ID into
the comment.
I think editing the TDML actually has benefits. For example, you could have
an attribute like `broken="DAFFODIL-123"`. This means that if a test is broken
it now *requires* there be a referenced bug, and the existence of the attribute
would tell the test-interface to skip it. It would also allow you to do
something like this:
```scala
sbt testOnly -- --broken
```
To run only the broken tests to see if any tests have been fixed without
realizing it.
Getting rid of scala files also avoid issues where tests are not run because
we forgot to add a `@Test` for them, or we run the same test multiple times
because we copy pasted an existing test without changing the names. I do this
frequently.
But as I said before, this all requires a unique test-interface plugin for
each thing we support, which is at least SBT and one or more IDES, which is
probably too much effort.
> I did a thing a while back, could help as a reference impl:
https://github.com/jw3/sbt-tdml
This is very cool, I forgot about this. This could be very useful, and the
test-interface code is very small and maintainable. Though, I image we really
just want the TDML generation aspect, and have it generate our current, verbose
format with all the `@Test`s.
Another potential issue with the test-interface approach, which looking at
sbt-tdml reminded me of, is that SBT doesn't separate plugin/test-inteface
dependencies from normal dependencies. So for example, if a DFDL schema project
depends on one version of daffodil-tdml, and the sbt-tdml test-interface
depends on another, we can run into problems. I wonder if we could resolve this
by saying that the sbt-tdml test interface has a "provided" dependency to
daffodil-tdml-processor. Then sbt-daffodil adds whatever version the user
specifies in `daffodilVersion` and the test-interface will just use that. And
as long as the daffodil-tdml API stays stable (which it pretty much is), this
would work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
