Hi,
I really like the automation that Skara is providing. It saves time for
committers as well as for reviewers. In order to increase productivity even
more (without decreasing quality), there might be an additional step we
might somehow automate. Many PR's contain changes in java code and in test
code. What I typically do as a first step of a review, is to first run the
tests on the latest code, then apply the PR, and run the tests again.
Clearly, no test should fail in the second run, while it would be ok (and
desired) that at least 1 tests fails in the first run.
My typical approach for this is to download the diff, and manually split it
in 2 parts: code + tests. I then apply the part with the tests, and run
them. Next, I apply the part with the code, and re-run the tests.
Clearly, this is not enough for a review, but if we automate this, it gives
an additional indication that a PR is going in the right direction.
I'm not sure how easy/hard this would be to implement. The bot should be
able to detect the `src/test/java`, and provide 2 downloads in the comments
of the PR ("download test diff" and "download code diff"). It can also
execute those steps in a github action, and fail if the resulting tests are
failing after applying the code diff.
Thoughts?
- Johan
