On Fri, 21 Feb 2020 at 23:41:06, Frédéric Mangano-Tarumi wrote: > prove comes with its conventions, which are from the original TAP. > First, it expects test programs to end with `.t`, which is restrictive > enough to exclude non-test files, but doesn\u2019t bind to any specific > language since the shebang is used to know how to run each test. This is > by the way a great opportunity to start writing tests in Python too.
The current tests mostly execute commands and compare their outputs, and I don't see much benefit in rewriting them in Python. That being said, I think we could benefit from additional unit tests. > Another convention is to name the test directory `t` and to call prove > from the root of the project. This one is harder to use since all the > existing tests assume they\u2019re run from the `test` directory. A simple cd > at the beginning of every script is probably a good solution, but let\u2019s > leave that problem open for now. While we can run prove from the `test` > directory, running tests from the root has the notable advantage of > letting us import aurweb Python modules without hacking PYTHONPATH. > > Here are various suggestions: > > 1. Change the extension of all the test programs to `.t`. Sounds good. > 2. Create `test\README` to tell developers about TAP and prove. Great idea! > 3. Make `make check` call prove. I like the idea of being able to use prove but I'd prefer still having a way to run tests without having prove installed. How about adding a "prove" target to the Makefile instead of changing `make check`? > 4. Delete the `make t1234-xxx.sh` rules. > 5. Delete the whole Makefile altogether. Why? Those are small, super low maintenance, and provide the benefit of being able to easily run the test suite without prove. > 6. Run the test programs from the project root. > 7. Rename the `test` directory to `t`. Sounds good. > 1 to 4 are cheap and have a significant added value in my opinion. > 5 to 7 less so, but I think that if we want to use TAP, we should stick > to its conventions. I agree with sticking to TAP conventions but it should be easy to keep almost everything working as-is at the same time without any extra work. Thanks for bringing this up! Lukas