garrensmith commented on a change in pull request #415: Add rfc for using ExUnit for unit tests URL: https://github.com/apache/couchdb-documentation/pull/415#discussion_r288424875
########## File path: rfcs/009-exunit.md ########## @@ -0,0 +1,129 @@ +--- +name: Formal RFC +about: Submit a formal Request For Comments for consideration by the team. +title: 'Use ExUnit testing framework for unit testing' +labels: rfc, discussion +assignees: '' + +--- + +# Introduction + +With the upgrade of supported Erlang version and introduction of Elixir into our +integration test suite we have an opportunity to replace currently used eunit +(for new tests only) with Elixir based ExUnit. + +The eunit testing framework is very hard to maintain. In particular, it has the +following problems: +- the process structure is designed in such a way that failure in setup or teardown + of one test affects the execution environment of subsequent tests. Which makes it + really hard to locate the place where the problem is coming from. +- inline test in the same module as the functions it tests might be skipped +- incorrect usage of ?assert vs ?_assert is not detectable since it makes tests pass +- there is a weird (and hard to debug) interaction when used in combination with meck + - https://github.com/eproxus/meck/issues/133#issuecomment-113189678 + - https://github.com/eproxus/meck/issues/61 + - meck:unload() must be used instead of meck:unload(Module) +- teardown is not always run, which affects all subsequent tests +- grouping of tests is tricky +- it is hard to group tests so individual tests have meaningful descriptions +- eunit implementation of `{with, Tests}` doesn't detect test name correctly +- it is hard to skip certain tests when needed + +ExUnit shouldn't have these problems: +- on_exit function is reliable in ExUnit +- it is easy to group tests using `describe` directive +- code-generation is trivial, which makes it is possible to generate tests from formal spec (if/when we have one) + +## Abstract + +Write necessary plumbing to be able to use ExUnit in unit tests. Review comment: I find the abstracts really confusing but I think this should be more a full summary of the whole RFC. Which is something like, we want to replace eunit because it has a bunch of issues, we will use exunit instead, it solves a lot of these problems. The advantage is we would then use a modern testing framework which is consistent for unit and integration tests. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
