I think doctests are extremely helpful and powerful since they encourage
developers to write testable, accurate documentation for their code. But
the lack of some of the niceties that exist in ExUnit might make it less
likely for developers to choose doctests instead of ExUnit tests, so I've
been thinking of ways to narrow this gap.
Currently if we are writing doctests and want to assert that the output of
the function we're testing matches a pattern instead of testing the output
for strict equality, we need to do one of the following two things:
```
iex> match?(%{c: _}, Map.put(%{a: :b}, :c, :d))
true
iex> %{c: _} = Map.put(%{a: :b}, :c, :d)
%{a: :b, c: :d}
```
There are a few problems I see with this at the moment. First is the
failure messages - they're not nearly as good as the failure messages in
ExUnit at the moment for pattern matching.
```
iex> %{f: _} = Map.put(%{a: :b}, :c, :d)
%{a: :b, c: :d}
```
1) doctest Benchee.Configuration.init/1 (1) (Benchee.ConfigurationTest)
test/benchee/configuration_test.exs:3
** (MatchError) no match of right hand side value: %{a: :b, c: :d}
stacktrace:
(for doctest at) lib/benchee/configuration.ex:167: (test)
And then for the other one we get an even less helpful error message:
```
iex> match?(%{f: _}, Map.put(%{a: :b}, :c, :d))
true
```
1) doctest Benchee.Configuration.init/1 (1) (Benchee.ConfigurationTest)
test/benchee/configuration_test.exs:3
Doctest failed
doctest:
iex> match?(%{f: _}, Map.put(%{a: :b}, :c, :d))
true
code: match?(%{f: _}, Map.put(%{a: :b}, :c, :d)) === true
left: false
right: true
stacktrace:
lib/benchee/configuration.ex:167: Benchee.Configuration (module)
To get the very nice new colored diff for matches available in 1.10 in our
failure message we can do the following:
```
iex> assert %{f: _} = Map.put(%{a: :b}, :c, :d)
%{a: :b, c: :d}
```
1) doctest Benchee.Configuration.init/1 (1) (Benchee.ConfigurationTest)
test/benchee/configuration_test.exs:3
match (=) failed
code: assert %{f: _} = Map.put(%{a: :b}, :c, :d)
left: %{f: _}
right: %{a: :b, c: :d}
stacktrace:
(for doctest at) lib/benchee/configuration.ex:167: (test)
But putting this assertion in a doctest takes away from value of the
example as documentation since the example is no longer really able to be
cut and pasted - it's now more test than documentation and doesn't really
belong here.
Also, If you want to pattern match in the final line of a doctest, you
still need to put the return value of the match (which is always the value
being matched against) at the end of the test or else it's not parsed as a
valid test, which makes the pattern match useless since you're already
making the more strict comparison with `===`. Basically, we can't do this
today:
```
iex> %{f: _} = Map.put(%{a: :b}, :c, :d)
iex> %{e: _} = Map.put(%{a: :b}, :e, :f)
iex> %{f: _} = Map.put(%{a: :b}, :f, :e)
```
My proposal is that we allow this to be a valid doctest, and that we use it
to give really good failure messages. Anytime the last line of a doctest is
a pattern match, we convert that into `assert pattern = expr` when parsing
and running the test.
The other thing I came up with (that I don't particularly like but figured
I'd put it out there just in case someone builds something good off of it)
is to use the anonymous function shorthand syntax to represent the output
of the previous line and do something like this:
```
iex> Map.put(%{a: :b}, :c, :d)
%{f: _} = &1
iex> Map.put(%{a: :b}, :e, :f)
%{e: _} = &1
```
This gets a little further away from the intended purpose of documentation
and towards more of a test, but it would still give a better failure
message for developers and keeps the requirement that the last line of a
doctest includes something to test against.
If folks have any better ideas for this I'd love to hear them!
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elixir-lang-core+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/b93ea3e4-9ebe-4090-b14c-bae138c60f4f%40googlegroups.com.
