Hm. I'll chip in with an opinion based on surprises I got experimenting a
bit with the mix test task.
If I wanted to run tests across multiple files identified by line numbers,
I would naively write `mix test test/some_test.exs:15
test/some_test.exs:20`.
`mix test test/some_test.exs` will run all tests of a single file.
`mix test test/some_test.exs test/another_test.exs` will run all tests of
both files.
`mix test test/some_test.exs:15` will run the test located at line 15 for
that file.
`mix test test/some_test.exs test/another_test.exs:20` will run all tests
in some_test.exs, but not the test at line 20 in another_test.exs. 樂
`mix test test/some_test.exs:15 test/another_test.exs` will run all tests
in another_test.exs, but not the test at line 15 in some_test.exs 樂
In both cases, my expectation was to run all tests of one file, and the
test at the specific line for the other file.
`mix test test/some_test.exs:15 test/another_test:20` will exit with an
error:
"Paths given to `mix test` did not match any directory/file:
test/some_test.exs:15, test/another_test.exs:20"
My expectation was that two tests would have been run, the one at line 15
in some_test.exs, and at line 20 in another_test.exs.
Do people agree that this naive expectation is an intuitive one? (I didn't
really know any of the internals of the test runner.) Or would other people
have other intuitions?
For running multiple tests in a single file identified by line numbers, I'd
support `mix test test/some_test.exs:15:20:35`.
I'd also expect to be able to run `mix test test/some_test.exs:15:20:35
test/another_test.exs:40:75:120`.
The question of shorthand for multiple lines within a single file is of
course separate from that of running individual tests across multiple
files. Simply commenting as the multi-file issue was brought up.
On Wed, Jan 9, 2019 at 4:56 PM Devon Estes wrote:
> `mix test --only line:74 --only line:92
> test/potion_web/plugs/persist_request_plug_test.exs` does work, but don't
> you think it's a little confusing to allow `--only` more than once?
>
> I'm cool with the `mix test
> test/potion_web/plugs/persist_request_plug_test.exs:92:94:102` syntax.
> Whatever syntax is chosen doesn't matter all that much to me, acutally.
>
> I've personally never wanted to run tests in more than one file, but I can
> imagine that someone might (i.e. running some unit tests & integration
> tests together).
>
> On Wednesday, January 9, 2019 at 12:44:07 PM UTC+1, Amos King wrote:
>>
>> The syntax that José proposed is common to other command line tools so I
>> vote for it.
>>
>> I have wanted to do more than one file. As a user I’m not at all
>> concerned with the verbosity of the parsed version. If it isn’t a big
>> impact on performance. Some setups can be large, and starting of
>> applications. I’d rather save the time there and not have to chain command
>> line calls. Chaining command line calls also has the added issue of one
>> command has a failure and the rest don’t run.
>>
>> Amos King
>> Binary Noggin
>>
>> On Jan 9, 2019, at 04:46, Sven Gehring wrote:
>>
>> I'd personally prefer the syntax proposed by José.
>>
>> Also, I would argue that allowing for multiple files introduces quite a
>> bit of unnecessary complexity.
>>
>> I never had the need to run only specific tests in multiple files but I
>> think if it is really necessary for
>> someone, you can always still chain the commands (ie. mix test ... && mix
>> test ...) which is not ideal
>> because you do the setup/teardown twice but I think it would work to
>> cover that edge case
>>
>> Sven
>>
>> On Wed, 9 Jan 2019 at 11:18 José Valim
>> wrote:
>>
>>> *Stream of thoughts below.*
>>>
>>> Can you use only instead?
>>>
>>> mix test --only line:74 --only line:92
>>> test/potion_web/plugs/persist_request_plug_test.exs
>>>
>>>
>>> My only concern with your feature proposal is that it may lead to
>>> wanting to run multiple tests across multiple files and that's a trickier
>>> problem. One option would be change our line filter to be something like
>>> file+line. In a way that:
>>>
>>> mix test test/potion_web/plugs/persist_request_plug_test.exs:92
>>>
>>>
>>> becomes:
>>>
>>> Including tags: [location:
>>> {"test/potion_web/plugs/persist_request_plug_test.exs", 92}]
>>>
>>>
>>> but that's quite more verbose. Thoughts?
>>>
>>> However, if we decide we don't need to support multiple files, then I
>>> would do:
>>>
>>> mix test test/potion_web/plugs/persist_request_plug_test.exs:92:94:102
>>>
>>>
>>> *José Valim*
>>> www.plataformatec.com.br
>>> Skype: jv.ptec
>>> Founder and Director of R
>>>
>>>
>>> On Wed, Jan 9, 2019 at 11:00 AM Devon Estes wrote:
>>>
Currently, if I want to run only a single test by line number, I can
easily do:
mix test test/potion_web/plugs/persist_request_plug_test.exs:92
This is parsed in ExUnit as:
Including tags: [line: "92"]
Excluding tags: [:test]
But if I want to only
