Thanks everyone! I believe this would be a good addition. My only question is where are the failed tests stored? In _build? Also, maybe we can also implement it as a special tag called "--only failed" or "--only failures"?
*José Valimwww.plataformatec.com.br <http://www.plataformatec.com.br/>Founder and Director of R&D* On Thu, Nov 23, 2017 at 6:03 AM, Myron Marston <[email protected]> wrote: > I too would love to see ExUnit support an `--only-failures` flag. It's > one of my favorite features of RSpec and I wish every test framework had > it. I find that it makes a huge difference to my workflow to be able to > quickly and easily filter to the tests that failed the last time they ran. > > In fact, I love this feature of RSpec so much that I was the one who added > it to the framework a couple years back :). I'd be happy to help see it > get added to ExUnit if José and others were amenable. ExUnit already has > most of the building blocks needed for it via tags and filtering. > > Myron > > On Wednesday, November 22, 2017 at 2:48:14 PM UTC-8, José Valim wrote: >> >> To clarify, --stale does not run previously failed tests. >> >> > I just changed the format of the message built within >> `MyApp.Mixpanel`. This caused `assert_receive` to fail in tests throughout >> my app, as expected. But since the tests didn't directly reference >> `MyApp.Mixpanel`, `--stale` didn't know which ones should be run when the >> message format changed; I had to run all tests to get them to fail. >> >> That feels like a bug. Maybe we are being conservative on how we compute >> the dependencies. If you can provide a sample app that reproduces the >> error, I would love to take a look at it. >> >> >> >> *José Valimwww.plataformatec.com.br >> <http://www.plataformatec.com.br/>Founder and Director of R&D* >> >> On Wed, Nov 22, 2017 at 8:06 PM, Nathan Long <[email protected]> >> wrote: >> >>> Sure. I have a module called `MyApp.Mixpanel` with functions like >>> `track_event(:user_signup, data_map)`. These are called from various places >>> throughout the codebase. There's a production adapter, which actually sends >>> the event data to Mixpanel for analytics purposes, a dev adapter, which >>> just logs it, and a test adapter, which sends it to `self()` as a message. >>> >>> Several of my tests say things like "if I POST the info required for a >>> new user signup, I should get a message showing that the correct info would >>> have been sent to Mixpanel." These use `assert_receive`. >>> >>> I just changed the format of the message built within `MyApp.Mixpanel`. >>> This caused `assert_receive` to fail in tests throughout my app, as >>> expected. But since the tests didn't directly reference `MyApp.Mixpanel`, >>> `--stale` didn't know which ones should be run when the message format >>> changed; I had to run all tests to get them to fail. >>> >>> This is no big deal, but it would be nice in such situations to run all >>> tests once, then be able to whittle down the failing tests without >>> re-running the whole suite. >>> >>> On Wednesday, November 22, 2017 at 4:54:51 PM UTC-5, Louis Pilfold wrote: >>>> >>>> Hi Nathan >>>> >>>> I feel ExUnit --stale should always be able to tell this. Could you >>>> share your example please? >>>> >>>> Cheers, >>>> Louis >>>> >>>> On Wed, 22 Nov 2017 at 20:43 Nathan Long <[email protected]> wrote: >>>> >>>>> Ruby's Rspec has a handy option, `--only-failures`, which "filters >>>>> what examples are run so that only those that failed the last time they >>>>> ran >>>>> are executed". https://relishapp.com/rspec/rs >>>>> pec-core/docs/command-line/only-failures >>>>> >>>>> I'd love to have this feature in ExUnit. The closest thing I see right >>>>> now is `--stale`, but if ExUnit can't accurately determine which tests may >>>>> have been broken by a change, it doesn't work. (I have such an example, >>>>> but >>>>> don't want to be long-winded; maybe the utility of this feature is clear >>>>> enough?) >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "elixir-lang-core" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/elixir-lang-core/f5881fa3- >>>>> ed51-44be-8f6b-81e5181fa449%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/elixir-lang-core/f5881fa3-ed51-44be-8f6b-81e5181fa449%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elixir-lang-core" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/elixir-lang-core/2aa483e6-f63c-42d6-9e4b-84efb8adf9de% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/elixir-lang-core/2aa483e6-f63c-42d6-9e4b-84efb8adf9de%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/elixir-lang-core/270ca4ee-aa76-4e05-b7ad- > c06427e748b9%40googlegroups.com > <https://groups.google.com/d/msgid/elixir-lang-core/270ca4ee-aa76-4e05-b7ad-c06427e748b9%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4J9wMEN4w3wZ4WPio%3DVvCSmgtpcdQJJsP8ggzTngnGuxw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
