Yes, thank you Paul and Adam for "translating" my intent. :) > I'm certainly not saying this is the best way for all projects - it requires care, much like running Ecto in async tests requires care and a good understanding of the problem.
I don't agree with the comparison because Ecto tests are sandboxed. Sure, there are consequences of using the sandbox, but it is a much safer by default approach than the approach proposed here. To be clear, I understand and agree with the problem, but I don't agree with the solution because it is not ultimately solving the problem at hand. For example, speaking about Ecto, you could also use Mox, which also has an ownership-like mechanism, similar to Ecto's. You could define a behaviour, provide a default value for said behaviour, and then mock it in specific tests. This means your tests can run concurrently all the time. However, that sounds like overengineering for something as simple as reading the application environment. In any case, I hope it provides another frame of reference. On Fri, Nov 26, 2021 at 5:49 PM Adam Lancaster <a...@a-corp.co.uk> wrote: > I think the point is that an exclusion group needs to be managed manually, > which means if you add a new module you have to check every test to see if > the module needs adding to an exclusion group there... which seems worse > than just making the test async: false. > > Worth pointing out (you may or may not know) you can pull your synchronous > tests into their own module so the absolute minimum number of tests are > running async which seems to be what you are after? Also that module can be > in the same test file as a test module with async tests inside them. > > Best > > Adam > > On 26 Nov 2021, at 16:42, Paul Dann <pdgid...@gmail.com> wrote: > > On Fri, 26 Nov 2021 at 14:24, José Valim <jose.va...@dashbit.co> wrote: > >> Right, my concern is not about the existing tests, which indeed need to >> revert, but future changes. Taking the application environment example, now >> anyone in the future that adds a feature that reads from that environment >> or writes a test that reads from that environment, they need to remember to >> annotate their new test case to exclude a completely unrelated file, that >> may have been written 1 week, 1 month, or 1 year ago. >> > > I think you're saying that in the Application.put_env() case, the safe > option is to always mark a test file async: false as soon as global state > such as this is used, and then future tests are isolated from this test as > well. And that's true; that probably remains the safest approach. But maybe > in that situation, when you're about to switch async: true to false in your > new test file, a second alternative is to check for existing uses of the > same global state in the test suite, and adding an exclusion group instead. > > If code is refactored to use affected global state, it shouldn't take too > long for the race conditions to start appearing in the CI, at which point > in my experience it's not too difficult to track down where global state is > used in that test file. I think that can pretty easily happen to regular > async tests too. The time saved in test suite runs can outweigh the pain of > diagnosing a false-failure, I think, depending on the project. I'm > certainly not saying this is the best way for all projects - it requires > care, much like running Ecto in async tests requires care and a good > understanding of the problem. > > Paul > > > > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/CALZj-Vp2YZRooJtB2oCd4HkHwHFBL9kXb6VapZFRPGV8j89XBw%40mail.gmail.com > <https://groups.google.com/d/msgid/elixir-lang-core/CALZj-Vp2YZRooJtB2oCd4HkHwHFBL9kXb6VapZFRPGV8j89XBw%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > > -- > You received this message because you are subscribed to the Google Groups > "elixir-lang-core" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to elixir-lang-core+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elixir-lang-core/0E0554D5-3C7D-4650-B77B-53B2A158DC26%40a-corp.co.uk > <https://groups.google.com/d/msgid/elixir-lang-core/0E0554D5-3C7D-4650-B77B-53B2A158DC26%40a-corp.co.uk?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K2SSn-QK4WY0beTs%2Baw%3DkC79p%2BHm13p2tZMrc1oo7jcg%40mail.gmail.com.
