On Wednesday, 14 June 2023 at 02:36:16 UTC+8 Matthias Koeppe wrote:
- Modularized distributions must be testable separately! And why is this a desirable goal? Integration tests almost by definition are testing multiple modules/units and thus don't fit into this scheme. Admittedly, the only multi-package monorepos I know are in the javascript world but there people usually don't bother to modularize their tests for exactly this reason (e.g. https://github.com/vuejs/vue/tree/main/test). Do you know of examples of projects that follow the "Modularized distributions must be testable separately!"-mantra? My thought was that when you declare a new module, you run the doctests of that module. They usually break for two reasons: a) some real dependency was not declared for that module - in this case, add the dependency and run the tests again. b) some doctests require a library that is not a dependency of the original module - in this case, add an optional tag to the failing doctest (current solution) My proposal would be in b) to add it the offending import to a whitelist ("test dependency of module xyz") and then all doctests for module xyz are ignoring import errors for these whitelisted dependencies. So in short, instead of annotating each doctest you annotate the module. After you whitelist an import from *M* because there is some valid doctest that uses *M* for integration-testing purposes (for example, by testing basic linear algebra functionality by running it on an element of an algebra defined in *M*), the modularized distribution is no longer guarded by its testsuite against modularization regressions that involve the whitelisted module. Sure, this hides issues from libraries that are declared to be test dependencies but that actually are/become actual dependencies. But the same problem, on a slightly smaller scale, you have with file-wide "optional" annotations - and even with line annotations. One way out would be to analyze the stack trace and see if the error is thrown in a pure doctest code or while running code in module M. But I think it would be better to reuse tools (or at least their ideas) that were developed exactly to manage and restrict dependencies between different layers/modules of an application, such as https://github.com/zyskarch/pytestarch, https://github.com/seddonym/import-linter, https://github.com/jwbargsten/pytest-archon, https://github.com/TNG/ArchUnit. All of them define external rules about admissible relationships. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/c18fb2bb-2644-49f2-8ea0-ca1c25642d35n%40googlegroups.com.
