[racket-users] Organizing tests in project
Hi, I'm attempting to organize tests in my package into subfolders/modules instead of having them in a giant main.rkt test submodule, but am running into some issues and was hoping for some advice on the best way to do it. I think the primary issue is related to source compilation order in raco, but am also curious how other people organize their tests. I've moved all of the tests into a tests/ subfolder in the main project tree. When I build the project using raco setup, it builds both the project files as well as the tests contained in the tests/ folder. At this point, if I run the tests as is, they result in an error. If instead I first delete the compiled/ subfolder in the tests folder, the tests then work fine. I think the tests may be getting compiled against the version of the compiled collection which is immediately replaced by a fresh compilation during raco setup. This is the error I'm seeing when I run the tests: default-load-handler: expected a `module' declaration, but found something else file: /Users/siddhartha/work/lisp/racket/relation/tests/compiled/algebraic-test_rkt.dep context...: default-load-handler standard-module-name-resolver module-path-index-resolve module-declared? I could add a make target to clean the test compiled folder prior to running tests, but it seemed like there must be a better way. So my main questions are: 1. Is there a way to exclude certain folders (such as tests) in the raco setup stage? For reference, the command I'm using is raco setup --no-docs --tidy --pkgs relation. 2. Is this a good way to organize tests? Are there any standard recommended ways? Would appreciate any input, -Sid -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/CACQBWF%3D5iwYRCOgwEmwNSh27QzpfE1pjFqvzzB_1KNs3zgHX5g%40mail.gmail.com.
Re: [racket-users] Organizing tests
Thank you Matthias This suggestion has enlightened me. I did not figure out a good way to filter tests based on their types. and, Konrad. I have a makefile.rkt to help me (and potential cooperators) build the system. The *scribble/lp* plays an important role in my development process not only because it can produce beautiful test report, but also as a hybrid of natural documentation and formal specification. The entry *scrbl *ensures that all testsuites or testcases are ordered logically. Moreover, to follow the Behavioral driven development (or just "Test First") principle, module sources can keep elegant on their own. This does require some additional work to do, but I think it's worth doing. On Thu, Apr 16, 2015 at 11:28 PM, Matthias Felleisen wrote: > > Use different names for the various test modules. > At the local level, you can use module+ test. For > the integration tests, you may wish to use module+ > integration. Then run raco test with the parameter > that takes the name of the submodule you want. (If > you really want to run unit tests together with > integration tests, import the former into the latter.) > > -- Matthias > > > > On Apr 16, 2015, at 11:17 AM, Konrad Hinsen wrote: > > > Hi everyone, > > > > I want to put some order into my tests, but after looking at various > > published Racket packages, I am not sure if there any accepted "best > > practices". > > > > For a single module, the best approach seems to be a submodule "test" > > for the test cases, which are then run by "raco test". That's nice and > > works fine. > > > > But what I have is a collection with multiple modules. Each modules > > has local tests, but there are also tests at a higher level that use > > code from several modules. > > > > I am looking for an approach that lets me run either all tests in my > > collection, or convenient subsets (e.g. one module), ideally using a > > single tool such as "raco test". Any suggestions? You get bonus points > > for solutions that integrate well with racket-mode in Emacs. > > > > Thanks in advance, > > Konrad. > > > > -- > > You received this message because you are subscribed to the Google > Groups "Racket Users" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to racket-users+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Organizing tests
That's the --submodule or -s flag. I did `raco help test` just now and discovered even more options than I remembered. Including fun things like running tests in parallel, printing a summary table, and so on. racket-mode has a couple features related to tests and coverage. They currently assume the module is `test`. That's been sufficient for me so far, and I handle more "exotic" test things in a Makefile. But feel free to ping me on GitHub Issues if there are enhancements you'd use. On Thu, Apr 16, 2015 at 11:28 AM, Matthias Felleisen wrote: > > Use different names for the various test modules. > At the local level, you can use module+ test. For > the integration tests, you may wish to use module+ > integration. Then run raco test with the parameter > that takes the name of the submodule you want. (If > you really want to run unit tests together with > integration tests, import the former into the latter.) > > -- Matthias > > > > On Apr 16, 2015, at 11:17 AM, Konrad Hinsen wrote: > >> Hi everyone, >> >> I want to put some order into my tests, but after looking at various >> published Racket packages, I am not sure if there any accepted "best >> practices". >> >> For a single module, the best approach seems to be a submodule "test" >> for the test cases, which are then run by "raco test". That's nice and >> works fine. >> >> But what I have is a collection with multiple modules. Each modules >> has local tests, but there are also tests at a higher level that use >> code from several modules. >> >> I am looking for an approach that lets me run either all tests in my >> collection, or convenient subsets (e.g. one module), ideally using a >> single tool such as "raco test". Any suggestions? You get bonus points >> for solutions that integrate well with racket-mode in Emacs. >> >> Thanks in advance, >> Konrad. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Racket Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to racket-users+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Organizing tests
Thanks to Matthias, John, and Greg for their suggestions! I explored the raco documentation in more detail and I will try to come up with some good solution. My main goal is to have convenient modes of operation: 1) Developers (currently me) should be able to run easily the subset of tests that is relevant for their current work. 2) Users and occasional contributors should be able to run easily the entire test suite of my collection/library. Matthias Felleisen writes: > Use different names for the various test modules. > At the local level, you can use module+ test. For > the integration tests, you may wish to use module+ > integration. Given that I frequently split modules as my code base grows, I'd like to avoid having integration tests in any one module, and prefer a specific module for the tests (that doesn't change). However, it just occurred to me that nothing prevents me from having a module that is empty except for a test submodule. I'll see if that works for me. John Clements writes: > The ‘raco test’ tool can be applied to a collection, using the > “—collection” (or simply “-c”) flag. You can see a full list of the > available flags by running > > raco help test True, but it isn't terribly clear what "interpreting arguments as collections" means. The Racket documentation is a bit more precise in saying that a collection is treated like a dictionary containing modules, so it's just another way to specify a directory path. Greg Hendershott writes: > I did `raco help test` just now and discovered even more options than > I remembered. Including fun things like running tests in parallel, > printing a summary table, and so on. It's indeed quite feature-rich. > racket-mode has a couple features related to tests and coverage. They > currently assume the module is `test`. That's been sufficient for me > so far, and I handle more "exotic" test things in a Makefile. But feel I think it will be sufficient for me as well. In racket-mode, I mainly want to run one module's unit tests. If I can integrate them into a global test-running scheme with raco, that's all I expect to need. Konrad -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Organizing tests
On Apr 16, 2015, at 3:17 AM, Konrad Hinsen wrote: > I am looking for an approach that lets me run either all tests in my > collection, or convenient subsets (e.g. one module), ideally using a > single tool such as "raco test". Any suggestions? You get bonus points > for solutions that integrate well with racket-mode in Emacs. Your wish is granted! (If I understand you correctly.) The ‘raco test’ tool can be applied to a collection, using the “—collection” (or simply “-c”) flag. You can see a full list of the available flags by running raco help test Hope I understood you correctly, John Clements -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [racket-users] Organizing tests
Use different names for the various test modules. At the local level, you can use module+ test. For the integration tests, you may wish to use module+ integration. Then run raco test with the parameter that takes the name of the submodule you want. (If you really want to run unit tests together with integration tests, import the former into the latter.) -- Matthias On Apr 16, 2015, at 11:17 AM, Konrad Hinsen wrote: > Hi everyone, > > I want to put some order into my tests, but after looking at various > published Racket packages, I am not sure if there any accepted "best > practices". > > For a single module, the best approach seems to be a submodule "test" > for the test cases, which are then run by "raco test". That's nice and > works fine. > > But what I have is a collection with multiple modules. Each modules > has local tests, but there are also tests at a higher level that use > code from several modules. > > I am looking for an approach that lets me run either all tests in my > collection, or convenient subsets (e.g. one module), ideally using a > single tool such as "raco test". Any suggestions? You get bonus points > for solutions that integrate well with racket-mode in Emacs. > > Thanks in advance, > Konrad. > > -- > You received this message because you are subscribed to the Google Groups > "Racket Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to racket-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[racket-users] Organizing tests
Hi everyone, I want to put some order into my tests, but after looking at various published Racket packages, I am not sure if there any accepted "best practices". For a single module, the best approach seems to be a submodule "test" for the test cases, which are then run by "raco test". That's nice and works fine. But what I have is a collection with multiple modules. Each modules has local tests, but there are also tests at a higher level that use code from several modules. I am looking for an approach that lets me run either all tests in my collection, or convenient subsets (e.g. one module), ideally using a single tool such as "raco test". Any suggestions? You get bonus points for solutions that integrate well with racket-mode in Emacs. Thanks in advance, Konrad. -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
