On Thu, Jul 17, 2014 at 9:52 AM, Pawel Jasinski <pawel.jasin...@gmail.com> wrote:
> First of all, there is a need to document where the tests are and how > to run them. It took me a moment to figure it out, so it make sense to > share it. > > The test config: IronLanguages/Test/IronPython.tests > An individual test can be started by: ./test-ipy.cmd > /test:test_zipimport_cpy > > I remember at one time compiling TestRunner. > IronLanguages/Test/TestRunner/TestRunner.sln, so if something doesn't > work, try to build it. > > The body of the tests is located in many places: > IronLanguages/Languages/IronPython/Tests > IronLanguages/External.LCA_RESTRICTED/Languages/IronPython/27/Lib/test > > > In addition to ./test-ipy.cmd the test from stdlib can be run with a usual: > ipy -X:FullFrames -m unittest test.test_zipimport_support > > I also had 'nose' running at some point, so it is likely to be still ok. > Thanks for writing this down! It could go on the github wiki as well, but at least search engines are likely to find it. > > > Now the second part, what is not so nice about our test framework and > perhaps how can we avoid the same mistakes in "3". > > As Jeff refers to it: "fabulously complicated test infrastructure" > (FCTI?), is a bit of inconvenient to use. I suspect the definition of > what individual test constitutes is not uniform. FCTI things a test is > entire file or even worse a package. I found no way to test a single > function out of a test file without reconfiguring or modifying a lot. > Yep. It's a legacy of the system used at MS (which I believe was called SNAP); the current TestRunner was the simplest thing Dino could put together to get it working when the code was permanently moved out of MS. > > For example: > test_stdconsole.py takes ~3 minutes. But a test where it fails 12 seconds. > so I nuke (or comment out) all other test functions in > test_stdconsole.py to get to fast test cycle. > > Second example: > modules_io_related_ipy is a single test, but it includes not only all > functions out of a file but entire package (all files out of > IronLanguages/Languages/IronPython/Tests/modules/io_related). Here I > basically remove all not interesting files and functions. > > If anybody has a better way, please share. > My thinking on testing for 3 can be found at https://github.com/jdhardy/ironpython3/commits/test-runner. The main reason I want to use NUnit (or xUnit.Net) to drive it is that they support multiple platforms, and it saves us having to write test runners everywhere (keeping in mind things like Android, iOS, Win8, etc.). Doing so, though, involves rewriting the discovery portions of both iptest and the CPython test suite, but at least we only have to do it once. Then the tests can be exposed to any NUnit-capable runner. Nothing to help much in the short term, but longer term things will get better. - Jeff > > --pawel > _______________________________________________ > Ironpython-users mailing list > Ironpython-users@python.org > https://mail.python.org/mailman/listinfo/ironpython-users >
_______________________________________________ Ironpython-users mailing list Ironpython-users@python.org https://mail.python.org/mailman/listinfo/ironpython-users
