Terry J. Reedy added the comment: Attached is a 3.3 patch that I *believe* is ready to commit and push. With my Win7 repository build, I tested running one test from a file (after "if __name__ == '__main__':") or command line ('python_d -m idlelib.PathBrowser'); all idle tests with an interactive interpreter (console or idle shell, see @template.txt for input); all idle tests from the command line ('python_d -m test.test_idle'); and idle tests as part of the CPython suite (python_d -m test). I also tested versions of all but the two batch-mode tests in my 3.3.1 installation.
>From what i know, I do not believe there should be much issue with the >framework working on non-Windows systems. But I would not mind verification. I >presume this patch will work as is with 3.4, but ditto. 2.7 may need one tweak >noted below. (Testing with 2.7 is difficult for me at the moment.) Nick: yes >to your 2.7 plan. PEP 343 changes things. Once applied to all three branches I think this patch is enough to close this issue and 'get the ball rolling'. Mock, gui testing, and any other big problems should be separate issues. The patch adds Itest/__init__.py (merges doc example, part of previous __init__.py) Itest/@template (for both test_x.py and startup from x.py) Itest/test_calltips.py (with first 2 of many tests) Itest/test_pathbrowser.py (revised, with 1 test, see note below) test/test_idle.py (revised skeleton of previous __init__.py) and revises CallTips.py PathBrowser.py to run the corresponding tests when run as '__main__'. Question: "Unittest supports skipping individual test methods and even whole classes of tests" How about skipping (possibly conditionally) an entire file -- especially test_idle, which has no classes, or and test_xxx with multiple classes? For multiple reasons related to the fact that Idle and idlelib *are* special, as recognized by PEP 343, I want to keep the individual test files in an idlelib subdirectory. as with tkinter tests. (I changed the name from 'test', so that 'import test' will a always import Lib/test.) * Once in idlelib, changing to Itest is easier than to ../test/test_idle. With 62 .py files in idlelib, we will be opening pairs of files a lot. * Copying code and test files to another directory, such as an installation idlelib/, will be easier. I will be doing that to run with new features and test things in the installation environment. * PEP343 makes most of idlelib/* a private arena. Test/* is a public tree mainly for the buildbots. Tests put there are supposed to pass. In brief, I personally feel much more comfortable mucking around in a private arena with a small public window that can selectively open and closed as needed. * We need an Itest directory anyway for things that do not belong in test/*. @template is an example, and I have in mind a couple of non-buildbot test scripts. We may also want an idle-specific support module, as tkinter has. * Once people install Python so it runs, some still have problems running Idle. They ask a class instructor or someone else; post here, python-list, stackoverflow, or elsewhere; or give up. Sometimes the problem is with tkinter, sometimes with idle, sometimes with their knowledge. A good diagnosis script should save support time and user frustration. Both tkinter and idle tests should be available for this. The Windows installer makes the 17 mb of test/* optional. Other changes from the previous patch: * Use unittest.main to run tests. * Guard test_idle execution with tkinter import, as it is _tkinter, not idlelib that might not be built. I left the guard for idlelib since someone who deletes idlelib/* might forget to delete test/test_idle. * Exceptions raised outside of self.assertXyz (and self.fail) calls are regarded as an error in the test code rather than a failure of the code tested (26.3.4. Organizing test code). So, there being no 'assertNotRaises' context manager, I added "try:...except ...: self.fail()" to test_pathbrowser.py so a failure will be reported as such and not as an error. If there is a better way to do this, let me know. Since 3.x chains exceptions and prints the original traceback, no message argument is needed with self.fail to explain the failure. For 2.7, I believe one should be added. Note: to empirically test that a test fails properly, it must be run with code that does not work properly. This is a problem with writing tests after the fact for code that works properly. I checked all 62 idlelib/*.py files for a test to see what we have to work with: the answer is 'not much'. Less than half the files have a test. All but 2 that do bring up a window and require a human to key, click, and look according to a script that is not provided. (This is not to say that the existing code will not be helpful for some gui tests.) One of the 2 remaining text tests prints multiple pages (a config file) for a person to check. By coincidence, the only automated tests are the ones in CallTips.py, the first file I looked at, last summer. ---------- stage: test needed -> patch review Added file: http://bugs.python.org/file30264/idletests.diff _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue15392> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com