Re: [Python-Dev] devguide: Issue #18871: make it more explicit that the test suite should be run before
On Wed, 28 Aug 2013 20:05:25 -0400 Terry Reedy wrote: > On 8/28/2013 5:51 PM, antoine.pitrou wrote: > > > +Does the test suite still pass? > > There are several assumptions packed in that question. > > 0. The test suite is relevant to the patch. > > Not true for doc patches, some idlelib patches, and probably some others. That was implicit in the formulation: "You must :ref:`run the whole test suite ` to ensure that it passes before pushing any *code changes*" (not doc changes, spelling corrections, etc.). The only reason to know that the test suite isn't relevant to a code change is to... *run the test suite*. Don't assume that you are omniscient and know that your change won't break something. Developers not being omniscient is why we have a test suite in the first place. (btw, if idlelib isn't tested, it should!) > 1. The test suite runs to completion. > > For at least a couple of years, that was not true on Windows desktops. > The popup Runtime Error boxes from now are now gone, but a month ago, a > crashing test froze Command Prompt. But was the test corrected? Did you open an issue at the time? Did you try to debug the error? We are collectively responsible for Python's quality. This is not some third-party software you have no control on. > 2. The test suite runs without error. > > I have hardly ever seen this on Windows, ever (with sporadic runs over > several years). Today, test_email and test_sax failed in successive > runs*. > > 3. If there is a new failure, it is due to the patch. > > There have been at least one, but I think more, intermittent failures of > Windows tests in the last few months Same answer as above. > 4. The gain of answering that question is worth the cost. Accepting responsibility for one's own changes is part of why we trust each others as committers. The cost of *you* running the test suite is smaller than the cost of other developers trying to investigate a sudden buildbot failure, where it comes from etc. Having you run the test suite encourages you to be conscious of its existence, of its imperfections, and to feel responsible in making it better. Shrugging it off because it sometimes doesn't work isn't helpful. We are *striving* to make it better (both in coverage and in robustness). > I worry that further emphasizing an overly broad, time-consuming, and > sometimes impractical rule will only discourage more participation. IMO, this is a calculated risk that is worthwhile to take. Times have changed, and being rigorous with testing is central in most successful projects nowadays. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a new tracemalloc module to trace memory allocations
2013/8/29 Victor Stinner : > My proposed implementation for Python 3.4 is different: > > * no enable() / disable() function: tracemalloc can only be enabled > before startup by setting PYTHONTRACEMALLOC=1 environment variable > > * traces (size of the memory block, Python filename, Python line > number) are stored directly in the memory block, not in a separated > hash table > > I chose PYTHONTRACEMALLOC env var instead of enable()/disable() > functions to be able to really trace *all* memory allocated by Python, > especially memory allocated at startup, during Python initialization. I'm not sure that having to set an environment variable is the most convinient option, especially on Windows. Storing traces directly into memory blocks should use less memory, but it requires to start tracemalloc before the first memory allocation. It is possible to add again enable() and disable() methods to dynamically install/uninstall the hook on memory allocators. I solved this issue in the current implementation by using a second hash table (pointer => trace). We can keep the environment variable as PYTHONFAULTHANDLER which enables faulthandler at startup. faulthandler has also a command line option: -X faulthandler. We may add -X tracemalloc. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Add a new tracemalloc module to trace memory allocations
Hi, Thanks to the PEP 445, it becomes possible to trace easily memory allocations. I propose to add a new tracemalloc module computing the memory usage per file and per line number. It has also a private method to retrieve the location (filename and line number) of a memory allocation of an object. tracemalloc is different than Heapy or PySizer because it is focused on the location of a memory allocation rather that the object type or object content. I have an implementation of the module for Python 2.5-3.4, but it requires to patch and recompile Python: https://pypi.python.org/pypi/pytracemalloc My proposed implementation for Python 3.4 is different: * reuse the PEP 445 to hook memory allocators * use a simple C implementation of an hash table called "cfuhash" (coming from the libcfu project, BSD license) instead of depending on the glib library. I simplified and adapted cfuhash for my usage * no enable() / disable() function: tracemalloc can only be enabled before startup by setting PYTHONTRACEMALLOC=1 environment variable * traces (size of the memory block, Python filename, Python line number) are stored directly in the memory block, not in a separated hash table I chose PYTHONTRACEMALLOC env var instead of enable()/disable() functions to be able to really trace *all* memory allocated by Python, especially memory allocated at startup, during Python initialization. The (high-level) API should be reviewed and discussed. The most interesting part is to take "snapshots" and compare snapshots. The module can load snapshots from disk and compare them later for deeper analysis (ex: ignore some files). For the documentation, see the following page: https://pypi.python.org/pypi/pytracemalloc I created the following issue to track the implementation: http://bugs.python.org/issue18874 The implementation: http://hg.python.org/features/tracemalloc * * * I also created a "pyfailmalloc" project based on the PEP 445 to inject MemoryError exceptions. I used this module to check if Python handles correctly memory allocation failures. The answer is no, I fixed many bugs (see issue #18408). Project homepage: https://bitbucket.org/haypo/pyfailmalloc Charles-François Natali and Serhiy Storchaka asked me to add this module somewhere in Python 3.4: "how about adding pyfailmalloc to the main repo (maybe under Tools), with a script making it easy to run the tests suite with it enabled?" What is the best place for such module? Add it to Modules/ directory but as a private module: "_failmalloc"? * * * Example of tracemalloc output (it is more readable with colors, try in a terminal). The first top is sorted by total size, whereas the second top is sorted (automatically) with the size difference. You can see for example that the linecache module likes caching data (1.5 MB after 10 seconds of tests). $ PYTHONTRACEMALLOC=1 ./python -m test ... == CPython 3.4.0a1+ (default:2ce9e5f6b47c+, Aug 29 2013, 02:03:02) [GCC 4.7.2 20121109 (Red Hat 4.7.2-8)] == Linux-3.9.4-200.fc18.x86_64-x86_64-with-fedora-18-Spherical_Cow little-endian == /home/haypo/prog/python/tracemalloc/build/test_python_11087 ... [ 1/380] test_grammar [ 2/380] test_opcodes [ 3/380] test_dict [ 4/380] test_builtin [ 5/380] test_exceptions [ 6/380] test_types [ 7/380] test_unittest 2013-08-29 02:06:22: Top 25 allocations per file and line #1: :704: size=5 MiB, count=56227, average=105 B #2: .../tracemalloc/Lib/linecache.py:127: size=1004 KiB, count=8706, average=118 B #3: .../Lib/unittest/mock.py:1764: size=895 KiB, count=7841, average=116 B #4: .../Lib/unittest/mock.py:1805: size=817 KiB, count=15101, average=55 B #5: .../Lib/test/test_dict.py:35: size=768 KiB, count=8, average=96 KiB #6: :274: size=703 KiB, count=4604, average=156 B #7: ???:?: size=511 KiB, count=4445, average=117 B #8: .../Lib/unittest/mock.py:350: size=370 KiB, count=1227, average=308 B #9: .../Lib/unittest/case.py:306: size=343 KiB, count=1390, average=253 B #10: .../Lib/unittest/case.py:496: size=330 KiB, count=650, average=521 B #11: .../Lib/unittest/case.py:327: size=291 KiB, count=717, average=416 B #12: .../Lib/collections/__init__.py:368: size=239 KiB, count=2170, average=113 B #13: .../Lib/test/test_grammar.py:132: size=195 KiB, count=1250, average=159 B #14: .../Lib/unittest/mock.py:379: size=118 KiB, count=152, average=800 B #15: .../tracemalloc/Lib/contextlib.py:37: size=102 KiB, count=672, average=156 B #16: :1430: size=91 KiB, count=1193, average=78 B #17: .../tracemalloc/Lib/inspect.py:1399: size=79 KiB, count=104, average=784 B #18: .../tracemalloc/Lib/abc.py:133: size=77 KiB, count=275, average=289 B #19: .../Lib/unittest/case.py:43: size=73 KiB, count=593, average=127 B #20: .../Lib/unittest/mock.py:491: size=67 KiB, count=153, average=450 B #21: :1438: size=64 KiB, count=20, average=3321 B #22: .../Lib/unittest/case.py:535: size=56 KiB, count=76, average=766 B #23: .../tracemalloc/Lib/sre_compile.py:508: size=54 KiB, count=115, average=485 B #24: ..
Re: [Python-Dev] Test the test suite?
On 29 Aug 2013 02:34, "Serhiy Storchaka" wrote: > > 28.08.13 14:37, Victor Stinner написав(ла): > >> No, my question is: how can we detect that a test is never run? Do we >> need test covertage on the test suite? Or inject faults in the code to >> test the test suite? Any other idea? > > > Currently a lot of tests are skipped silently. See issue18702 [1]. Perhaps we need a tool which collects skipped and runned tests, compare these sets with sets from a previous run on the same buildbot and reports if they are different. > > [1] http://bugs.python.org/issue18702 Figuring out a way to collect and merge coverage data would likely be more useful, since that could be applied to the standard library as well. Ned Batchelder's coverage.py supports aggregating data from multiple runs. Cheers, Nick. > > > > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18571: Implementation of the PEP 446: file descriptors and file handles
2013/8/28 Antoine Pitrou : > Well, reviewing a 1500-line commit is not very doable. You can use Rietveld if you prefer: http://bugs.python.org/review/18571/#ps9085 The commit is this patch + changes to Misc/NEWS and Doc/whatnews/3.4.rst. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18571: Implementation of the PEP 446: file descriptors and file handles
On Wed, 28 Aug 2013 21:43:00 +0200 Victor Stinner wrote: > 2013/8/28 Antoine Pitrou : > > I don't want to sound too demanding, but was this patch actually > > reviewed? I can't find a single review comment in > > http://bugs.python.org/issue18571 > > No, it was not. The first patch for the PEP 446 (issue #18571) was > available for a review approximatively one month ago. The > implementation of the PEP 433 is very close and the issue #17036 had > patches since january. > > I tested my implementation on many different platforms, I didn't see > any regression. > > You can still review the commit and modify directly the source code if > you would like to change something. You can also use python-dev@ > mailing list if you have comments or questions. Well, reviewing a 1500-line commit is not very doable. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18571: Implementation of the PEP 446: file descriptors and file handles
2013/8/28 Antoine Pitrou : > I don't want to sound too demanding, but was this patch actually > reviewed? I can't find a single review comment in > http://bugs.python.org/issue18571 No, it was not. The first patch for the PEP 446 (issue #18571) was available for a review approximatively one month ago. The implementation of the PEP 433 is very close and the issue #17036 had patches since january. I tested my implementation on many different platforms, I didn't see any regression. You can still review the commit and modify directly the source code if you would like to change something. You can also use python-dev@ mailing list if you have comments or questions. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18571: Implementation of the PEP 446: file descriptors and file handles
Hi, On Wed, 28 Aug 2013 01:20:56 +0200 (CEST) victor.stinner wrote: > http://hg.python.org/cpython/rev/ef889c3d5dc6 > changeset: 85420:ef889c3d5dc6 > user:Victor Stinner > date:Wed Aug 28 00:53:59 2013 +0200 > summary: > Issue #18571: Implementation of the PEP 446: file descriptors and file > handles > are now created non-inheritable; add functions os.get/set_inheritable(), > os.get/set_handle_inheritable() and socket.socket.get/set_inheritable(). I don't want to sound too demanding, but was this patch actually reviewed? I can't find a single review comment in http://bugs.python.org/issue18571 Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test the test suite?
28.08.13 14:37, Victor Stinner написав(ла): No, my question is: how can we detect that a test is never run? Do we need test covertage on the test suite? Or inject faults in the code to test the test suite? Any other idea? Currently a lot of tests are skipped silently. See issue18702 [1]. Perhaps we need a tool which collects skipped and runned tests, compare these sets with sets from a previous run on the same buildbot and reports if they are different. [1] http://bugs.python.org/issue18702 ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepted: PEP 446 -- Make newly created file descriptors non-inheritable
On 28/08/2013 07:29, Paul Moore wrote: On 27 August 2013 23:17, Guido van Rossum mailto:gu...@python.org>> wrote: Thanks for your tiresome work I'm guessing you meant "tireless" here :-) That depends. It might have been tiresome for the one doing it! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Test the test suite?
It happens that few tests are also never run because of name conflicts. See issue 16056. Xavier On Wed, Aug 28, 2013 at 1:37 PM, Victor Stinner wrote: > Hi, > > I just noticed that tests using @requires_freebsd_version and > @requires_linux_version decorator from test.support are never run > since this commit (almost 2 years ago): > > changeset: 72618:3b1859f80e6d > user:Charles-François Natali > date:Mon Oct 03 19:40:37 2011 +0200 > files: Lib/test/support.py > description: > Introduce support.requires_freebsd_version decorator. > > ... > > raise unittest.SkipTest( > -"Linux kernel %s or higher required, not %s" > -% (min_version_txt, version_txt)) > -return func(*args, **kw) > -wrapper.min_version = min_version > +"%s version %s or higher required, not %s" > +% (sysname, min_version_txt, version_txt)) > > > I don't want to blame Charles-François, nobody saw the issue during 2 years! > > No, my question is: how can we detect that a test is never run? Do we > need test covertage on the test suite? Or inject faults in the code to > test the test suite? Any other idea? > > I fixed the decorators in Python 3.3 (84debb4abd50) and 3.4 (f98fd5712b0e). > > Victor > ___ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/xdegaye%40gmail.com -- Xavier Les Chemins de Lokoti: http://lokoti.alwaysdata.net ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepted: PEP 446 -- Make newly created file descriptors non-inheritable
Whoop. Yes. I guess it was me who was tired. :-) On Tuesday, August 27, 2013, Paul Moore wrote: > On 27 August 2013 23:17, Guido van Rossum 'cvml', 'gu...@python.org');> > > wrote: > >> Thanks for your tiresome work > > > I'm guessing you meant "tireless" here :-) > > Paul > -- --Guido van Rossum (on iPad) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Test the test suite?
Hi, I just noticed that tests using @requires_freebsd_version and @requires_linux_version decorator from test.support are never run since this commit (almost 2 years ago): changeset: 72618:3b1859f80e6d user:Charles-François Natali date:Mon Oct 03 19:40:37 2011 +0200 files: Lib/test/support.py description: Introduce support.requires_freebsd_version decorator. ... raise unittest.SkipTest( -"Linux kernel %s or higher required, not %s" -% (min_version_txt, version_txt)) -return func(*args, **kw) -wrapper.min_version = min_version +"%s version %s or higher required, not %s" +% (sysname, min_version_txt, version_txt)) I don't want to blame Charles-François, nobody saw the issue during 2 years! No, my question is: how can we detect that a test is never run? Do we need test covertage on the test suite? Or inject faults in the code to test the test suite? Any other idea? I fixed the decorators in Python 3.3 (84debb4abd50) and 3.4 (f98fd5712b0e). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
