Re: [Python-Dev] Tests failing on Windows with TESTFN
On Wed, Jul 25, 2018 at 8:07 AM, Tim Golden wrote: > One problem is that certain tests use support.TESTFN (a local directory > constructed from the pid) for output files etc. However this can cause > issues on Windows when recreating the folders / files for multiple tests, > especially when running in parallel. > > Here's an example on my laptop deliberately running 3 tests with -j0 which I > know will generate an error about one time in three: > > C:\work-in-progress\cpython>python -mtest -j0 test_urllib2 test_bz2 > test_importlib > > Running Debug|Win32 interpreter... > Run tests in parallel using 6 child processes > 0:00:23 [1/3/1] test_urllib2 failed > test test_urllib2 failed -- Traceback (most recent call last): > File "C:\work-in-progress\cpython\lib\test\test_urllib2.py", line 821, in > test_file > f = open(TESTFN, "wb") > PermissionError: [Errno 13] Permission denied: '@test_15564_tmp' > > Although these errors are both intermittent and fairly easily spotted, the > effect is that I rarely get a clean test run when I'm applying a patch. > > I started to address this years ago but things stalled. I'm happy to pick > this up again and have another go, but I wanted to ask first whether there > was any objection to my converting tests to using tempfile functions which > should avoid the problem? Do you know what's causing the issue on Windows? I thought TESTFN was designed to work for parallel testing, so it would surprise me if there was a problem with it. Alternatively, if TESTFN should be okay, I wonder if it's an issue with another test or tests not cleaning up after itself correctly, in which case it seems like this is an opportunity to track down and fix that issue. Switching to something else would just serve to hide / mask the issue with those other tests. --Chris ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue34221] Any plans to combine collections.OrderedDict with dict
On 7/26/2018 2:15 AM, Raymond Hettinger wrote:
On Jul 25, 2018, at 8:23 PM, INADA Naoki wrote:
On Thu, Jul 26, 2018 at 12:04 PM Zhao Lee wrote:
Since Python 3.7，dicts remember the order that items were inserted, so any
plans to combine collections.OrderedDict with dict?
https://docs.python.org/3/library/collections.html?#collections.OrderedDict
https://docs.python.org/3/library/stdtypes.html#dict
No. There are some major difference.
* d1 == d2 ignores order / od1 == od2 compares order
* OrderedDict has move_to_end() method.
* OrderedDict.pop() takes `last=True` keyword.
In addition to the API differences noted by Naoki, there are also
implementation differences. The regular dict implements a low-cost solution
for common cases. The OrderedDict has a more complex scheme that can handle
frequent rearrangements (move_to_end operations) without touching, resizing, or
reordering the underlying dictionary. Roughly speaking, regular dicts emphasize
fast, space-efficient core dictionary operations over ordering requirements
while OrderedDicts prioritize ordering operations over other considerations.
That said, now that regular dicts are ordered by default, the need for
collections.OrderedDict() should diminish quite a bit. Mostly, I think people
will ignore OrderedDict unless their application heavily exercises move to end
operations.
On python-idea, Miro Hrončok asked today whether we can change the
OrderedDict repr from, for instance,
OrderedDict([('a', '1'), ('b', '2')]) # to
OrderedDict({'a': '1', 'b': '2'})
I am not sure what our repr change policy is, as there is a
back-compatibility issue but I remember there being changes.
--
Terry Jan Reedy
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tests failing on Windows with TESTFN
On Wed, 25 Jul 2018 at 15:16 Tim Golden wrote: > I'm just easing back into core development work by trying to get a > stable testing environment for Python development on Windows. > > One problem is that certain tests use support.TESTFN (a local directory > constructed from the pid) for output files etc. However this can cause > issues on Windows when recreating the folders / files for multiple > tests, especially when running in parallel. > > Here's an example on my laptop deliberately running 3 tests with -j0 > which I know will generate an error about one time in three: > > C:\work-in-progress\cpython>python -mtest -j0 test_urllib2 test_bz2 > test_importlib > > Running Debug|Win32 interpreter... > Run tests in parallel using 6 child processes > 0:00:23 [1/3/1] test_urllib2 failed > test test_urllib2 failed -- Traceback (most recent call last): >File "C:\work-in-progress\cpython\lib\test\test_urllib2.py", line > 821, in test_file > f = open(TESTFN, "wb") > PermissionError: [Errno 13] Permission denied: '@test_15564_tmp' > > Although these errors are both intermittent and fairly easily spotted, > the effect is that I rarely get a clean test run when I'm applying a patch. > > I started to address this years ago but things stalled. I'm happy to > pick this up again and have another go, but I wanted to ask first > whether there was any objection to my converting tests to using tempfile > functions which should avoid the problem? > I personally don't see any reason to block a move away from TESTFN since it obviously has some inherent issues in parallel test running which means flaky tests. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] What's the status of the PEP 572 implementation?
Emily Morehouse is working on it, and I'm helping out. We probably won't get to the PR phase until the core dev sprint in September though. On Thu, Jul 26, 2018 at 2:22 AM, Victor Stinner wrote: > Hi, > > Is the PEP 572 implemented? If no, who is working on that? Is there a WIP > pull request? An open issue? > > One month ago, I tried Chis Angelo's implementation but it implemented an > old version of the PEP which evolved in the meanwhile. > > Victor > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > guido%40python.org > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] What's the status of the PEP 572 implementation?
Hi, Is the PEP 572 implemented? If no, who is working on that? Is there a WIP pull request? An open issue? One month ago, I tried Chis Angelo's implementation but it implemented an old version of the PEP which evolved in the meanwhile. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue34221] Any plans to combine collections.OrderedDict with dict
> On Jul 25, 2018, at 8:23 PM, INADA Naoki wrote: > > On Thu, Jul 26, 2018 at 12:04 PM Zhao Lee wrote: >> >> >> Since Python 3.7，dicts remember the order that items were inserted, so any >> plans to combine collections.OrderedDict with dict? >> https://docs.python.org/3/library/collections.html?#collections.OrderedDict >> https://docs.python.org/3/library/stdtypes.html#dict > > No. There are some major difference. > > * d1 == d2 ignores order / od1 == od2 compares order > * OrderedDict has move_to_end() method. > * OrderedDict.pop() takes `last=True` keyword. In addition to the API differences noted by Naoki, there are also implementation differences. The regular dict implements a low-cost solution for common cases. The OrderedDict has a more complex scheme that can handle frequent rearrangements (move_to_end operations) without touching, resizing, or reordering the underlying dictionary. Roughly speaking, regular dicts emphasize fast, space-efficient core dictionary operations over ordering requirements while OrderedDicts prioritize ordering operations over other considerations. That said, now that regular dicts are ordered by default, the need for collections.OrderedDict() should diminish quite a bit. Mostly, I think people will ignore OrderedDict unless their application heavily exercises move to end operations. Raymond ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
