[issue15496] harden directory removal for tests on Windows

Changes by Brian Curtin : -- assignee: -> brian.curtin components: +Windows resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker ___

[issue15496] harden directory removal for tests on Windows

Roundup Robot added the comment: New changeset c863dadc65eb by Brian Curtin in branch '2.7': Fix #15496. Add directory removal helpers to make Windows tests more reliable. Patch by Jeremy Kloth http://hg.python.org/cpython/rev/c863dadc65eb -- ___ Pyt

[issue15496] harden directory removal for tests on Windows

Roundup Robot added the comment: New changeset fcad4566910b by Brian Curtin in branch '3.2': Fix #15496. Add directory removal helpers to make Windows tests more reliable. Patch by Jeremy Kloth http://hg.python.org/cpython/rev/fcad4566910b -- nosy: +python-dev _

[issue15496] harden directory removal for tests on Windows

Tim Golden added the comment: Fine with me -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pytho

[issue15496] harden directory removal for tests on Windows

Brian Curtin added the comment: The latest patch to test.support looks reasonable. Go for it. -- ___ Python tracker ___ ___ Python-bug

[issue15496] harden directory removal for tests on Windows

Antoine Pitrou added the comment: Brian, Tim, do you think this should be committed? -- versions: -Python 3.4 ___ Python tracker ___

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: > Not that some change isn't necessary, but what else are you running on your > build slave? I ran a Windows 2008 R2 x64 slave for some time and it never had > issues around file/directory removal. I only had to decommission it because > the physical machine bec

[issue15496] harden directory removal for tests on Windows

Brian Curtin added the comment: > Without some change, the Win64 buildbot is relatively irrelevant as it is nearly always in a state of failure due to these errors. Not that some change isn't necessary, but what else are you running on your build slave? I ran a Windows 2008 R2 x64 slave for som

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: With the latest changes, is there anything left preventing the inclusion of this patch? Without some change, the Win64 buildbot is relatively irrelevant as it is nearly always in a state of failure due to these errors. --

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: I've updated the comment in the patch to reflect Martin's concern. Martin is partially correct in that the handle opened in the stat() call will not prolong the pending status. It is due to the fact that it does not open the handle with any sharing mode set, th

[issue15496] harden directory removal for tests on Windows

Martin v. Löwis added the comment: I fail to see the point of not using os.stat. IIUC, Windows will delete the file once the last handle is been closed. Since stat will close any handle it temporarily gets, it will not prolong the live of the file; the file will still go away when the last pro

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: OK, here is another patch that uses just os.listdir -- Added file: http://bugs.python.org/file26663/support.diff ___ Python tracker ___ __

[issue15496] harden directory removal for tests on Windows

Antoine Pitrou added the comment: > > I wonder why it couldn't use os.listdir to find out whether the directory > > is empty, and os.stat to find out whether a specific file or directory > > still exists. > > It is possible to do the same thing with just os.listdir. The use of > the Find*File

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: > I wonder why it couldn't use os.listdir to find out whether the directory is > empty, and os.stat to find out whether a specific file or directory still > exists. It is possible to do the same thing with just os.listdir. The use of the Find*File functions was

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: > Tim Golden added the comment: > I'm tentative about using ctypes this just because I don't believe we use it > anywhere else. ctypes is already used later in test_support so I figured it was fine to use for this as well. -- ___

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: > What are those programs exactly? A buildbot should ideally have a lean system > install that does not interfere with the tests running. My development machine has add'l programs, not the buildbot machine. Sorry is there was any confusion. I get the same occasi

[issue15496] harden directory removal for tests on Windows

Antoine Pitrou added the comment: > On my machine I have programs that simply do not have an option to > ignore any directories thus causing some grief during testing. What are those programs exactly? A buildbot should ideally have a lean system install that does not interfere with the tests ru

[issue15496] harden directory removal for tests on Windows

Martin v. Löwis added the comment: I wonder why it couldn't use os.listdir to find out whether the directory is empty, and os.stat to find out whether a specific file or directory still exists. -- ___ Python tracker

[issue15496] harden directory removal for tests on Windows

Nick Coghlan added the comment: My inclination is to say that using ctypes is a reasonable option for improving Windows buildbot stability in the near term, but we'd probably want to move this into _winapi long term. Adding Antoine & MvL to get their opinion. -- nosy: +loewis, pitrou

[issue15496] harden directory removal for tests on Windows

Tim Golden added the comment: I'm +1 on the approach in principle. I'm tentative about using ctypes for this just because I don't believe we use it anywhere else. But at the least I suggest applying the patch to see how Jeremy's buildbot behaves. If there are wider objections to ctypes we could a

[issue15496] harden directory removal for tests on Windows

Changes by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue15496] harden directory removal for tests on Windows

Changes by Chris Jerdonek : -- nosy: +cjerdonek ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyt

[issue15496] harden directory removal for tests on Windows

Changes by Antoine Pitrou : -- nosy: +brian.curtin stage: -> patch review versions: +Python 2.7, Python 3.2 ___ Python tracker ___ __

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: I must also add that the proposed solution works well within the test suite as the access denied error can also occur when creating subsequent files, not just removing them. This solution eliminates the need to wrap all creation calls with access denied handlin

[issue15496] harden directory removal for tests on Windows

Jeremy Kloth added the comment: > This is a (near) duplicate of issue7443, I think. Partially so it seems. However, my testing with Process Monitor (from sysinterals) shows that most of the access denied errors occur when removing directories. The blind rename-remove dance doesn't work when re

[issue15496] harden directory removal for tests on Windows

Tim Golden added the comment: This is a (near) duplicate of issue7443, I think. -- nosy: +tim.golden ___ Python tracker ___ ___ Python

[issue15496] harden directory removal for tests on Windows

New submission from Jeremy Kloth: Currently, removing directories during testing on Windows w/NTFS can causing sporadic failures due to access denied errors. This is caused by other processes getting a handle to the directory itself (change notifications) or a handle to a file within the dire