[issue22107] tempfile module misinterprets access denied error on Windows

Change by bugale bugale : -- nosy: +bugale bugale ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Eryk Sun : -- versions: +Python 3.10, Python 3.9 -Python 2.7, Python 3.5, Python 3.6, Python 3.7 ___ Python tracker ___

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Georg Brandl : -- nosy: -georg.brandl ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Jeffrey Kintscher : -- nosy: -Jeffrey.Kintscher ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Václav Dvořák : -- nosy: +Václav Dvořák ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Tor Colvin : -- nosy: +Tor.Colvin ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Jonathan Mills : -- nosy: +Jonathan Mills ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Jonathan Mills : -- versions: +Python 3.8 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Erik Aronesty added the comment: This is the fist of what I'm using: https://gist.github.com/earonesty/a052ce176e99d5a659472d0dab6ea361 Seems OK for my use cases. There's probably issues with relying on __del__ this way. But it solves the Windows close/reopen problem, too. --

[issue22107] tempfile module misinterprets access denied error on Windows

Erik Aronesty added the comment: i would like to point out that the primary reason any of this nonsense exists is because of short filename restrictions. i've replaces nearly all of my temp file creation code in all of my project to `return os.urandom(32).hex()` ... which is reliable

[issue22107] tempfile module misinterprets access denied error on Windows

And Clover added the comment: Attempting to answer the question "did this open call fail because the path was a directory" by implication from "do we think we ought to be able to write a file to this directory" is IMO doomed. There's no reliable way to determine whether one should be able

[issue22107] tempfile module misinterprets access denied error on Windows

Change by Jeffrey Kintscher : -- nosy: +Jeffrey.Kintscher ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue22107] tempfile module misinterprets access denied error on Windows

Eryk Sun added the comment: CanAccessFolder is incomplete for the following reasons: (1) It doesn't account for SeBackupPrivilege (read and execute access) and SeRestorePrivilege (write and delete access). If a create or open call requests backup semantics, these two privileges are checked

[issue22107] tempfile module misinterprets access denied error on Windows

Erik Aronesty added the comment: Series of operations needed to answer the questions os.access is not answering on windows: bool CanAccessFolder( LPCTSTR folderName, DWORD genericAccessRights ) { bool bRet = false; DWORD length = 0; if (!::GetFileSecurity( folderName,

[issue22107] tempfile module misinterprets access denied error on Windows

Billy McCulloch added the comment: I stand by the patch file I previously submitted on 2016-05-04. A more detailed analysis / description of my reasoning follows. Change 1 in _get_default_tempdir: A PermissionError is thrown on Windows if you attempt to create a file whose filename matches

[issue22107] tempfile module misinterprets access denied error on Windows

Jim Maloy added the comment: This issue persists as of today (March 2019), in Python 3.7.2 (64 bit) running on Windows 10. I gather from the comments that fixing it is no trivial matter, although I don't fully understand why. The hang of "several seconds" that was originally described is

[issue22107] tempfile module misinterprets access denied error on Windows

Changes by Serhiy Storchaka : -- components: +Library (Lib) resolution: fixed -> stage: resolved -> versions: +Python 3.7 -Python 3.4 ___ Python tracker

[issue22107] tempfile module misinterprets access denied error on Windows

Rocco Matano added the comment: I just experienced the described problem using Python 3.6.0 (Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] on win32), but i do not understand the current status of this issue: On the one hand it is marked as 'open', which

[issue22107] tempfile module misinterprets access denied error on Windows

Paul Doom added the comment: Can the _mkstemp_inner portion of Billy McCulloch's patch be applied? Due to a large default os.TMP_MAX value (2147483647 - seems to be the current value on Win 7/8.1/10 I have access to), the following will push the CPU to 100% for a very long time when run

[issue22107] tempfile module misinterprets access denied error on Windows

Eryk Sun added the comment: The Windows API loses information when mapping kernel status values to Windows error codes. For example, the following status values are all mapped to ERROR_ACCESS_DENIED: STATUS_INVALID_LOCK_SEQUENCE 0xc01e STATUS_INVALID_VIEW_SIZE 0xc01f

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Billy, no, the parent directory is tested intentionally. If it isn't a directory, we should fail. May be there is a way to distinguish a failure because the name is already used for a subdirectory from a failure because we don't have permission to create a

[issue22107] tempfile module misinterprets access denied error on Windows

Billy McCulloch added the comment: I've also run into this bug on Windows. In my case, the tempdir path includes directories on a network share, which I lack write access permissions to. Python tries to generate a *lot* of files, and never figures out it should move on to another directory.

[issue22107] tempfile module misinterprets access denied error on Windows

Changes by Serhiy Storchaka : -- status: closed -> open ___ Python tracker ___ ___

[issue22107] tempfile module misinterprets access denied error on Windows

Changes by Mark Lawrence : -- nosy: -BreamoreBoy ___ Python tracker ___ ___

[issue22107] tempfile module misinterprets access denied error on Windows

Thomas Kluyver added the comment: This issue was closed, but I believe the original bug reported was not fixed: trying to create a temporary file in a directory where you don't have write permissions hangs for a long time before failing with a misleading FileExistsError, rather than failing

[issue22107] tempfile module misinterprets access denied error on Windows

eryksun added the comment: My reluctance to commit the os.access patch is because it will cause such a behaviour change in a function which has been pretty stable for a long while. The original behavior could be preserved as the default. A keyword-only argument could enable checking

[issue22107] tempfile module misinterprets access denied error on Windows

Tim Golden added the comment: That is a possibility which hadn't occurred to me. @eryksun, would you mind eyeballing the patch over on issue2582? It almost certainly won't apply cleanly as it's been almost two years since I last refreshed it, but you can hopefully gauge the intent. --

[issue22107] tempfile module misinterprets access denied error on Windows

eryksun added the comment: Ok, I think I understand now. You chose an indirect check to avoid the race condition. If we have write access to the directory, then the PermissionError must be because a directory exists with the same name. Unfortunately os.access doesn't currently tell us this

[issue22107] tempfile module misinterprets access denied error on Windows

Paul Moore added the comment: I'm not sure I follow. Isn't the point of this patch to try again in certain cases of a PermissionError, where currently the code breaks out of the loop early? How can the result be worse than the current behaviour? Suerly sometimes (maybe not always) it works

[issue22107] tempfile module misinterprets access denied error on Windows

Tim Golden added the comment: My reluctance to commit the os.access patch is because it will cause such a behaviour change in a function which has been pretty stable for a long while. It'll certainly be more correct, but at the undoubted expense of breaking someone's long-working code.

[issue22107] tempfile module misinterprets access denied error on Windows

eryksun added the comment: Shouldn't it be checking whether `file` (or `filename`) is a directory [1]? For example: except PermissionError: # This exception is thrown when a directory with # the chosen name already exists on windows. if _os.name == 'nt' and

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: There is a risk of race condition. One process can create a directory `file`, then other process fails to create a file with the same name `file`, then the first process removes directory `file`, then the second process handles PermissionError. The same is

[issue22107] tempfile module misinterprets access denied error on Windows

Paul Moore added the comment: == ERROR: test_read_only_directory (test.test_tempfile.TestMkdtemp) -- Traceback (most recent call last): File

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Thank you Paul. What with updated patch? -- Added file: http://bugs.python.org/file39434/tempfile_bad_tempdir_4.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22107

[issue22107] tempfile module misinterprets access denied error on Windows

Roundup Robot added the comment: New changeset 63f0ae6e218a by Serhiy Storchaka in branch '2.7': Issue #22107: tempfile.gettempdir() and tempfile.mkdtemp() now try again https://hg.python.org/cpython/rev/63f0ae6e218a New changeset 3a387854d106 by Serhiy Storchaka in branch '3.4': Issue #22107:

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Unfortunately the patch doesn't fix original issue and looks as this can't be fixed until os.access will be more useful on Windows. But it fixes several related issues. mkstemp() now fails early if parent directory doesn't exist or is a file. gettempdir()

[issue22107] tempfile module misinterprets access denied error on Windows

Paul Moore added the comment: Works fine with the new patch: .\rt.bat -x64 -q test_tempfile C:\Work\Projects\cpython\PCbuildC:\Work\Projects\cpython\PCbuild\amd64\python.exe -Wd -E -bb C:\Work\Projects\cpython\PCbuild\..\lib\test\regrtest.py test_tempfile [1/1] test_tempfile 1 test OK.

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Added os.path.isdir(). Could anybody please run tests on Windows? -- Added file: http://bugs.python.org/file39421/tempfile_bad_tempdir_3.patch ___ Python tracker rep...@bugs.python.org

[issue22107] tempfile module misinterprets access denied error on Windows

Mark Lawrence added the comment: The feedback here https://mail.python.org/pipermail/python-dev/2011-May/111530.html seems positive. It references #2528 which is still open but strikes me as the way forward. Why don't we go for it and nail this issue once and for all? --

[issue22107] tempfile module misinterprets access denied error on Windows

Roger Upole added the comment: It doesn't actually do anything, so why do it at all? In order to distinguish why it failed, you might try checking if the file actually exists, and if it is a folder. -- ___ Python tracker rep...@bugs.python.org

[issue22107] tempfile module misinterprets access denied error on Windows

Tim Golden added the comment: And, just to be clear to Serhiy who I know doesn't use Windows, os.access really is a worthless function in its present form: worse, even, because it can be misleading. I have a long-standing patch to convert it to use AccessCheck but I've never quite had the guts

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: The main issue is not tempfile raises a FileExistsError, but that it hangs for several seconds (for example if the temp dir doesn't exist). The patch allows to fail early and try other temp dir. os.access() is not enough, we can add os.path.isdir(). Could

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: It is best that we can do. How else we can check filesystem permissions? Only trying to create a file, But we just tried this and it failed. -- ___ Python tracker rep...@bugs.python.org

[issue22107] tempfile module misinterprets access denied error on Windows

Roger Upole added the comment: os.access doesn't check filesystem permissions, so the patch will not catch the condition that creates the problem. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22107

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: What exceptions are raised on Windows when try to open a file in bad directory? open('foo').close() open('foo/bar') # what raised? open('nonexistent/bar') # what raised? If raised the same exceptions as on Linux, then perhaps the following patch

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: And what returns os.access for writable directories and non-existent files? os.mkdir('dir') os.access('dir', os.W_OK) # what returns? os.access('nonexistent', os.W_OK) # what returns or raises? -- ___

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Is there a difference if you do open(..., 'w')? It's a different enough operation that it may have a different error. Oh, yes, I forgot the 'w' mode. Mark, could you please run following test on Windows? import os open('foo', 'wb').close() flags =

[issue22107] tempfile module misinterprets access denied error on Windows

Mark Lawrence added the comment: os.mkdir('dir') os.access('dir', os.W_OK) True os.access('nonexistent', os.W_OK) False open('dir/bar') Traceback (most recent call last): File stdin, line 1, in module FileNotFoundError: [Errno 2] No such file or directory: 'dir/bar'

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Thank you Mark. Could you please make first test in msg236028 (when first part of the path is a file, not a directory)? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22107

[issue22107] tempfile module misinterprets access denied error on Windows

Mark Lawrence added the comment: open('README').close() open('README/bar') Traceback (most recent call last): File stdin, line 1, in module FileNotFoundError: [Errno 2] No such file or directory: 'README/bar' -- ___ Python tracker

[issue22107] tempfile module misinterprets access denied error on Windows

Steve Dower added the comment: Is there a difference if you do open(..., 'w')? It's a different enough operation that it may have a different error. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22107

[issue22107] tempfile module misinterprets access denied error on Windows

Serhiy Storchaka added the comment: Great. There is only one difference between Windows and Linux, but it affects only error type in tests. Here is a patch with updated test. It should now work on Windows. -- Added file: http://bugs.python.org/file38151/tempfile_bad_tempdir_2.patch

[issue22107] tempfile module misinterprets access denied error on Windows

Mark Lawrence added the comment: open('foo', 'wb').close() flags = os.O_RDWR | os.O_CREAT | os.O_EXCL | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_BINARY', 0) os.open('foo/bar', flags, 0o600) Traceback (most recent call last): File stdin, line 1, in module FileNotFoundError: [Errno 2]

[issue22107] tempfile module misinterprets access denied error on Windows

Mark Lawrence added the comment: changeset 035b61b52caa has this:- return (fd, _os.path.abspath(file)) except FileExistsError: continue# try again +except PermissionError: +# This exception is thrown when a directory with the chosen

[issue22107] tempfile module misinterprets access denied error on Windows

Changes by Serhiy Storchaka storch...@gmail.com: -- type: - behavior versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22107 ___

[issue22107] tempfile module misinterprets access denied error on Windows

Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: +georg.brandl, ncoghlan ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22107 ___ ___

[issue22107] tempfile module misinterprets access denied error on Windows

New submission from Roger Upole: _mkstemp_inner assumes that an access denied error means that it has generated a filename that matches an existing foldername. However, in the case of a folder for which you don't have permissions to create a file, this means it will loop thru the maximum