[issue19643] shutil rmtree fails on readonly files in Windows
Spencer Judge added the comment: Although this is closed, I stumbled across it while looking to see if this behavior had changed at all recently, and I have a suggestion I think might work. How about we take Tim's example error function which was added to the docs, and it's bound to something like shutil.REMOVE_WINDOWS_READONLY, so it can be used in the following way (probably with a better name): shutil.rmtree(C:/readonlyfilesinhere, onerror=shutil.REMOVE_WINDOWS_READONLY) Good idea? Too weird? -- nosy: +sushisource ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
R. David Murray added the comment: I think it is an interesting idea. Probably worth opening a new enhancement request with the suggestion. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Paul Moore added the comment: Not a bad idea. I would want the implementation to remain in the documentation as well, as code that wants to be portable back to earlier versions of Python will need it. But for code that targets Python 3.5+ only, having it available out of the box is a nice addition. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Tim Golden added the comment: The attached patch adds an example to the shutil documentation showing how to use an onerror handler to reattempt the removal of a read-only file. It's deliberately low-tech and simply removes the attribute and retries. If there's some other obstacle, it will continue to fail as it would have done in any case. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Changes by Tim Golden m...@timgolden.me.uk: -- keywords: +patch Added file: http://bugs.python.org/file35168/issue19643-doc.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Zachary Ware added the comment: Fair point, Paul. Patch looks good to me, Tim, barring a couple of nits pointed out on Rietveld. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Tim Golden added the comment: Thanks, Zach. Updated patch. -- assignee: - tim.golden Added file: http://bugs.python.org/file35170/issue19643-doc.2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Zachary Ware added the comment: LGTM! -- stage: - commit review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Tim Golden added the comment: Thanks. I'll hold off pushing until I've had a chance to run it on a Unix system. I'm not 100% whether it will operate in the same way there. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Roundup Robot added the comment: New changeset 31d63ea5dffa by Tim Golden in branch 'default': Issue19643 Add an example of shutil.rmtree which shows how to cope with readonly files on Windows http://hg.python.org/cpython/rev/31d63ea5dffa New changeset a7560c8f38ee by Tim Golden in branch 'default': Issue19643 Fix whitespace http://hg.python.org/cpython/rev/a7560c8f38ee -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Changes by Tim Golden m...@timgolden.me.uk: -- resolution: - fixed stage: commit review - resolved status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Tim Golden added the comment: Ok, so to move this forward we have essentially two proposals: 1) Add a remove_readonly flag 2) Add a doc example which shows how to use the onerror handler to remove a recalcitrant file. I'm -0.5 on (1) because it feels like Windows-specific clutter; and +0 on (2). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Zachary Ware added the comment: I'm good with just adding an example to the docs, along the lines of Paul's del_rw. I think it would be better to use a more conservative example though, something like: def readonly_handler(rm_func, path, exc_info): if issubclass(exc_info[0], PermissionError) and exc_info[1].winerror == 5: os.chmod(path, stat.S_IWRITE) return rm_func(path) raise exc_info[1] -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Paul Moore added the comment: Checking the exact error could be a bit fragile. I have a feeling I recently saw an issue somewhere with code that stopped working on Python 3.4 because the precise error raised for a read-only file changed. I don't recall where the issue was, unfortunately. It's also worth noting that trapping too broad a set of errors won't actually matter much, because the retry will simply fail again if the actual problem wasn't a read-only file... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Jovik added the comment: This could be at least part of docs; I found that people tend to avoid shutil.rmtree(...) on Windows because of such issues. Some of them call subprocess(rmdir /S /Q path) to get desired behavior. -- nosy: +Jovik ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Paul Moore added the comment:
The most obvious solution would be if the onerror argument allowed for retries.
At the moment, all it can do is report issues, not recover. Suppose that
returning True from onerror meant retry the operation. Then you could do
def set_rw(operation, name, exc):
os.chmod(name, stat.S_IWRITE)
return True
shutil.rmtree('path', onerror=set_rw)
--
nosy: +pmoore
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19643
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
R. David Murray added the comment: See issue 8523 for a discussion of changing the way onerror behaves. I think it is addressing this same use case, but I didn't reread it in detail. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Paul Moore added the comment:
It's similar. But the problem is that it only returns a list of errors, it
doesn't let you address the error *while the rmtree is in progress*.
The key thing is that if you can fix the problem in onerror, you can avoid
needing to restart the whole tree walk, which is the key aspect of rmtree.
As things stand, you can use the set_rw function I showed above, and run the
rmtree twice:
shutil.rmtree('path', onerror=set_rw)
shutil.rmtree('path')
The first run fixes the error and then the second one deletes the remaining
files. But this is clearly inefficient, and makes the limitations of report
errors to the user who can then address them fairly obvious.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19643
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
R. David Murray added the comment: OK, rereading that issue, I disagree with Tarek and I think that the patch on that issue is ill-advised as it looks like it changes behavior in a non-backward-compatible way. If you changed your set_rw onerror handler to a rm_ro_file error handler, would things work? That is, have the handler delete the file? Adding a 'retry' capability is interesting, but would be a non-trivial change in logic, and should be addressed in a new issue, not this one. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Paul Moore added the comment: Looks like that works. At least in my case - I just did def del_rw(action, name, exc): os.chmod(name, stat.S_IWRITE) os.remove(name) shutil.rmtree(path, onerror=del_rw) Something more robust might check if name is a directory and os.rmdir that - I didn't need it for my case though. Thanks. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
New submission from Ivan Radic: shutil.rmtree works nice on Windows until it hits file with read only attribute set. Workaround is to provide a onerror parameter as a function that checks and removes file attribute before attempting to delete it. Can option to delete read_only files be integrated in shutil.rmtree? Example output in In Python 2.7: shutil.rmtree(C:\\2) Traceback (most recent call last): File pyshell#60, line 1, in module shutil.rmtree(C:\\2) File C:\Program Files (x86)\Python.2.7.3\lib\shutil.py, line 250, in rmtree onerror(os.remove, fullname, sys.exc_info()) File C:\Program Files (x86)\Python.2.7.3\lib\shutil.py, line 248, in rmtree os.remove(fullname) WindowsError: [Error 5] Access is denied: 'C:\\2\\read_only_file.txt' Example output in In Python 3.3: shutil.rmtree(C:\\2) Traceback (most recent call last): File pyshell#1, line 1, in module shutil.rmtree(C:\\2) File C:\Program Files (x86)\Python.3.3.0\lib\shutil.py, line 460, in rmtree return _rmtree_unsafe(path, onerror) File C:\Program Files (x86)\Python.3.3.0\lib\shutil.py, line 367, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File C:\Program Files (x86)\Python.3.3.0\lib\shutil.py, line 365, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 5] Access is denied: 'C:\\2\\read_only_file.txt' -- components: IO messages: 203285 nosy: ivan.radic priority: normal severity: normal status: open title: shutil rmtree fails on readonly files in Windows type: behavior versions: Python 2.7, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
R. David Murray added the comment: You are essentially asking that we have an option to make the windows behavior mirror the posix behavior? (A read only file in a writable directory can be deleted in posix, since only the directory entry, not the file, is being deleted.) That makes some sense to me. I wonder what the windows devs think? -- components: +Library (Lib), Windows -IO nosy: +r.david.murray, tim.golden, zach.ware type: behavior - enhancement versions: +Python 3.5 -Python 2.7, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Tim Golden added the comment: This, unfortunately, is the classic edge-case where intra-platform consistency and inter-platform consistency clash. I (on Windows) would certainly be surprised if a tree delete removed read-only files without my specifying some kind of override. I understand why cross-platform behaviour might be preferred, and I'm open to being convinced, but I start at -0. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
R. David Murray added the comment: Well, it would *definitely* need to be a new explicit option whose default value was the current behavior. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Ivan Radic added the comment: You are essentially asking that we have an option to make the windows behavior mirror the posix behavior? (A read only file in a writable directory can be deleted in posix, since only the directory entry, not the file, is being deleted.) Actually, your explanation is perfect. I want to be able to remove some directory after I am done using it. When similar operation is done through file manager, dialog pops up asking for confirmation, I would like to have function parameter equivalent of yes to all dialog that file manager gives me. The thing is, anyone working with files is used to think in rm -rf kind of way, and on Windows read_only files break this workflow. I discovered this problem few days ago when I was working on custom backup script that needs to work both on Linux (at home) and Windows (at work). Currently, I need to have some extra *windows only* code just to be able to successfully remove a directory. Quick Google search discovered the workaround (http://stackoverflow.com/questions/1889597/deleting-directory-in-python), so I am set, but the original question: Why oh why is this such a pain? and the comment: Maybe nobody has taken the five minutes to file a bug at bugs.python.org resonated in my head long enough to give it a try. For me it makes sense to have this option configurable. And it make a ton of sense to support one line equivalent of rm -rf. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Zachary Ware added the comment: I like the idea of a remove_readonly flag. I was going to say that I'm a bit worried about the fact that shutil.rmtree already has a couple of keyword arguments, but it's nowhere near what, say, copytree has. Call me +0.75. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
R. David Murray added the comment: That's not a good name for the flag. The problem is that 'read-only' means different things on Windows than it does on Unix. Which is why I suggested that the flag control whether or not it acts posix like on Windows. So perhaps 'posix_compat'? That feels a bit odd, though, since on posix it does nothing...so it's really about behavioral consistency across platforms Hmm. It's really hard to think of a name that conveys succinctly what we are talking about here. A more radical notion would be something like 'delete_control' and have it be tri-valued: 'unixlike', 'windowslike', and 'native', with the default being native. Bad names, most likely, but you get the idea. The disadvantage is that it would be even more code to implement ;) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Tim Golden added the comment: TBH I'm still fairly -0 and edging towards -0.5. If we didn't already have two keyword args I might be convinced towards a jfdi=True flag. But, as the OP described, the current params already allow for a workaround of sorts and another param of the semantics we're discussing would surely just be confusing? (Or you could just preprocess on Windows, eg attrib -r /s) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19643] shutil rmtree fails on readonly files in Windows
Zachary Ware added the comment: I make no claims of being good at naming things :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue19643 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
