[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Guido van Rossum]

Change by Guido van Rossum :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Guido van Rossum]

Guido van Rossum  added the comment:


New changeset bd2fa3c416ffe6107b500a2180fa1764645c0a61 by CAM Gerlach in branch 
'master':
bpo-29982: Add "ignore_cleanup_errors" param to tempfile.TemporaryDirectory() 
(GH-24793)
https://github.com/python/cpython/commit/bd2fa3c416ffe6107b500a2180fa1764645c0a61


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[C.A.M. Gerlach]

C.A.M. Gerlach  added the comment:

Thanks! Submitted as PR #24793 https://github.com/python/cpython/pull/24793

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[C.A.M. Gerlach]

Change by C.A.M. Gerlach :


--
keywords: +patch
pull_requests: +23560
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/24793

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Guido van Rossum]

Guido van Rossum  added the comment:

SGTM

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[C.A.M. Gerlach]

C.A.M. Gerlach  added the comment:

In addition to transient failures, this can also occur when, for example, files 
opened in the temporary directory (perhaps by library or application code not 
under direct control of the caller) haven't been properly cleaned up and their 
file handles don't get closed, resulting in permissions errors trying to delete 
them (particularly on platforms like Windows, that automatically lock files 
when opening them).

This case came up in e.g. [this recent 
PR](https://github.com/regebro/pyroma/pull/57) and rendered 
`tempfile.TemporaryDirectory` unusable for such use cases, forcing a reversion 
to the lower-level `tempfile.mkdtemp` without the cleaner, more robust and 
easier to interpret high-level interface that the former provides. Wrapping a 
`with` statement in a try-finally is syntactically ugly and semantically 
incongruous, and requires a second shutil.rmtree(..., ignore_errors=True)` call 
to clean up in a best-effort manner, while when manually calling `cleanup()` in 
a try-except, the finalizer still gets executed at a a non-deterministic later 
time when Python exits, raising an error.

Therefore, in the spirit of Guido's statements above in terms of providing a 
"best-effort" cleanup, I propose (and am willing to submit a PR to implement) 
adding an `ignore_errors` bool parameter (defaulting to False, of course, for 
backward compat--and should it be keyword only like `errors` to 
`TemporaryFile`?) to the `tempfile.TemporaryDirectory` constructor, which gets  
passed to `shutil.rmtree` on cleanup. This would not only address both cases 
here, but also one of the two discussed by Anthony Sotitle on 
[BPO-25024](https://bugs.python.org/issue25024), in a cleaner and simpler 
fashion that would take advantage of existing `tempfile.TemporaryDirectory` 
functionality and behavior.

Would a PR be accepted on this? If so, any specific guidance on tests and 
whether to mention it in What's New, etc., would be appreciated. Thanks!

--
nosy: +CAM-Gerlach

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Guido van Rossum]

Guido van Rossum added the comment:

A simpler approach would be to simply ignore the error when it occurs in 
TemporaryDirectory._cleanup. There are no absolute guarantees about tempfile 
always cleaning up -- just a best effort. The complexity (and potential delays) 
of a retry loop seem more risky than simply occasionally not cleaning up -- 
that happens anyways, for a variety of reasons.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Eryk Sun]

Changes by Eryk Sun :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29982] tempfile.TemporaryDirectory fails to delete itself

[Max]

New submission from Max:

There's a known issue with `shutil.rmtree` on Windows, in that it fails 
intermittently. 

The issue is well known 
(https://mail.python.org/pipermail/python-dev/2013-September/128353.html), and 
the agreement is that it cannot be cleanly solved inside `shutil` and should 
instead be solved by the calling app. Specifically, python devs themselves 
faced it in their test suite and solved it by retrying delete.

However, what to do about `tempfile.TemporaryDirectory`? Is it considered the 
calling app, and therefore should retry delete when it calls `shutil.rmtree` in 
its `cleanup` method?

I don't think `tempfile` is protected by the same argument that `shutil.rmtree` 
is protected, in that it's too messy to solve it in the standard library. My 
rationale is that while it's very easy for the end user to retry 
`shutil.rmtree`, it's far more difficult to fix the problem with 
`tempfile.TempDirectory` not deleting itself - how would the end user retry the 
`cleanup` method (which is called from `weakref.finalizer`)?

So perhaps the retry loop should be added to `cleanup`.

--
components: Library (Lib)
messages: 291130
nosy: max
priority: normal
severity: normal
status: open
title: tempfile.TemporaryDirectory fails to delete itself
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com