[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Change by Giampaolo Rodola' : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Giampaolo Rodola' added the comment: New changeset 65c92c5870944b972a879031abd4c20c4f0d7981 by Giampaolo Rodola (Bruno P. Kinoshita) in branch '3.8': [3.8] bpo-38688, shutil.copytree: consume iterator and create list of entries to prevent infinite recursion (GH-17397)

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Change by Bruno P. Kinoshita : -- pull_requests: +16878 stage: commit review -> patch review pull_request: https://github.com/python/cpython/pull/17397 ___ Python tracker ___

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Change by Giampaolo Rodola' : -- stage: patch review -> commit review versions: +Python 3.9 ___ Python tracker ___ ___

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Giampaolo Rodola' added the comment: New changeset 9bbcbc9f6dfe1368fe7330b117707f828e6a2c18 by Giampaolo Rodola (Bruno P. Kinoshita) in branch 'master': bpo-38688, shutil.copytree: consume iterator and create list of entries to prevent infinite recursion (GH-17098)

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Bruno P. Kinoshita added the comment: Done. Rebased on master too, and edited commit message & GH PR title. Thanks Giampaolo! -- ___ Python tracker ___

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Giampaolo Rodola' added the comment: PR-17098 as it stands re-introduces some stat() syscall. I suggest to just consume the iterator: it's a small change and it should fix the issue. -- ___ Python tracker

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Bruno P. Kinoshita added the comment: I really liked that improvement, and didn't think it needed to be removed. That's why the PR reverts it partially. I think the os.stat improvements were in the other methods changed, and should not be changed in my PR - unless I changed it by accident.

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Giampaolo Rodola' added the comment: The speedup introduced in issue33695 is mostly because the number of os.stat() syscall was reduced from 6 to 1 per file (both by using scandir() and because stat() results are cached and passed around between function calls). As such, even if we

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Bruno P. Kinoshita added the comment: Hi Giampaolo, I think it is more or less the same as the previous code, which was using os.list to return a list in memory. My first tentative fix was: def copytree(src, ...): entries = os.list(src) return _copytree(entries=entries,

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Giampaolo Rodola' added the comment: PR-17098 basically reverts https://bugs.python.org/issue33695. Not good. =) I think we can simply consume the iterator immediately as in: def copytree(src, ...): with os.scandir(src) as itr: entries = list(itr) return

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Bruno P. Kinoshita added the comment: Hi, I did a quick `git bisect` using the example provided, and looks like this regression was added in the fix for bpo-33695, commit 19c46a4c96553b2a8390bf8a0e138f2b23e28ed6. It looks to me that the iterator returned by with os.scandir(...) is

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Change by Bruno P. Kinoshita : -- keywords: +patch pull_requests: +16604 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/17098 ___ Python tracker

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Change by Terry J. Reedy : -- keywords: +3.8regression stage: -> needs patch type: crash -> behavior ___ Python tracker ___ ___

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

Change by STINNER Victor : -- nosy: +giampaolo.rodola ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38688] Python 3.8 regression: endless loop in shutil.copytree

New submission from Christian Boltz : The following test script works with Python 3.7 (and older), but triggers an endless loop with Python 3.8: #!/usr/bin/python3 import shutil import os os.mkdir('/dev/shm/t') os.mkdir('/dev/shm/t/pg') with open('/dev/shm/t/pg/pol', 'w+') as f: