[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Roundup Robot]

Roundup Robot added the comment:

New changeset 767262c149ca by Serhiy Storchaka in branch '3.5':
Issue #25860: os.fwalk() no longer skips remaining directories when error 
occurs.
https://hg.python.org/cpython/rev/767262c149ca

New changeset a85675dabb8f by Serhiy Storchaka in branch 'default':
Issue #25860: os.fwalk() no longer skips remaining directories when error 
occurs.
https://hg.python.org/cpython/rev/a85675dabb8f

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Roundup Robot]

Roundup Robot added the comment:

New changeset 7995a81236b6 by Serhiy Storchaka in branch '3.5':
Issue #25860: Fixed test failure caused by inconsistency of os.walk() and
https://hg.python.org/cpython/rev/7995a81236b6

New changeset dcf9e9ae5393 by Serhiy Storchaka in branch 'default':
Issue #25860: Fixed test failure caused by inconsistency of os.walk() and
https://hg.python.org/cpython/rev/dcf9e9ae5393

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

LGTM. Here is a patch with tests.

--
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka
stage:  -> patch review
versions: +Python 3.6 -Python 3.4
Added file: 
http://bugs.python.org/file41357/fwalk-silently-skips-dirs-when-error-occurs-with-tests.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Samson Lee]

Changes by Samson Lee :


--
nosy: +hynek, neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Samson Lee]

Changes by Samson Lee :


--
nosy: +benhoyt, haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25860] os.fwalk() silently skips remaining directories when error occurs

[Samson Lee]

New submission from Samson Lee:

I noticed os.fwalk() produced different results as os.walk(). It turned out
that os.fwalk() skips all remaining directories when OSError occurs (e.g. due
to PermissionError).

To reproduce the bug, first create a test directory structure:

$ mkdir 1; touch 1/a
$ mkdir 2; touch 2/b
$ mkdir 3; touch 3/c
$ mkdir 4; touch 4/c

At this stage, everything is okay, both os.fwalk() os.walk() produce the same
results:

>>> import os
>>> for root, dirs, files in os.walk('.'):
... dirs.sort()
... print(root, dirs, files)
. ['1', '2', '3', '4'] []
./1 [] ['a']
./2 [] ['b']
./3 [] ['c']
./4 [] ['d']
>>> for root, dirs, files, fd in os.fwalk('.'):
... dirs.sort()
... print(root, dirs, files)
. ['1', '2', '3', '4'] []
./1 [] ['a']
./2 [] ['b']
./3 [] ['c']
./4 [] ['d']

To introduce an error, force a PermissionError on one of the directories:

$ sudo chown root:root 2
$ sudo chmod 700 2

Now, the os.fwalk() results are different (trust me, os.walk() is still okay):

>>> for root, dirs, files, fd in os.fwalk('.'):
... dirs.sort()
... print(root, dirs, files)
. ['1', '2', '3', '4'] []
./1 [] ['a']

So it seems that os.fwalk skips remaining directories after an error occurs.

The cause of the problem is in this part of os.py:

for name in dirs:
try:
orig_st = stat(name, dir_fd=topfd, follow_symlinks=follow_symlinks)
dirfd = open(name, O_RDONLY, dir_fd=topfd)
except OSError as err:
if onerror is not None:
onerror(err)
break

To fix it, simply replace break with continue. Patch attached.

Cheers

--
components: Library (Lib)
files: fwalk-silently-skips-dirs-when-error-occurs.patch
keywords: patch
messages: 256377
nosy: Samson Lee
priority: normal
severity: normal
status: open
title: os.fwalk() silently skips remaining directories when error occurs
type: behavior
versions: Python 3.4, Python 3.5
Added file: 
http://bugs.python.org/file41304/fwalk-silently-skips-dirs-when-error-occurs.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com