New submission from Samson Lee: I noticed os.fwalk() produced different results as os.walk(). It turned out that os.fwalk() skips all remaining directories when OSError occurs (e.g. due to PermissionError).
To reproduce the bug, first create a test directory structure: $ mkdir 1; touch 1/a $ mkdir 2; touch 2/b $ mkdir 3; touch 3/c $ mkdir 4; touch 4/c At this stage, everything is okay, both os.fwalk() os.walk() produce the same results: >>> import os >>> for root, dirs, files in os.walk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] ./2 [] ['b'] ./3 [] ['c'] ./4 [] ['d'] >>> for root, dirs, files, fd in os.fwalk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] ./2 [] ['b'] ./3 [] ['c'] ./4 [] ['d'] To introduce an error, force a PermissionError on one of the directories: $ sudo chown root:root 2 $ sudo chmod 700 2 Now, the os.fwalk() results are different (trust me, os.walk() is still okay): >>> for root, dirs, files, fd in os.fwalk('.'): ... dirs.sort() ... print(root, dirs, files) . ['1', '2', '3', '4'] [] ./1 [] ['a'] So it seems that os.fwalk skips remaining directories after an error occurs. The cause of the problem is in this part of os.py: for name in dirs: try: orig_st = stat(name, dir_fd=topfd, follow_symlinks=follow_symlinks) dirfd = open(name, O_RDONLY, dir_fd=topfd) except OSError as err: if onerror is not None: onerror(err) break To fix it, simply replace break with continue. Patch attached. Cheers ---------- components: Library (Lib) files: fwalk-silently-skips-dirs-when-error-occurs.patch keywords: patch messages: 256377 nosy: Samson Lee priority: normal severity: normal status: open title: os.fwalk() silently skips remaining directories when error occurs type: behavior versions: Python 3.4, Python 3.5 Added file: http://bugs.python.org/file41304/fwalk-silently-skips-dirs-when-error-occurs.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25860> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com