[issue36243] Python os.listdir fails with FileNotFoundError when directory exists

[Irit Katriel]

Irit Katriel  added the comment:

Geoff, it's been over two years. Do you think there is a chance you will want 
to follow up with a reproducer, or shall we close this issue?

--
nosy: +iritkatriel
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36243] Python os.listdir fails with FileNotFoundError when directory exists

[Geoff Alexander]

Change by Geoff Alexander :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36243] Python os.listdir fails with FileNotFoundError when directory exists

[Geoff Alexander]

Geoff Alexander  added the comment:

This problem does not appear to be a race condition ("Time Of Check To Time Of 
Use" bug) in my case as the directory in question exists both before and after 
the os.listdir call.

I got a workaround saying to wrap the os.listdir call in try/except in my Stack 
Overflow question at 
https://stackoverflow.com/questions/55067904/python-os-listdir-fails-with-filenotfounderror-when-directory-exists.
  This worked as no longer got any spurious FileNotFound exceptions.

I realize that the code snippet I gave is not runnable.  I'll look to put 
together a small example illustrating the problem and add to the issue.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36243] Python os.listdir fails with FileNotFoundError when directory exists

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> How can this happen?

Easily -- this looks like a "Time Of Check To Time Of Use" bug. You check for 
the existence of a directory, and then a fraction of a second later you attempt 
to use that directory. But on a multi-processing operating system like Windows, 
Mac or Linux, a lot can happen in that fraction of a second, including the 
directory being deleted by another process.

https://en.wikipedia.org/wiki/Time_of_check_to_time_of_use

I'm not saying that this absolutely cannot be a bug in Python, but the initial 
indication is that this is simply a TOCTTOU bug in your code.

Unfortunately, your code is unrunnable by us, as it if full of mysterious 
objects like "shouter.shout" which we have no access too.

In addition, there's a problem that "handle_empty_directories" calls itself 
recursively as if it were a method of some "Commiter" object:

Commiter.handle_empty_directories

but it has no "self" parameter, so I don't know what is going on there.

For us to investigate further, you need to replicate the error using much 
simple code that we can run ourselves.

Please read http://www.sscce.org/ for some hints in producing short, 
self-contained, runnable code so that we can replicate this error.

In the meantime, I'm going to change this to Pending. When you have replicated 
this with simpler code, please feel free to change the status back to Open.

--
nosy: +steven.daprano
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36243] Python os.listdir fails with FileNotFoundError when directory exists

[Geoff Alexander]

Change by Geoff Alexander :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36243] Python os.listdir fails with FileNotFoundError when directory exists

[Geoff Alexander]

New submission from Geoff Alexander :

I have the following code:

```
def handle_empty_directories(dir):
if os.path.exists(dir):
shouter.shout("%s exists" % dir)
else:
   shouter.shout("%s doesn't exists" % dir)
entries = os.listdir(dir)
if entries == []:
open(os.path.join(dir, Commiter.hed_file), "w").close()
else:
if (len(entries) > 1) and (Commiter.hed_file in entries):
os.remove(os.path.join(dir, Commiter.hed_file))
for entry in entries:
if entry not in Commiter.hed_ignore:
full_entry = os.path.join(dir, entry)
if (os.path.isdir(full_entry)):
Commiter.handle_empty_directories(full_entry)
```

Occasionally, the call to os.listdir(dir) fails with FileNotFoundError even 
though the os.path.exists(dir) call says that exists:

```
08:57:56 - 
C:\r2g-wd\sport-6.0.5\SBS\SBS\Light\bin\com\ibm\ArtifactTechnology\ABS\ArtifactBroker
 exists
Traceback (most recent call last):
  File "migration.py", line 169, in 
migrate()
  File "migration.py", line 80, in migrate

rtc.acceptchangesintoworkspace(rtc.getchangeentriestoaccept(changeentries, 
history))
  File 
"c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\rtcFunctions.py",
 line 304, in acceptchangesintoworkspace
Commiter.handle_empty_directories(os.getcwd())
  File 
"c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py",
 line 126, in handle_empty_directories
Commiter.handle_empty_directories(full_entry)
  File 
"c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py",
 line 126, in handle_empty_directories
Commiter.handle_empty_directories(full_entry)
  File 
"c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py",
 line 126, in handle_empty_directories
Commiter.handle_empty_directories(full_entry)
  [Previous line repeated 6 more times]
  File 
"c:\Users\GeoffAlexander\Documents\Nirvana\RTC2Git\git-repositories\rtc2git-migration-tool\gitFunctions.py",
 line 116, in handle_empty_directories
entries = os.listdir(dir)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 
'C:\\r2g-wd\\sport-6.0.5\\SBS\\SBS\\Light\\bin\\com\\ibm\\ArtifactTechnology\\ABS\\ArtifactBroker'
```

I've also seen a similar FileNoteFound on the 

```
open(os.path.join(dir, Commiter.hed_file), "w").close()
```

line when os.path.exists(dir) says that the directory exists.

How can this happen? I'm running Python 3.7.2 64-bit on Windows 10.

--
components: Windows
messages: 337516
nosy: Geoff.Alexander, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: Python os.listdir fails with FileNotFoundError when directory exists
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com