[issue22834] Unexpected FileNotFoundError when current directory is removed

[Christopher Meng]

Changes by Christopher Meng :


--
nosy: +Christopher Meng

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Brett Cannon added the comment:

Thanks for catches the mistakes, guys!

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Roundup Robot]

Roundup Robot added the comment:

New changeset 38c503c2c066 by Brett Cannon in branch 'default':
Issue #22834: Drop a redundant comment and use errno instead of an
https://hg.python.org/cpython/rev/38c503c2c066

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Changes by Brett Cannon :


--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Martin Panter]

Martin Panter added the comment:

+1 to EINVAL, also the bug reference comment is redundant with the one at the 
top of the test case :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

May be better use errno.EINVAL?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Brett Cannon added the comment:

Thanks for the suggestion, Martin. Went with a variant of what you proposed.

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Roundup Robot]

Roundup Robot added the comment:

New changeset f4f2096ab6f8 by Brett Cannon in branch 'default':
Issue #22834: Fix a failing test under Solaris due to the platform not
https://hg.python.org/cpython/rev/f4f2096ab6f8

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Martin Panter]

Martin Panter added the comment:

I don’t have a Solaris to test this, but maybe changing the first half of the 
test to the following would work:

dir = tempfile.TemporaryDirectory()
self.addCleanup(dir.cleanup)  # In case removal after chdir() fails
self.addCleanup(os.chdir, os.getcwd())
os.chdir(dir.name)
try:
dir.cleanup()
except OSError as err:  # Invalid argument on Solaris
self.skipTest("Couldn't remove current directory: {}".format(err))

with util.import_state(...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Tests failed on Solaris:
http://buildbot.python.org/all/builders/AMD64%20Solaris%2011%20%5BSB%5D%203.x/builds/3895/steps/test/logs/stdio

==
ERROR: test_deleted_cwd 
(test.test_importlib.import_.test_path.Source_FinderTests)
--
Traceback (most recent call last):
  File 
"/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/test/test_importlib/import_/test_path.py",
 line 167, in test_deleted_cwd
os.chdir(path)
  File 
"/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/tempfile.py",
 line 711, in __exit__
self.cleanup()
  File 
"/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/tempfile.py",
 line 715, in cleanup
_shutil.rmtree(self.name)
  File 
"/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/shutil.py",
 line 474, in rmtree
onerror(os.rmdir, path, sys.exc_info())
  File 
"/home/cpython/buildslave/cc-32/3.x.snakebite-solaris11-amd64/build/Lib/shutil.py",
 line 472, in rmtree
os.rmdir(path)
OSError: [Errno 22] Invalid argument: '/tmp/tmpsx3fm0t4'

--

--
nosy: +serhiy.storchaka
resolution: fixed -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Roundup Robot]

Roundup Robot added the comment:

New changeset d065e6474b67 by Zachary Ware in branch 'default':
Issue #22834: cwd can't not exist on Windows, skip the test
https://hg.python.org/cpython/rev/d065e6474b67

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Brett Cannon added the comment:

Along with fixing this I also updated the import reference to mention how the 
current working directory is handled.

Thanks to Martin for the report and everyone for their input!

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed
versions:  -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Roundup Robot]

Roundup Robot added the comment:

New changeset 8558fff73032 by Brett Cannon in branch 'default':
Issue #22834: Have import suppress FileNotFoundError when the current
https://hg.python.org/cpython/rev/8558fff73032

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Brett Cannon added the comment:

I have a patch to silence the exception and I'm running the test suite now.

I'm planning to keep this to a 3.5 fix and not changing the semantics in Python 
3.4 as the fix is a little different from the standard "directory in sys.path 
is invalid" since '' is dynamic and I can see someone relying on the current 
exception bubbling out somehow and not caching the dead directory in 
sys.path_importer_cache.

--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Martin Panter]

Martin Panter added the comment:

The only time I see a warning would be useful is if you intended to override a 
standard module with a module of the same name in the current directory. In all 
other cases I think it would be better to either generate an ImportError if the 
module is not found, or import it from wherever it is found. So I think a 
warning would not be useful in most cases.

Having any other non-existant directory in the search path is not an error and 
there is no warning either:
$ python3 -btWall
Python 3.4.2 (default, Oct  8 2014, 14:33:30) 
[GCC 4.9.1 20140903 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, "/blaua")
>>> import sadface
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named 'sadface'
>>> import urllib
>>> # Interpreter = happy
...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Terry J. Reedy]

Terry J. Reedy added the comment:

How about issue a verbose warning for possible debugging and continue for 
robustness?

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[R. David Murray]

R. David Murray added the comment:

Well, once I've launched a program, regardless of whether or not I expected 
some stuff to come from the CWD, I generally don't think about whether or not 
the CWD might go away, and in a complex setup it would most likely be getting 
deleted by some other process.  So I think I would prefer an import that is of 
something that wasn't ever in the CWD to succeed if the CWD goes away.  I'm not 
even sure I'd want a warning.

What happens if a directory on the path disappears?  If that is treated as a 
non-error, then I think a missing CWD on the path should also be treated as a 
non-error.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Brett Cannon added the comment:

So it isn't about importlib not handling missing directories on sys.path 
directly, it has to do with the fact that os.getcwd() raises FileNotFoundError 
when CWD is no longer valid and that is in a fundamental part of importlib that 
isn't worrying about non-existent directories.

We could be robust and simply have instances of os.getcwd() failing just move 
on, e.g. when '' is hit in sys.path and os.getcwd() raises an exception just 
give up. The other option is leaving this to raise FileNotFoundError but 
throwing a new copy with a better error message mentioning this is because the 
current working directory no longer exists. That would lead to easier debugging 
since paths on sys.path are typically obvious -- since site.py makes them 
absolute -- but I'm willing to bet people don't bother checking CWD is still 
good. So it's robustness vs. debugging when you make this mistake.

Any opinions on the matter?

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Brett Cannon]

Changes by Brett Cannon :


--
assignee:  -> brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[R. David Murray]

R. David Murray added the comment:

Looks like importlib doesn't handle the case of a directory on the path being 
deleted?  If so, I'm surprised this hasn't been reported before.

--
nosy: +brett.cannon, eric.snow, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22834] Unexpected FileNotFoundError when current directory is removed

[Martin Panter]

New submission from Martin Panter:

I encountered this when I added a unit test case that invoked os.chdir() with a 
temporary directory on Linux. After the directory was removed, some of the 
subsequent test cases failed, although I don’t think they should depend on a 
particular CWD.

I suspect the main problem might be code that is trying to dynamically import 
modules, and the interpreter is trying to search for modules in the current 
directory. I would expect it to happily go on to the other standard module 
directories or raise ImportError, just like if the current directory is valid 
but empty, or an nonexistent directory is in the module search path list.

Code to set up missing CWD:

import os
from tempfile import TemporaryDirectory
with TemporaryDirectory() as dir:
os.chdir(dir)

Quick recovery:

os.chdir("/")

Examples of failures:

>>> "\N{COPYRIGHT SIGN}"
  File "", line 1
SyntaxError: (unicode error) \N escapes not supported (can't load unicodedata 
module)
>>> datetime.strptime("", "")
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2237, in _find_and_load
  File "", line , in _find_and_load_unlocked
  File "", line 2164, in _find_spec
  File "", line 1940, in find_spec
  File "", line 1911, in _get_spec
  File "", line 1879, in _path_importer_cache
FileNotFoundError: [Errno 2] No such file or directory
>>> HTTPConnection("localhost").request("GET", "/")
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.4/http/client.py", line 1090, in request
self._send_request(method, url, body, headers)
  File "/usr/lib/python3.4/http/client.py", line 1128, in _send_request
self.endheaders(body)
  File "/usr/lib/python3.4/http/client.py", line 1086, in endheaders
self._send_output(message_body)
  File "/usr/lib/python3.4/http/client.py", line 924, in _send_output
self.send(msg)
  File "/usr/lib/python3.4/http/client.py", line 859, in send
self.connect()
  File "/usr/lib/python3.4/http/client.py", line 836, in connect
self.timeout, self.source_address)
  File "/usr/lib/python3.4/socket.py", line 491, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/lib/python3.4/socket.py", line 530, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
  File "/usr/lib/python3.4/encodings/__init__.py", line 98, in search_function
level=0)
  File "/usr/lib/python3.4/encodings/idna.py", line 3, in 
import stringprep, re, codecs
  File "", line 2237, in _find_and_load
  File "", line , in _find_and_load_unlocked
  File "", line 2164, in _find_spec
  File "", line 1940, in find_spec
  File "", line 1911, in _get_spec
  File "", line 1879, in _path_importer_cache
FileNotFoundError: [Errno 2] No such file or directory

>>> from datetime import datetime
>>> from http.client import HTTPConnection
These two also generate the FileNotFoundError

My workaround is to add this to my test case:
self.addCleanup(os.chdir, os.getcwd())

--
components: Interpreter Core
messages: 230933
nosy: vadmium
priority: normal
severity: normal
status: open
title: Unexpected FileNotFoundError when current directory is removed
type: behavior
versions: Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com