#28241: module_has_submodule raises exceptions in Python 3
-------------------------------------+-------------------------------------
Reporter: Thomas Khyn | Owner: nobody
Type: Bug | Status: new
Component: Utilities | Version: 1.11
Severity: Normal | Resolution:
Keywords: module_loading | Triage Stage:
python3 | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Thomas Khyn:
Old description:
> Hello,
>
> Porting a django project from python 2.7 to 3.6, I noticed an issue with
> ``utils.module_loading.module_has_submodule``. At some stage my project
> makes use of ``autodiscover_modules('module.submodule')`` to try and
> discover modules that are nested in my app.
>
> I'm using django 1.11.1, but this bug probably also affects 1.8 as the
> code of ``module_has_submodule`` is the same.
>
> So here we go, with python 2.7 we have the expected behavior (taking
> contenttypes as an example app):
>
> {{{
> python2.7
> >>> from django.utils.module_loading import module_has_submodule
> >>> from django.contrib import contenttypes
> >>> module_has_submodule('invalid_module.submodule')
> False
> >>> module_has_submodule('checks')
> True
> >>> module_has_submodule('checks.submodule')
> False
> }}}
>
> But, with python 3.6
>
> {{{
> python3.6
> >>> from django.utils.module_loading import module_has_submodule
> >>> from django.contrib import contenttypes
> >>> module_has_submodule('invalid_module.submodule')
> File "<console>", line 1, in <module>
> File
> "d:\dev\.env\buildout\eggs\django-1.11.1-py3.6.egg\django\utils\module_loading.py",
> line 79, in module_has_submodule
> return importlib_find(full_module_name, package_path) is not None
> File "D:\dev\.env\venv\buildout\lib\importlib\util.py", line 88, in
> find_spec
> parent = __import__(parent_name, fromlist=['__path__'])
> ModuleNotFoundError: No module named
> 'django.contrib.contenttypes.invalid_module'
> >>> module_has_submodule('checks')
> True
> >>> module_has_submodule('checks.submodule')
> File "<console>", line 1, in <module>
> File
> "d:\dev\.env\buildout\eggs\django-1.11.1-py3.6.egg\django\utils\module_loading.py",
> line 79, in module_has_submodule
> return importlib_find(full_module_name, package_path) is not None
> File "D:\dev\.env\venv\buildout\lib\importlib\util.py", line 89, in
> find_spec
> return _find_spec(fullname, parent.__path__)
> AttributeError: module 'django.contrib.contenttypes.checks' has no
> attribute '__path__'
> }}}
>
> From the replies I got on the python bug tracker
> (http://bugs.python.org/issue30436) the `AttributeError` will be
> converted to `ModuleNotFoundError` only in Python 3.7, but that behavior
> is apparently not expected to be fixed in previous versions.
>
> We'll definitely need to catch `ModuleNotFoundError`, and to have a
> proper fix for python < 3.7 we'll need to:
> - either catch `AttributeError` as well. The problem I see is
> ``AttributeError`` is too broad and may result in 'false catches'
> - or convert a dotted path to [package, name] (basically do what
> ``find_spec`` actually does) but detect if the module has a `__path__`
> attribute before calling `find_spec` so that it can not raise exceptions
>
> What are your thoughts ?
New description:
Hello,
Porting a django project from python 2.7 to 3.6, I noticed an issue with
``utils.module_loading.module_has_submodule``. At some stage my project
makes use of ``autodiscover_modules('module.submodule')`` to try and
discover modules that are nested in my app.
I'm using django 1.11.1, but this bug probably also affects 1.8 as the
code of ``module_has_submodule`` is the same.
So here we go, with python 2.7 we have the expected behavior (taking
contenttypes as an example app):
{{{
python2.7
>>> from django.utils.module_loading import module_has_submodule
>>> from django.contrib import contenttypes
>>> module_has_submodule(contenttypes, 'invalid_module.submodule')
False
>>> module_has_submodule(contenttypes, 'checks')
True
>>> module_has_submodule(contenttypes, 'checks.submodule')
False
}}}
But, with python 3.6
{{{
python3.6
>>> from django.utils.module_loading import module_has_submodule
>>> from django.contrib import contenttypes
>>> module_has_submodule(contenttypes, 'invalid_module.submodule')
File "<console>", line 1, in <module>
File
"d:\dev\.env\buildout\eggs\django-1.11.1-py3.6.egg\django\utils\module_loading.py",
line 79, in module_has_submodule
return importlib_find(full_module_name, package_path) is not None
File "D:\dev\.env\venv\buildout\lib\importlib\util.py", line 88, in
find_spec
parent = __import__(parent_name, fromlist=['__path__'])
ModuleNotFoundError: No module named
'django.contrib.contenttypes.invalid_module'
>>> module_has_submodule(contenttypes, 'checks')
True
>>> module_has_submodule(contenttypes, 'checks.submodule')
File "<console>", line 1, in <module>
File
"d:\dev\.env\buildout\eggs\django-1.11.1-py3.6.egg\django\utils\module_loading.py",
line 79, in module_has_submodule
return importlib_find(full_module_name, package_path) is not None
File "D:\dev\.env\venv\buildout\lib\importlib\util.py", line 89, in
find_spec
return _find_spec(fullname, parent.__path__)
AttributeError: module 'django.contrib.contenttypes.checks' has no
attribute '__path__'
}}}
From the replies I got on the python bug tracker
(http://bugs.python.org/issue30436) the `AttributeError` will be converted
to `ModuleNotFoundError` only in Python 3.7, but that behavior is
apparently not expected to be fixed in previous versions.
We'll definitely need to catch `ModuleNotFoundError`, and to have a proper
fix for python < 3.7 we'll need to:
- either catch `AttributeError` as well. The problem I see is
``AttributeError`` is too broad and may result in 'false catches'
- or convert a dotted path to [package, name] (basically do what
``find_spec`` actually does) but detect if the module has a `__path__`
attribute before calling `find_spec` so that it can not raise exceptions
What are your thoughts ?
--
--
Ticket URL: <https://code.djangoproject.com/ticket/28241#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/063.82fac40e05375f6cde9f23975899cd93%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
