#30950: Replace __file__ with importlib.resources
---------------------------------+-----------------------------------------
Reporter: John Vandenberg | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Someday/Maybe
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+-----------------------------------------
Changes (by William Schwartz):
* cc: William Schwartz (added)
Comment:
Support for PyOxidizer is important to my use case.
In addition to the issue of package data, for which `importlib.resources`
is a good solution, there's also the matter of management commands. The
current algorithm uses the standard library's `pkgutil.iter_modules`,
which only works with file-system based code. A good alternative that
would work for non-file code would be to use the import mechanism as
follows. The following code would replace
[https://github.com/django/django/blob/0a306f7da668e53af2516bfad759b52d6c650b69/django/core/management/__init__.py#L41-L73
django.core.management.get_commands()] and
[https://github.com/django/django/blob/0a306f7da668e53af2516bfad759b52d6c650b69/django/core/management/__init__.py#L21-L28
django.core.management.find_commands()]. I made some effort to keep the
code similar to the existing code. I dressed up some of the docstring for
Sphinx in case a description of the algorithm would need to go into the
docs.
{{{
#!python
import functools
from importlib import import_module
from types import ModuleType
from typing import Dict, List
from django.apps import apps
from django.conf import settings
from django.core.management.base import BaseCommand
def find_commands(management_pkg_name: str) -> List[str]:
"""Given the name of a management package, return a list of all the
command
names that are available.
"""
commands_pkg_name = '.'.join([management_pkg_name, 'commands'])
try:
pkg = import_module(commands_pkg_name)
except ModuleNotFoundError:
continue
return [name for name, obj in pkg.__dict__.items()
if isinstance(obj, ModuleType)
# Django's rules ignore modules whose names start with _ or is a
package
# A module has a __path__ attribute if and only if it's a package
and not hasattr(obj, '__path__')
and not obj.__name__.startswith('_')
# Ignore incidental imports from other packages into __init__.py
and obj.__package__ == commands_pkg_name
# We only care if there's a BaseCommand subclass named Command
and hasattr(obj, 'Command')
and issubclass(obj.Command, BaseCommand)
]
@functools.lru_cache(maxsize=None)
def get_commands_by_import() -> Dict[str, str]:
r"""Find :doc:`django:ref/django-admin` commands via the import
system.
Look for a management.commands package in django.core, and in each
installed application — if a commands package exists, register all
commands in that package.
Core commands are always included. If a settings module has been
specified, also include user-defined commands.
The dictionary is in the format {command_name: app_name}. Key-value
pairs from this dictionary can then be used in calls to
load_command_class(app_name, command_name)
If a specific version of a command must be loaded (e.g., with the
startapp command), the instantiated module can be placed in the
dictionary in place of the application name.
The dictionary is cached on the first call and reused on subsequent
calls.
:returns:
A dictionary mapping :samp:`{command}` names to the :samp:`{app}`
they
come from. For each :samp:`{app}` in :setting:`INSTALLED_APPS`, we
search for an :samp:`{app}.management.commands.{command}` module
by
trying to import :samp:`{app}.management.commands`, iterating
through
the module's :attr:`~object.__dict__`, and testing its values. We
include any :class:`types.ModuleType` :samp:`{command}` if its
name does
not start with an underscore (``_``), it's not a
:term:`python:package`,
and it has a :class:`django.core.management.BaseCommand` subclass
called
``Command``.
"""
commands = {name: 'django.core' for name in
find_commands(__package__)}
if not settings.configured:
return commands
for app_config in reversed(list(apps.get_app_configs())):
pkg_name = '.'.join([app_config.name, 'management'])
commands.update({name: app_config.name for name in
find_commands(pkg_name)})
return commands
}}}
The main backward incompatibility is that this would require apps'
`management.commands` packages to import all command modules in
`__init__.py`. If this is unacceptable, `find_commands` could fall back to
the existing `pkgutil`-based search algorithm.
A minor backward incompatibility is that this search is more conservative
than the existing algorithm. In particular, if a module doesn't contain a
`BaseCommand` subclass called `Command`, no attempt is made to include it.
A minor performance consideration here is that we're importing more
modules' code at command-line start. Maybe some use of
[https://docs.python.org/3/library/importlib.html#importlib.util.LazyLoader
importlib.util.LazyLoader] and foregoing to the check for the presence of
the `Command` class could mitigate this performance penalty.
--
Ticket URL: <https://code.djangoproject.com/ticket/30950#comment:6>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.332b40d5496ceaca8ac561f5c9e497c7%40djangoproject.com.
