#34204: Django cannot load when Python is compiled with --without-doc-strings
enabled
-------------------------------+--------------------------------------
Reporter: Jon Janzen | Owner: nobody
Type: Uncategorized | Status: new
Component: Uncategorized | Version: 4.1
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------+--------------------------------------
Description changed by Jon Janzen:
Old description:
> I'm not sure that this is even a supported configuration for Django, but
> CPython's `./configure` script offers an option called
> [https://docs.python.org/3/using/configure.html#cmdoption-without-doc-
> strings --without-doc-strings] which improves memory utilization
> slightly.
>
> Quoting the relevant portion from CPython docs:
>
> > --without-doc-strings
> > Disable static documentation strings to reduce the memory footprint
> (enabled by default). Documentation strings defined in Python are not
> affected.
>
> I have an use-case where I need to deploy a Django service on a device
> with low available memory. I'd like to disable built-in doc strings as
> part of an overall strategy to reduce memory but compiling CPython with
> that option and running the django service crashes:
>
> {{{
> File ".../asgi.py", line 3, in <module>
> from django.core.asgi import get_asgi_application
> File ".../lib/python3.11/site-packages/django/core/asgi.py", line 2, in
> <module>
> from django.core.handlers.asgi import ASGIHandler
> File ".../lib/python3.11/site-packages/django/core/handlers/asgi.py",
> line 11, in <module>
> from django.core.handlers import base
> File ".../lib/python3.11/site-packages/django/core/handlers/base.py",
> line 11, in <module>
> from django.urls import get_resolver, set_urlconf
> File ".../lib/python3.11/site-packages/django/urls/__init__.py", line
> 1, in <module>
> from .base import (
> File ".../lib/python3.11/site-packages/django/urls/base.py", line 8, in
> <module>
> from .exceptions import NoReverseMatch, Resolver404
> File ".../lib/python3.11/site-packages/django/urls/exceptions.py", line
> 1, in <module>
> from django.http import Http404
> File ".../lib/python3.11/site-packages/django/http/__init__.py", line
> 2, in <module>
> from django.http.request import (
> File ".../lib/python3.11/site-packages/django/http/request.py", line 8,
> in <module>
> from django.core import signing
> File ".../lib/python3.11/site-packages/django/core/signing.py", line
> 43, in <module>
> from django.utils.crypto import constant_time_compare, salted_hmac
> File ".../lib/python3.11/site-packages/django/utils/crypto.py", line
> 85, in <module>
> if func_supports_parameter(hashlib.md5, 'usedforsecurity'):
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
> 74, in func_supports_parameter
> return any(param.name == name for param in
> _get_callable_parameters(func))
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
> 16, in _get_callable_parameters
> return _get_func_parameters(func, remove_first=is_method)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
> 7, in _get_func_parameters
> parameters = tuple(inspect.signature(func).parameters.values())
> ^^^^^^^^^^^^^^^^^^^^^^^
> File ".../cpython/Lib/inspect.py", line 3270, in signature
> return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File ".../cpython/Lib/inspect.py", line 3018, in from_callable
> return _signature_from_callable(obj, sigcls=cls,
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File ".../cpython/Lib/inspect.py", line 2510, in
> _signature_from_callable
> return _signature_from_builtin(sigcls, obj,
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> File ".../cpython/Lib/inspect.py", line 2317, in
> _signature_from_builtin
> raise ValueError("no signature found for builtin {!r}".format(func))
> ValueError: no signature found for builtin <built-in function
> openssl_md5>
> }}}
>
> (irrelevant information lightly censored)
>
> An easy way to repro is to follow this script:
> {{{
> git clone https://github.com/python/cpython.git
> cd cpython
> git checkout 3.11 # or whatever version you have that already has django
> installed
> ./configure --without-doc-strings
> make -j
> ./python
> >>> import django.utils.crypto
> *crash*
> }}}
>
> Looking through this stack trace the problem looks to be
> [https://github.com/django/django/blob/d10c7bfe56f025ccc690721c9f13e7029b777b9c/django/utils/crypto.py#L80-L92
> some code] that executes at module import to determine whether or not md5
> supports the `usedforsecurity` parameter.
>
> If this ticket is accepted I'm happy to put up a PR to fix this, I've
> included my proposed solution on this ticket to save a roundtrip in
> discussion:
>
> {{{
> diff --git a/django/utils/inspect.py b/django/utils/inspect.py
> index 28418f7312..d8622a22df 100644
> --- a/django/utils/inspect.py
> +++ b/django/utils/inspect.py
> @@ -70,4 +70,12 @@ def method_has_no_args(meth):
>
> def func_supports_parameter(func, name):
> - return any(param.name == name for param in
> _get_callable_parameters(func))
> + try:
> + callable_parameters = _get_callable_parameters(func)
> + except ValueError:
> + # When Python is compiled with the --without-doc-strings
> + # argument to ./configure the signatures for built-in
> + # functions are not available. In such a case, be
> + # conservative and assume no:
> + return False
> + return any(param.name == name for param in callable_parameters)
> }}}
New description:
I'm not sure that this is even a supported configuration for Django, but
CPython's `./configure` script offers an option called
[https://docs.python.org/3/using/configure.html#cmdoption-without-doc-
strings --without-doc-strings] which improves memory utilization slightly.
Quoting the relevant portion from CPython docs:
> --without-doc-strings
> Disable static documentation strings to reduce the memory footprint
(enabled by default). Documentation strings defined in Python are not
affected.
I have an use-case where I need to deploy a Django service on a device
with low available memory. I'd like to disable built-in doc strings as
part of an overall strategy to reduce memory but compiling CPython with
that option and running the django service crashes:
{{{
File ".../asgi.py", line 3, in <module>
from django.core.asgi import get_asgi_application
File ".../lib/python3.11/site-packages/django/core/asgi.py", line 2, in
<module>
from django.core.handlers.asgi import ASGIHandler
File ".../lib/python3.11/site-packages/django/core/handlers/asgi.py",
line 11, in <module>
from django.core.handlers import base
File ".../lib/python3.11/site-packages/django/core/handlers/base.py",
line 11, in <module>
from django.urls import get_resolver, set_urlconf
File ".../lib/python3.11/site-packages/django/urls/__init__.py", line 1,
in <module>
from .base import (
File ".../lib/python3.11/site-packages/django/urls/base.py", line 8, in
<module>
from .exceptions import NoReverseMatch, Resolver404
File ".../lib/python3.11/site-packages/django/urls/exceptions.py", line
1, in <module>
from django.http import Http404
File ".../lib/python3.11/site-packages/django/http/__init__.py", line 2,
in <module>
from django.http.request import (
File ".../lib/python3.11/site-packages/django/http/request.py", line 8,
in <module>
from django.core import signing
File ".../lib/python3.11/site-packages/django/core/signing.py", line 43,
in <module>
from django.utils.crypto import constant_time_compare, salted_hmac
File ".../lib/python3.11/site-packages/django/utils/crypto.py", line 85,
in <module>
if func_supports_parameter(hashlib.md5, 'usedforsecurity'):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
74, in func_supports_parameter
return any(param.name == name for param in
_get_callable_parameters(func))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/django/utils/inspect.py", line
16, in _get_callable_parameters
return _get_func_parameters(func, remove_first=is_method)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../lib/python3.11/site-packages/django/utils/inspect.py", line 7,
in _get_func_parameters
parameters = tuple(inspect.signature(func).parameters.values())
^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 3270, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 3018, in from_callable
return _signature_from_callable(obj, sigcls=cls,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 2510, in
_signature_from_callable
return _signature_from_builtin(sigcls, obj,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".../cpython/Lib/inspect.py", line 2317, in _signature_from_builtin
raise ValueError("no signature found for builtin {!r}".format(func))
ValueError: no signature found for builtin <built-in function openssl_md5>
}}}
(irrelevant information lightly censored)
An easy way to repro is to follow this script:
{{{
git clone https://github.com/python/cpython.git
cd cpython
git checkout 3.11 # or whatever version you have that already has django
installed
./configure --without-doc-strings
make -j
./python -c "import django.utils.crypto"
}}}
Looking through this stack trace the problem looks to be
[https://github.com/django/django/blob/d10c7bfe56f025ccc690721c9f13e7029b777b9c/django/utils/crypto.py#L80-L92
some code] that executes at module import to determine whether or not md5
supports the `usedforsecurity` parameter.
If this ticket is accepted I'm happy to put up a PR to fix this, I've
included my proposed solution on this ticket to save a roundtrip in
discussion:
{{{
diff --git a/django/utils/inspect.py b/django/utils/inspect.py
index 28418f7312..d8622a22df 100644
--- a/django/utils/inspect.py
+++ b/django/utils/inspect.py
@@ -70,4 +70,12 @@ def method_has_no_args(meth):
def func_supports_parameter(func, name):
- return any(param.name == name for param in
_get_callable_parameters(func))
+ try:
+ callable_parameters = _get_callable_parameters(func)
+ except ValueError:
+ # When Python is compiled with the --without-doc-strings
+ # argument to ./configure the signatures for built-in
+ # functions are not available. In such a case, be
+ # conservative and assume no:
+ return False
+ return any(param.name == name for param in callable_parameters)
}}}
--
--
Ticket URL: <https://code.djangoproject.com/ticket/34204#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070184f57c79d3-a8d1c280-569f-4eb7-84be-54c04bb4cfde-000000%40eu-central-1.amazonses.com.
