#36653: FORCE_SCRIPT_NAME is not respected for static URLs
-------------------------------------+-------------------------------------
Reporter: Brian Helba | Owner: (none)
Type: Bug | Status: new
Component: contrib.staticfiles | Version: 5.2
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Description changed by Brian Helba:
Old description:
> The documentation for the `STATIC_URL` setting states:
> > If `STATIC_URL` is a relative path, then it will be prefixed by the
> server-provided value of `SCRIPT_NAME` (or / if not set). This makes it
> easier to serve a Django application in a subpath without adding an extra
> configuration to the settings.
>
> However, when using `FORCE_SCRIPT_NAME`, the value of `STATIC_URL` when
> serving requests is never updated appropriately. This breaks URL
> construction via `django.templatetags.static.static` (used in templates
> as `{% static ... %}`).
>
> For example, this causes the Django Admin pages to break when using
> `FORCE_SCRIPT_NAME` to serve Django under a subpath. Generally, using
> `FORCE_SCRIPT_NAME` causes Django to behave incorrectly: view URLs are
> constructed to respect it, static URLs are constructed to not respect it.
>
> I believe that this bug likely also affects static URLs when using the
> `SCRIPT_NAME` WSGI environment variable too, but I haven't verified that
> yet.
>
> ----
> There is definitely some history here, but I believe previous bug reports
> have struggled to articulate this problem.
>
> Long ago, the following reports were made, which I believe are ''not''
> relevant to this problem (but I'm summarizing them because they've been
> claimed to be duplicates of this problem):
> * #7930 only discussed view URLs (and using `reverse()`); this now works
> as expected, but static URLs are still broken
> * #30634 is probably irrelevant; it claimed vague problems with
> `SCRIPT_NAME` and runserver; the problem here is general to all servers,
> including both runserver and WSGI
> * #31724 is probably a true duplicate of #7930
>
> More recently, we've seen:
> * #34892 is probably the same as this problem, but the reporter struggled
> to articulate the behavior and I believe it was mistakenly closed as a
> duplicate of the aforementioned old issues
> * #35985 claimed that this is problem is limited to using threading
> within management commands, then got derailed by the niche use case and
> suggestions around the low-level `set_script_prefix` API; it does contain
> the very useful suggestion to invoke `django.setup()` in each thread,
> which I don't believe was adequately explored
>
> ---
> I believe that I understand the exact cause of the bug. Note, my use of
> some lifecycle events and specific thread names may be limited to the
> runserver case, but the exact same behaviors manifest with WSGI (and I
> believe that equivalent things are occurring with a multiprocess
> lifecycle).
>
> 1. `django.setup()` is
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/management/__init__.py#L395
> called very early by runserver]
> 2. `django.setup()`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/__init__.py#L20-L23
> calls `set_script_prefix`]
> 3. `set_script_prefix`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/urls/base.py#L120C5-L126C20
> correctly sets] `django.urls.base._prefixes.value`, but only for the
> current thread (since `_prefixes` is a thread-local object)
> 4. a new thread, `django-main-thread`,
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/utils/autoreload.py#L666-L670
> is spawned]; again, I believe (and can locate if necessary) that an
> equivalent event also happens in a WSGI lifecycle
> 5. `check_url_settings`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/checks/urls.py#L108-L109
> runs and accesses `settings.STATIC_URL`]; importantly, this is the first
> time in the startup lifecycle that `settings.STATIC_URL` has ever been
> accessed
> 6. `django.conf.__getattr__`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/conf/__init__.py#L82-L83
> has special logic for `STATIC_URL`], so it calls the staticmethod
> `LazySettings._add_script_prefix`
> 7. `LazySettings._add_script_prefix`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/conf/__init__.py#L134
> calls `get_script_prefix`]
> 8. `get_script_prefix`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/urls/base.py#L129-L135
> looks at `django.urls.base._prefixes.value`], but it's running in a new
> thread (step 4), so it doesn't contain the `FORCE_SCRIPT_NAME` value
> (step 3) and returns `"/"`
> 9. `django.conf.__getattr__` (step 6) permanently caches the incorrect
> value of `settings.STATIC_URL` in `LazySettings.__dict__` (which is not
> thread-local); all future requests for `settings.STATIC_URL` will receive
> the incorrect value (`"/"`, instead of `settings.FORCE_SCRIPT_NAME`)
> 10. The first HTTP request comes in
> 11. `WSGIHandler.__call__`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/handlers/wsgi.py#L121
> calls] `get_script_name`
> 12. `get_script_name`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/handlers/wsgi.py#L162-L163
> correctly returns] `settings.FORCE_SCRIPT_NAME`
> 13. `WSGIHandler.__call__`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/handlers/wsgi.py#L121
> calls] `set_script_prefix` with the correct argument (the value
> `settings.FORCE_SCRIPT_NAME`)
> 14. `set_script_prefix` (just as in step 3)
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/urls/base.py#L120C5-L126C20
> finally sets] `django.urls.base._prefixes.value` (which, remember, is a
> thread-local variable, but will persist for at least the remainder of
> this HTTP request) with the correct value
> 15. While rendering the HTTP response, a template or some code calls
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/templatetags/static.py#L174-L179
> `django.templatetags.static`]; assume that the app
> `"django.contrib.staticfiles"` is installed and configured to use some
> subclass of `StaticFilesStorage` (which is Django's typical
> configuration)
> 16. `django.contrib.staticfiles.storage.staticfiles_storage.url()`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/templatetags/static.py#L129
> is called]
> 17. `staticfiles_storage` (an instance of `StaticFilesStorage`)
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/files/storage/handler.py#L46
> is lazily constructed]
> 18. `StaticFilesStorage.__init__` is called; assume it has no arguments
> from `settings.STORAGES["staticfiles"]["OPTIONS"]
> ([https://docs.djangoproject.com/en/5.2/ref/settings/#storages this is
> Django's default])
> 19. `StaticFilesStorage.__init__`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/contrib/staticfiles/storage.py#L27-L30
> defaults to set its `self._base_url`] to `settings.STATIC_URL`, but
> `settings.STATIC_URL` returns an incorrect value (step 9)
> 20. Continuing the call in step 16, `FilesystemStorage.url`
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/files/storage/filesystem.py#L212
> forms the actual URL] from `self.base_url`
> 21. `self.base_url`,
> [https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/files/storage/filesystem.py#L47-L51
> a cached property], relies on the incorrect `self._base_url`
> 22. A static file URL is returned with the incorrect base URL, not
> respecting `settings.FORCE_SCRIPT_NAME`
> 23. Subsequent HTTP requests skip steps 17-19, but otherwise reply the
> work from step 10 onwards (and also result in incorrect static URLs)
>
> In summary, the problem is that although there's code to attempt to set
> (via `set_script_prefix`) the correct script name both on Django's
> startup and again on every individual HTTP request, `settings.STATIC_URL`
> slips between a lifecycle "crack" and ends up with the wrong value (which
> doesn't incorporate the script name, ''contrary to its documentation''),
> which persists over the entire request-response lifecycle.
New description:
The documentation for the `STATIC_URL` setting states:
> If `STATIC_URL` is a relative path, then it will be prefixed by the
server-provided value of `SCRIPT_NAME` (or / if not set). This makes it
easier to serve a Django application in a subpath without adding an extra
configuration to the settings.
However, when using `FORCE_SCRIPT_NAME`, the value of `STATIC_URL` when
serving requests is never updated appropriately. This breaks URL
construction via `django.templatetags.static.static` (used in templates as
`{% static ... %}`).
For example, this causes the Django Admin pages to break when using
`FORCE_SCRIPT_NAME` to serve Django under a subpath. Generally, using
`FORCE_SCRIPT_NAME` causes Django to behave incorrectly: view URLs are
constructed to respect it, static URLs are constructed to not respect it.
I believe that this bug likely also affects static URLs when using the
`SCRIPT_NAME` WSGI environment variable too, but I haven't verified that
yet.
----
There is definitely some history here, but I believe previous bug reports
have struggled to articulate this problem.
Long ago, the following reports were made, which I believe are ''not''
relevant to this problem (but I'm summarizing them because they've been
claimed to be duplicates of this problem):
* #7930 only discussed view URLs (and using `reverse()`); this now works
as expected, but static URLs are still broken
* #30634 is probably irrelevant; it claimed vague problems with
`SCRIPT_NAME` and runserver; the problem here is general to all servers,
including both runserver and WSGI
* #31724 is probably a true duplicate of #7930
More recently, we've seen:
* #34892 is probably the same as this problem, but the reporter struggled
to articulate the behavior and I believe it was mistakenly closed as a
duplicate of the aforementioned old issues
* #35985 claimed that this is problem is limited to using threading within
management commands, then got derailed by the niche use case and
suggestions around the low-level `set_script_prefix` API; it does contain
the very useful suggestion to invoke `django.setup()` in each thread,
which I don't believe was adequately explored
----
I believe that I understand the exact cause of the bug. Note, my use of
some lifecycle events and specific thread names may be limited to the
runserver case, but the exact same behaviors manifest with WSGI (and I
believe that equivalent things are occurring with a multiprocess
lifecycle).
1. `django.setup()` is
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/management/__init__.py#L395
called very early by runserver]
2. `django.setup()`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/__init__.py#L20-L23
calls `set_script_prefix`]
3. `set_script_prefix`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/urls/base.py#L120C5-L126C20
correctly sets] `django.urls.base._prefixes.value`, but only for the
current thread (since `_prefixes` is a thread-local object)
4. a new thread, `django-main-thread`,
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/utils/autoreload.py#L666-L670
is spawned]; again, I believe (and can locate if necessary) that an
equivalent event also happens in a WSGI lifecycle
5. `check_url_settings`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/checks/urls.py#L108-L109
runs and accesses `settings.STATIC_URL`]; importantly, this is the first
time in the startup lifecycle that `settings.STATIC_URL` has ever been
accessed
6. `django.conf.__getattr__`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/conf/__init__.py#L82-L83
has special logic for `STATIC_URL`], so it calls the staticmethod
`LazySettings._add_script_prefix`
7. `LazySettings._add_script_prefix`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/conf/__init__.py#L134
calls `get_script_prefix`]
8. `get_script_prefix`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/urls/base.py#L129-L135
looks at `django.urls.base._prefixes.value`], but it's running in a new
thread (step 4), so it doesn't contain the `FORCE_SCRIPT_NAME` value (step
3) and returns `"/"`
9. `django.conf.__getattr__` (step 6) permanently caches the incorrect
value of `settings.STATIC_URL` in `LazySettings.__dict__` (which is not
thread-local); all future requests for `settings.STATIC_URL` will receive
the incorrect value (`"/"`, instead of `settings.FORCE_SCRIPT_NAME`)
10. The first HTTP request comes in
11. `WSGIHandler.__call__`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/handlers/wsgi.py#L121
calls] `get_script_name`
12. `get_script_name`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/handlers/wsgi.py#L162-L163
correctly returns] `settings.FORCE_SCRIPT_NAME`
13. `WSGIHandler.__call__`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/handlers/wsgi.py#L121
calls] `set_script_prefix` with the correct argument (the value
`settings.FORCE_SCRIPT_NAME`)
14. `set_script_prefix` (just as in step 3)
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/urls/base.py#L120C5-L126C20
finally sets] `django.urls.base._prefixes.value` (which, remember, is a
thread-local variable, but will persist for at least the remainder of this
HTTP request) with the correct value
15. While rendering the HTTP response, a template or some code calls
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/templatetags/static.py#L174-L179
`django.templatetags.static`]; assume that the app
`"django.contrib.staticfiles"` is installed and configured to use some
subclass of `StaticFilesStorage` (which is Django's typical configuration)
16. `django.contrib.staticfiles.storage.staticfiles_storage.url()`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/templatetags/static.py#L129
is called]
17. `staticfiles_storage` (an instance of `StaticFilesStorage`)
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/files/storage/handler.py#L46
is lazily constructed]
18. `StaticFilesStorage.__init__` is called; assume it has no arguments
from `settings.STORAGES["staticfiles"]["OPTIONS"]`
([https://docs.djangoproject.com/en/5.2/ref/settings/#storages this is
Django's default])
19. `StaticFilesStorage.__init__`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/contrib/staticfiles/storage.py#L27-L30
defaults to set its `self._base_url`] to `settings.STATIC_URL`, but
`settings.STATIC_URL` returns an incorrect value (step 9)
20. Continuing the call in step 16, `FilesystemStorage.url`
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/files/storage/filesystem.py#L212
forms the actual URL] from `self.base_url`
21. `self.base_url`,
[https://github.com/django/django/blob/1167cd1d639c3fee69dbdef351d31e8a17d1fedf/django/core/files/storage/filesystem.py#L47-L51
a cached property], relies on the incorrect `self._base_url`
22. A static file URL is returned with the incorrect base URL, not
respecting `settings.FORCE_SCRIPT_NAME`
23. Subsequent HTTP requests skip steps 17-19, but otherwise reply the
work from step 10 onwards (and also result in incorrect static URLs)
In summary, the problem is that although there's code to attempt to set
(via `set_script_prefix`) the correct script name both on Django's startup
and again on every individual HTTP request, `settings.STATIC_URL` slips
between a lifecycle "crack" and ends up with the wrong value (which
doesn't incorporate the script name, ''contrary to its documentation''),
which persists over the entire request-response lifecycle.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/36653#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070199cacdf883-20f97d23-de2f-4658-8966-e283db5c22d7-000000%40eu-central-1.amazonses.com.
