#35187: Django 5.0+ doesn't work with pyc files only
---------------------------------+--------------------------------------
Reporter: Marcus Hoffmann | Owner: nobody
Type: Bug | Status: new
Component: Core (Other) | Version: 5.0
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
---------------------------------+--------------------------------------
Comment (by William Schwartz):
I agree that what's happening here is the use (where the traceback shows)
of [https://docs.python.org/3/library/inspect.html#inspect.getsourcelines
inspect.getsourcelines], the documentation of which says, "An `OSError` is
raised if the source code cannot be retrieved." I also agree that the
problem started with https://github.com/django/django/pull/16831, which
says,
To persist the sensitive variables list for async functions, we can
use a global dict with the hash of the filename and line number of the
function as the key (more formally: `hash(f"{file_path}:{line_number}")`
and the value is the tuple or sensitive variables (or `__ALL__`). Then,
during traceback cleanup we can read this dict by reconstructing the same
key with `f_code.co_filename` and `f_code.co_firstlineno` to lookup the
relevant value in the dict.
I don't have the bandwidth these days to offer an alternative solution.
All I can say is that the implementation assumes the existence of a file
path that Python does not guarantee exists. To be clear on this: **Python
does not guarantee that there is a source file or cache file associated
with any object.** Modules without source files exist in *all* CPython
instances because of the `sys` module, and in most CPython instances
because of the frozen `importlib` submodules. They occur frequently in
embedded instance of Python, e.g., when running code directly from pyc
cache files (which I have always [back to 2010] understood was a fully
supported solution) or using a compiler such as PyOxidizer, which I use.
So a correct solution to whatever
https://github.com/django/django/pull/16831 was trying to fix for async
functions cannot rely on source files. I think what went wrong was
essentially what comment:10:ticket:30950 warned of:
I could be wrong but I believe that whatever we do to tackle this
ticket, it's important ensure that __file__ isn't added back by new
contributions in the future. Ideally this shouldn't be a burden on
reviewers, there should be a linting rule, or a test or something that
ensures that any contributions following the resolution of this ticket
isn't adding this back.
Ideally there'd be a test run of Django in CI that runs in embedded form
on an embedded project/app (whether cache-only, frozen, or PyOxidered or
whatever). That would have caught this issue since `__file__` isn't being
''directly'' used, just indirectly via `inspect.getsourcelines`.
Unfortunately I don't have the time to do this myself.
PS, I'm excited that someone else is using Django embedded! FWIW I've been
successfully using Django 3.2.13 with PyOxidizer Python applications for
awhile. However, I haven't embedded Django itself, which continues to
exist in source form next to my applications, i.e., only my own
application code is embedded in the PyOxidizer-generated binary, not
Django itself. Moreover, we do not use the admin or auth contrib apps.
That said, I would certainly prefer to embed Django one day. If I recall
correctly, at the time I ran out of budget to work on this, the barrier to
embedding Django for us was the `loaddata` command.
--
Ticket URL: <https://code.djangoproject.com/ticket/35187#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/0107018da412984e-ae561941-b328-4142-b216-4456904bda0f-000000%40eu-central-1.amazonses.com.
