#27059: autoreload loose the sys.path in some cases (buildout install of python
for
instance, maybe others)
-------------------------------------+-------------------------------------
Reporter: Annakan | Owner: nobody
Type: Bug | Status: new
Component: Core (Management | Version: 1.10
commands) |
Severity: Normal | Resolution:
Keywords: reloader runserver | Triage Stage:
buildout isolation pythonpath | Unreviewed
sys.path |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Annakan):
* needs_better_patch: => 0
* needs_tests: => 0
* needs_docs: => 0
Old description:
> In my configuration, aiming for isolation, I installed django with pyenv
> (virtualenv) and buildout.
> In that case when using {{{manage.py runserver}}} the reloading loose the
> sys.path (and thus the eggs installed in the buildout)
> In the case of a buildout sys.executable give the path of the python
> executable the buildout was build with and not the python script in
> <buildout-root>/bin/
>
> I am not sure why nobody had the problem before but I devised a hack
> since restart_with_reloader properly copy the environment I patched the
> code to copy the sys.path into the pythonpath envar of the process.
>
> The modified code is below :
>
> {{{
> def restart_with_reloader():
> while True:
> args = [sys.executable] + ['-W%s' % o for o in sys.warnoptions] +
> sys.argv
> if sys.platform == "win32":
> args = ['"%s"' % arg for arg in args]
> # Simply restarting the executable won't preserve
> virtualenv/pyenv/buildout path properly in case of a "buildout"
> # sys.executable point at the the python binary used to create
> the buildout
> # and not the bin/python script used to launch the manage.py
> script
> cur_python_path = ':'.join(sys.path)
> if 'PYTHONPATH' in os.environ:
> if cur_python_path not in os.environ['PYTHONPATH']:
> os.environ['PYTHONPATH'] += cur_python_path
> else:
> os.environ['PYTHONPATH'] = cur_python_path
> new_environ = os.environ.copy()
> new_environ["RUN_MAIN"] = 'true'
> exit_code = os.spawnve(os.P_WAIT, sys.executable, args,
> new_environ)
> if exit_code != 3:
> return exit_code
>
> }}}
>
> The 6 lines added are
>
> {{{
> cur_python_path = ':'.join(sys.path)
> if 'PYTHONPATH' in os.environ:
> if cur_python_path not in os.environ['PYTHONPATH']:
> os.environ['PYTHONPATH'] += cur_python_path
> else:
> os.environ['PYTHONPATH'] = cur_python_path
> }}}
>
> I am not sure there are side effects for project without a buildout
> though but is seems to me that if site.py is not too messed up with (we
> are looking at you ubuntu and thanks pyenv ;) ) pythonpath is second in
> peeking order and we should aim to preserve the original sys.path when
> reloading.
> The worse that could happen is that the sys.path contains the same path
> twice, but I checked and Python seems to properly clean up the sys.path
> to keep module paths de-duplicated.
New description:
In my configuration, aiming for isolation, I installed django with pyenv
(virtualenv) and buildout.
In that case when using {{{manage.py runserver}}} the reloading loose the
sys.path (and thus the eggs installed in the buildout)
In the case of a buildout sys.executable returns the path of the python
executable the buildout was build with, and not the python script in
<buildout-root>/bin/
I am not sure why nobody had the problem before but I devised a hack since
restart_with_reloader properly copy the environment I patched the code to
copy the sys.path into the pythonpath envar of the process.
The modified code is below :
{{{
def restart_with_reloader():
while True:
args = [sys.executable] + ['-W%s' % o for o in sys.warnoptions] +
sys.argv
if sys.platform == "win32":
args = ['"%s"' % arg for arg in args]
# Simply restarting the executable won't preserve
virtualenv/pyenv/buildout path properly in case of a "buildout"
# sys.executable point at the the python binary used to create
the buildout
# and not the bin/python script used to launch the manage.py
script
cur_python_path = ':'.join(sys.path)
if 'PYTHONPATH' in os.environ:
if cur_python_path not in os.environ['PYTHONPATH']:
os.environ['PYTHONPATH'] += cur_python_path
else:
os.environ['PYTHONPATH'] = cur_python_path
new_environ = os.environ.copy()
new_environ["RUN_MAIN"] = 'true'
exit_code = os.spawnve(os.P_WAIT, sys.executable, args,
new_environ)
if exit_code != 3:
return exit_code
}}}
The 6 lines added are
{{{
cur_python_path = ':'.join(sys.path)
if 'PYTHONPATH' in os.environ:
if cur_python_path not in os.environ['PYTHONPATH']:
os.environ['PYTHONPATH'] += cur_python_path
else:
os.environ['PYTHONPATH'] = cur_python_path
}}}
I am not sure there are side effects for project without a buildout though
but is seems to me that if site.py is not too messed up with (we are
looking at you ubuntu and thanks pyenv ;) ) pythonpath is second in
peeking order and we should aim to preserve the original sys.path when
reloading.
The worse that could happen is that the sys.path contains the same path
twice, but I checked and Python seems to properly clean up the sys.path to
keep module paths de-duplicated.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/27059#comment:1>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/065.109bb4c4b094987f6b32066da82dc490%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
