[issue15307] Patch for --symlink support in pyvenv with framework python
Roundup Robot added the comment: New changeset 4610ac42130e by Ned Deily in branch 'default': Issue #15678: Fix menu customization for IDLE started from OS X http://hg.python.org/cpython/rev/4610ac42130e -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: There is no NEWS entry, I'll commit one when I get home. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset b79d276041a8 by Vinay Sajip in branch 'default': Closes #15307: symlinks now work on OS X with framework Python builds. Patch by Ronald Oussoren. http://hg.python.org/cpython/rev/b79d276041a8 -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: The test failure in test_osx_env is in itself fairly harmless because PYTHONEXECUTABLE is only used to ensure IDLE.app works correct, and IDLE.app doesn't use the pythonw executable and hence won't hit the code that special-cases the venv location. That said, the failure should be removed and points to a real issue in my patch: with my patch calculate_path replaces the value set by Py_SetProgramName and that is a real problem that could break 3th-party code without a good reason. I've attached v5 of the patch, this fixes the test_osx_env failure by moving the the code that uses __PYVENV_LAUNCHER__ to main.c (where Py_SetProgramName is called) -- Added file: http://bugs.python.org/file26396/venv-symlinks-v6.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: And a final update: don't use TAB characters -- Added file: http://bugs.python.org/file26398/venv-symlinks-v7.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ned Deily n...@acm.org added the comment: v7 looks good to me -- stage: - commit review ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset f954ee489896 by Vinay Sajip in branch 'default': Issue #15307: Skipped test_venv:test_prefixes when run from a venv. http://hg.python.org/cpython/rev/f954ee489896 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ned Deily n...@acm.org added the comment: Fixing sys.executable to point to the stub launcher instead of the interpreter in the fw will also fix other unrelated issues, like making python3-32 work properly in 64-/32-bit builds for IDLE and for tests that spawn interpreters in subprocesses. Today, while the main process is 32-bit, the interpreters in subprocesses are 64-bit. I did a little clean up and fixed up the documentation somewhat; see the revised patch. But building a stock installer and running the tests showed two issues: 1) test_osx_env fails at line 28, checking PYTHONEXECUTABLE 2) when running the tests within a venv, test_venv now fails at line 95; sys.baseprefix != sys.prefix. This did not fail previously Also, I saw another test failure when building and running with a test framework environment that included a relative link. It appeared that _NSGetExecutablePath in pythonw.c returned an unnormalized path: sys.executable '/py/dev/default/b10.7_t10.7_x4.3_cclang_d/fw/./root/bin/python3.3' which caused a test_venv failure. I probably won't have time to look at this again for a few days. -- Added file: http://bugs.python.org/file26373/venv-symlinks-v4.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: I removed the call to realpath from pythonw because resolving symlinks breaks the feature. Realpath also converts relative paths to absolute paths, and that probably explains the new failure you're having. I'll try to find a solution for this, possibly by calling realpath on dirname(argv[0]) instead of the whole path. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: v5 adds test cases for sys.executable and calls realpath on dirname(argv[0]) in pythonw.c and also has the changes in Ned's v4. The test failure for test_venv is expected, the note in http://docs.python.org/dev/library/venv.html explains that sys.base_prefix and sys.base_exec_prefix point to the non-venv Python installation which was used to create the venv. The test fails because it assumes that sys.base_prefix of the created venv is same as sys.prefix of the running python, which is not true when you create a venv in a venv. In particular line 95 of test_venv explicitly tests that sys.prefix == sys.base_prefix, which should fail when running the test within a virtual env. Tweaking the test suite to avoid that failure is beyond the scope of this issue. -- Added file: http://bugs.python.org/file26375/venv-symlinks-v5.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: Ned mentioned that test_venv fails on line 95 when you run test_venv *in* a virtualenv, that is: $ pyvenv testenv $ testenv/bin/python3 -m test.test_venv This fails because sys.prefix != sys.base_prefix for testenv/bin/python3, and that's expected. Unless I'm mistaken nothing is wrong here. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ned Deily n...@acm.org added the comment: That's correct, the failing test was being run from a venv. I see now that what had changed is that the fixes for Issue15241 recently added the test_prefixes test case to test_venv and that fails when the test is run from within a venv. Without that new test case, test_venv didn't fail at 3.3.0b1 when run from a venv. Whether it should is another matter and is out of scope for this issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: As the test is constructed now, that test will fail if run from a venv. It will be a simple matter to skip it if test_venv being run from a venv - I can add that at some point, but I agree it's orthogonal to the crux of this issue. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ned Deily n...@acm.org added the comment: v5 fixes the non normalized path issue. However, the PYTHONEXECUTABLE env var - argv processing is still broken, as detected by the test_osx_env failure. It's definitely caused by interaction between pythonw.c and the main interpreter; if the interpreter is launched directly (not via pythonw), PYTHONEXECUTABLE works correctly. Also, the patch needs to be tab-free: no more tabs for C files! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ned Deily n...@acm.org added the comment: I think you're right that the casts are incorrect. I think the existing cast ia a day one bug in Python 3. The question is why hasn't it been a problem? That area needs fixing up since NSModuleForSymbol, NSLookupAndBindSymbol, and NSLibraryNameForModule are now deprecated interfaces. Also, in the patch, shouldn't the wcsncpy be followed by a wcscpy(tmpbuffer, argv0_path) as well? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: The current code works, and I don't understand why it does. I'd love to get rid of the long deprecated APIs like NSModuleForSymbol as well, but we'll have to ask the RM if that is an acceptable change at this point in the 3.3 release process. BTW. Is adding symlink support on OSX a bug fix or a new feature? I'd prefer to treat this a bug fix, framework builds currently don't support the --symlinks option while they easily could do it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: I've attached a new version of the patch: * copy to tmp buffer instead of argv0_buffer (see comment by Ned) * add include in pythonw.c to avoid compiler warning * use _Py_char2wchar instead of blindly casting a char* to a wchar_t* The behavior is not perfect yet, sys.executable is set to the wrong value (both with and without --symlinks). I haven't checked yet if the value is correct without my patch. -- Added file: http://bugs.python.org/file26364/venv-symlinks-v2.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Ronald Oussoren ronaldousso...@mac.com added the comment: based on code inspection I'd say that sys.executable was broken without my patch as well. The code that sets that value is unchanged from Python 3.2, and that points to the executable inside the Python.app application bundle. I've attached v3 of my patch, that ensures that sys.executable is the path used to start the executable. This gives points to the right executable when using venv and is a nicer (and arguably better) result outside of vent's as well. There should probably be new tests as well that test that sys.executable has the right value in a venv. -- Added file: http://bugs.python.org/file26366/venv-symlinks-v3.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
New submission from Ronald Oussoren ronaldousso...@mac.com: The attached patch ensures that python3.3 -mvenv --symlinks myenv works with framework builds. There are two functional changes: 1) don't call 'realpath' in pythonw.c because realpath resolves symlinks and that breaks the '--symlinks' options because the python command is a symlink with that option (and resolving that gives the path to the python executable in the framework) 2) Look for the __PYVENV_LAUNCHER__ environment variable in Modules/getpath.c and use that in preference on the already calculated value of argv0_path. That code is only active for framework builds I'm not happy with the following line in this patch: +wcsncpy(argv0_path, (wchar_t*)pyvenv_launcher, MAXPATHLEN); That mirrors a similar cast of the result of NSLibraryNameForModule earlier in Modules/getpath.c, but in both cases the input stream is a narrow character string (char, not wchar_t) and wcsncpy expects a wchar_t. The the cast seems to paper over a real problem here. Shouldn't both lines use _Py_char2wchar to convert the char* buffer to a wchar_t* one? -- assignee: ronaldoussoren components: Library (Lib), Macintosh files: venv-symlinks.txt keywords: needs review, patch messages: 165087 nosy: ronaldoussoren priority: normal severity: normal status: open title: Patch for --symlink support in pyvenv with framework python versions: Python 3.3 Added file: http://bugs.python.org/file26332/venv-symlinks.txt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15307] Patch for --symlink support in pyvenv with framework python
Changes by Ronald Oussoren ronaldousso...@mac.com: -- nosy: +ned.deily, vinay.sajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15307 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
