On Feb 21, 2005, at 2:53 PM, Roger Binns wrote:
Did you read and understand the versioned frameworks blog entry I linked to? What you're saying has basically no relevance to what I said.
I read it all. I see the layout. The section about Mach-O, MH_DYLIB
and dyld tie directly in to what I was asking about ELF, LD_LIBRARY_PATH
and rpath which you said wasn't relevant.
You quoted something about link-time linker support for frameworks and then talk about runtime linker support..
Under certain circumstances, prior to Python 2.3.5/2.4.1, then extensions built on 2.3 would magically work for neither Python 2.3 or 2.4 because they were linked against the 2.4 dylib with 2.3 headers, so you'd get an API mismatch (which is a warning not a hard error, but still..).
Ok, now we are getting closer. How would one make a failed extension like this? On the other platforms extensions end up below the relevant site-packages directory and distutils ensures the correct header and library directories are supplied.
If you have Python 2.X and Python 2.Y both installed, as frameworks, in the same location, then "-framework Python" will pick the version marked as Current by way of symlink, which is the one that is installed last. The headers are picked up by way of -I, which points to the correct headers. If there is a mismatch, then you will have the API of Python 2.X due to correct headers but you will be using the symbols from Python 2.Y because GCC will have linked to the wrong Python.
Extensions linked in this way are stillborn, if loaded into a Python 2.X process then they will end up loading two Pythons in-process and will get "Fatal Python error: Interpreter not initialized (version mismatch?)". If loaded into a Python 2.Y process (which would take some serious effort, since they will be installed to the 2.X site-packages) then they will get "warning: Python C API version mismatch for module".
The fix was to not use "-framework Python" and instead point directly to the correct dylib. These fixes are fixing clearly broken behavior, not adding features or functionality.
On Windows the Python interpretter shared library has a version number as part of the filename - eg python23.dll. Consequently, once built you cannot force an extension to use any other version of Python.
This is true on Mac OS X if NOT using "-undefined dynamic_lookup" (10.2 compatible).
On Linux(ELF) the extensions aren't linked to Python in any way. The
symbols are unresolved in the extension. When the extension is loaded into a process, they are resolved at that point against the symbols
that exist in the running process. Consequently you could try to
build an extension against 2.3 and load it into a process with a
2.4 interpretter. But this never happens in practise because the
extensions are stored in a Python version specific site-packages
directory.
This is true on Mac OS X if using "-undefined dynamic_lookup" (not 10.2 compatible).
When Python is built to target Mac OS X 10.3 or later using the "-undefined dynamic_lookup" linker flag, when using PantherPythonFix on 2.3.0, Python 2.3.5, or Python 2.4.x, then extensions are not even linked to the Python interpreter so this problem goes away and the invariant is preserved.
Ok, so that gets the same behaviour as ELF. And presumably if someone tried hard enough they could end up with an extension for 2.3.x running in a process with 2.4.x Python.
You don't have to try *that* hard, but yes.
On other platforms the extensions always end up in a site-packages directory below the relevant Python version. If you have 3 different Pythons installed, you build your extensions 3 times.
Is there some goal to avoid having to do that on Mac?
No.
I understand your frustration in trying to understand the issues at hand here, but please know that they are tricky but solved. This topic comes up a lot and I'm pretty tired of talking about it.
I guess the issue is that things are done differently, there appear to be some breakages, and then some unstated goals, and finally lots of information about fixes for all this, mixed in with various version numbers. The initial thing that came to my mind was "then why do
things differently". The unstated goals is probably the hardest bit to understand.
Things aren't done differently..
Is there some attempt to make extensions work with multiple versions, or did that happen anyway and was a bug? Is there an attempt to make extensions compiled on one machine work with the same version of Python but installed in a different location on another machine?
There is no attempt to make extensions work on multiple versions of Python. Extensions built without the fix may have had a mismatch between Pythons but they didn't actually work in either Python. Yes it was clearly a bug.
Having extensions that can migrate between Python interpreters installed in different locations (such as inside of an application bundle) is definitely a goal, and was achieved with "-undefined dynamic_lookup" -- but is incompatible with Mac OS X 10.2 and earlier -- so both behaviors have to be supported.
py2app of course prefers relocatable extensions, but it can solve the issue the hard way by rewriting the extension's Mach-O load commands to relocate it at build time, but there are some edge cases where it won't work (if there isn't enough room in the header to accommodate the new load commands, which has a very, very, very low chance of occurring -- I have never seen it nor heard any reports of it).
Jack's list from Friday is a good start since it doesn't require understanding. Being in the form of "if your goal is X, then install Y" helps a lot. It will also need some idea of what environment variables (eg PATH) should be set.
What's hard to understand about "Install this fix if you use Python at all on Mac OS X 10.3"? If you don't do it, you will probably run across a problem eventually.
I'd like to add the following goal to the list:
If your goal is to use any version of Python at all on Mac OS X 10.3, then install PantherPythonFix as soon as possible.
Yes, a tutorial about how to manipulate your PATH and environment plist is certainly useful information.
This can include the Python interpreter, if you are using
a Python interpreter shipped with Mac OS X. Thus your application may be "tightly bound" to a particular major version of Mac OS X if you are using the vendor Python.
What is a major version in this context? (ie is it tightly bound to
version 10 or 10.3.x?) If the former I don't understand why it is even mentioned. If the latter then it appears to be another Mac-ism. The Windows Python works on Win95 onwards. The Linux one will work on that version of Linux and most other ones released after that version.
Like Python, a major version of OS X is the combination of the first *two* numbers in the version. So yes, Mac OS X 10.3 is a major version.
You didn't really include enough context here. What py2app says is that it will never, under any circumstance, include ANY vendor files. In other words, it will not (probably illegally) redistribute any files that Apple dropped on your computer if they are in their original locations.
Therefore, if you build an application with a Python interpreter that was distributed by Apple, then your application *will not embed Python*. So, it will *only* work on machines that have the *same* version of Python installed to the *same location*. Therefore, if Mac OS X 10.4 does not include Python 2.3, then applications built using the standard Mac OS X 10.3 interpreter will no longer work. If it does include Python 2.3, and it is installed in the same location, then it is going to work.
This is *not* a Mac-ism.
-bob
_______________________________________________ Pythonmac-SIG maillist - Pythonmac-SIG@python.org http://mail.python.org/mailman/listinfo/pythonmac-sig
