Gregory Szorc <gregory.sz...@gmail.com> added the comment:
I ran into a minor issue with the ctypes extension and the _dyld_shared_cache_contains_path dependency that was added as part of this work (commit e8b1c038b14b5fc8120aab62c9bf5fb840274cb6). When building on Intel macOS 10.15 using an 11.x SDK, the configure check for this symbol passes (#define HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH 1) and the extension compiles (albeit with a lot of deprecation warnings spewing from the Apple SDK headers). However, at run-time, we blow up due to a missing symbol error: ``` 2021-02-21T21:57:21.4116370Z cpython-3.9> clang -bundle -undefined dynamic_lookup -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/_decimal.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/basearith.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/constants.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/context.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/convolute.o build/temp.macosx-11.0-x86_64-3.9/private/var/fol ders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/crt.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/difradix2.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/fnt.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/fourstep.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/io.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/mpalloc.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/mpdecimal .o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/numbertheory.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/sixstep.o build/temp.macosx-11.0-x86_64-3.9/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/Python-3.9.2/Modules/_decimal/libmpdec/transpose.o -L/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/tmpasb48bzk/tools/pyhost/lib -L/usr/local/lib -lm -o build/lib.macosx-11.0-x86_64-3.9/_decimal.cpython-39-darwin.so 2021-02-21T21:57:27.1635100Z cpython-3.9> *** WARNING: renaming "_ctypes" since importing it failed: dlopen(build/lib.macosx-11.0-x86_64-3.9/_ctypes.cpython-39-darwin.so, 2): Symbol not found: __dyld_shared_cache_contains_path 2021-02-21T21:57:27.1637220Z cpython-3.9> Referenced from: build/lib.macosx-11.0-x86_64-3.9/_ctypes.cpython-39-darwin.so (which was built for Mac OS X 11.0) 2021-02-21T21:57:27.1638560Z cpython-3.9> Expected in: /usr/lib/libSystem.B.dylib 2021-02-21T21:57:27.1639770Z cpython-3.9> in build/lib.macosx-11.0-x86_64-3.9/_ctypes.cpython-39-darwin.so ``` (This build output is from https://github.com/indygreg/python-build-standalone/runs/1947611547?check_suite_focus=true) The missing symbol is provided by libdyld.dylib, which doesn't appear to be linked directly by CPython's build system. I'm only able to reproduce this on a GitHub Actions 10.5 machine using the 11.1 SDK: it seems to work fine on both an 11.1 Intel and M1 device. I reckon the correct fix is to have _ctypes link libdyld is this build configuration. I was also able to work around it in python-build-standalone by unsetting HAVE_DYLD_SHARED_CACHE_CONTAINS_PATH (hacky solution but this feature isn't needed by the build that was failing). And of course linking a non-Apple libffi should also work, which is potentially why pyenv/Homebrew/etc users aren't noticing. python-build-standalone uses its own Clang built from source, so it is entirely possible there's something differing between the Apple Clang and my Clang causing this to fail. However, at least 1 other person ran into this in the wild (https://stackoverflow.com/questions/65853539/symbol-not-found-dyld-shared-cache-contains-path-on-osx-after-installl-from), so it might be generic. While I'm here, I've taught python-build-standalone to cross-compile CPython 3.9 from both Intel and M1 Apple devices (without using universal binaries). I also have some (hacky) support for targeting iPhone/iWatch/iTV. If there's interest, I could start sending patches upstream. Let me know! ---------- nosy: +indygreg _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue41100> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
