Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Sun, Nov 2, 2014 at 3:38 PM, Ned Deily n...@acm.org wrote: In article cahu5pryq3xegtd-7ahzmdbwk32nprfxz24dfdm1oj-wnmyj...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: Just checking: is sincos() the same as sin() and cos()? Nm output for my toolchain's libm does show sin() and cos() just not sincos() See, this is what you get when you ask for free help: bad info. Silly me...what was I thinking? sincos isn't the same, as a little of googling informs me. But, as the thread starting here indicates: I did a fairly exhaustive search for the sincos() error code I was getting and that link did not turn up for me at all. Instead I found a reference that seemed to imply that such a macro should be implemented but it was not implemented in GCC-4.8.0. Then again, the link you found doesn't seem to indicate which version of GCC has this feature...maybe it is versions later than 4.8.0... https://sourceware.org/ml/binutils/2010-04/msg00219.html gcc can do an optimization that turns a pair of sin() and cos() calls with the same arguments into a single call to sincos(). And there is such a pair in Python's complexobject.c. As the thread explains, there are some platforms where sin() and cos() are implemented but not sincos(). As suggested in the thread, one approach is to give options to gcc to avoid the optimizations: CFLAGS='-fno-builtin-sin -fno-builtin-cos' That does seem to result in a libpython with no references to sincos. Sounds good...I'll try it if bootstrapping 4.9.0 goes south. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Sat, Nov 1, 2014 at 5:25 PM, Ned Deily n...@acm.org wrote: In article cahu5praf_9rkgyw5yzp4tqs3dzsx0qm70esunuknodcqztm...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: On Sat, Nov 1, 2014 at 1:47 PM, Cyd Haselton chasel...@gmail.com wrote: [...] Sure enough, nm -D libm.so shows that sincos is NOT available in that library on my Android device. Now to figure out what to do about it. [...] UPDATE: After doing a bit of research it looks like it would be easier to build Python without sincos...is that possible? If not, I'll need to figure out how to get bionic libm sources with a proper Makefile (instead of Android's build system) While sin() and/or cos() are primarily used in the expected places in the Python standard library (like the math module), sin() is also used in the implementation of type complex objects, code that is part of the core interpreter. Just checking: is sincos() the same as sin() and cos()? Nm output for my toolchain's libm does show sin() and cos() just not sincos() I see that there is an old, undocumented, and unsupported macro to disable building of complex number support. But more hacking is needed to even get a somewhat working build with it these days and then, without complex support, many tests in the standard library fail and what you end up isn't really Python. So, I think rather than hacking up Python, you should try to fix the broken platform. There seem to be a number of project that claim to support Python on Android. Perhaps they could be of help. *sigh* Well, I had planned on bootstrapping GCC 4.9.0...I guess I'll be doing it sooner rather than later. Interestingly enough, the toolchain sources at android.googlesource.com only have Python 2.7.5...perhaps the reason is related to this issue. If GCC supports building Python in the source tree, I'll try it with that version and report back. FWIW, the macro is WITHOUT_COMPLEX: ./configure [...] CPPFLAGS='-DWITHOUT_COMPLEX' and Lib/optparse.py would need to be patched to comment out its use of complex. Again, good luck! And thanks again for all your help. I'll report back with results, if and when I have them. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
In article cahu5pryq3xegtd-7ahzmdbwk32nprfxz24dfdm1oj-wnmyj...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: Just checking: is sincos() the same as sin() and cos()? Nm output for my toolchain's libm does show sin() and cos() just not sincos() See, this is what you get when you ask for free help: bad info. sincos isn't the same, as a little of googling informs me. But, as the thread starting here indicates: https://sourceware.org/ml/binutils/2010-04/msg00219.html gcc can do an optimization that turns a pair of sin() and cos() calls with the same arguments into a single call to sincos(). And there is such a pair in Python's complexobject.c. As the thread explains, there are some platforms where sin() and cos() are implemented but not sincos(). As suggested in the thread, one approach is to give options to gcc to avoid the optimizations: CFLAGS='-fno-builtin-sin -fno-builtin-cos' That does seem to result in a libpython with no references to sincos. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
In article CAHu5PrbYMUzfs1AZYzPaN__51h7Q=4d5bfjm2eqofx90okx...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: So, after trying various ways to add that flag before lpython in setup.py, I stripped all --allow-shlib-undefined --no-allow-shlib-undefined feom the Makefile, ran make clean, and make. I still get the following error: Modules/python.o \ -lc -ldl -lm -L. -lpython2.7 -lm ./libpython2.7.so: undefined reference to `sincos' collect2: error: ld returned 1 exit status make: *** [python] Error 1 I don't understand why you are seeing that problem *unless you might be running into this issue that I found by searching for sincos Android: https://code.google.com/p/android/issues/detail?id=38423 That looks awfully suspicious. If it isn't that, I don't know what to tell you. You could open an issue on the Python bug tracker (bugs.python.org) but we don't officially support Android so I doubt that will help much. If you haven't already, you might try asking on some Android forums; I know other oddities of building things on Android have been reported. Good luck! -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Sat, Nov 1, 2014 at 5:52 AM, Ned Deily n...@acm.org wrote: In article CAHu5PrbYMUzfs1AZYzPaN__51h7Q=4d5bfjm2eqofx90okx...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: So, after trying various ways to add that flag before lpython in setup.py, I stripped all --allow-shlib-undefined --no-allow-shlib-undefined feom the Makefile, ran make clean, and make. I still get the following error: Modules/python.o \ -lc -ldl -lm -L. -lpython2.7 -lm ./libpython2.7.so: undefined reference to `sincos' collect2: error: ld returned 1 exit status make: *** [python] Error 1 I don't understand why you are seeing that problem *unless you might be running into this issue that I found by searching for sincos Android: https://code.google.com/p/android/issues/detail?id=38423 That looks awfully suspicious. If it isn't that, I don't know what to tell you. You could open an issue on the Python bug tracker (bugs.python.org) but we don't officially support Android so I doubt that will help much. If you haven't already, you might try asking on some Android forums; I know other oddities of building things on Android have been reported. Good luck! -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list Hammer - Head - Nail. Sure enough, nm -D libm.so shows that sincos is NOT available in that library on my Android device. Now to figure out what to do about it. Nice find! Cyd -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Sat, Nov 1, 2014 at 1:47 PM, Cyd Haselton chasel...@gmail.com wrote: On Sat, Nov 1, 2014 at 5:52 AM, Ned Deily n...@acm.org wrote: In article CAHu5PrbYMUzfs1AZYzPaN__51h7Q=4d5bfjm2eqofx90okx...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: So, after trying various ways to add that flag before lpython in setup.py, I stripped all --allow-shlib-undefined --no-allow-shlib-undefined feom the Makefile, ran make clean, and make. I still get the following error: Modules/python.o \ -lc -ldl -lm -L. -lpython2.7 -lm ./libpython2.7.so: undefined reference to `sincos' collect2: error: ld returned 1 exit status make: *** [python] Error 1 I don't understand why you are seeing that problem *unless you might be running into this issue that I found by searching for sincos Android: https://code.google.com/p/android/issues/detail?id=38423 That looks awfully suspicious. If it isn't that, I don't know what to tell you. You could open an issue on the Python bug tracker (bugs.python.org) but we don't officially support Android so I doubt that will help much. If you haven't already, you might try asking on some Android forums; I know other oddities of building things on Android have been reported. Good luck! -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list Hammer - Head - Nail. Sure enough, nm -D libm.so shows that sincos is NOT available in that library on my Android device. Now to figure out what to do about it. Nice find! Cyd UPDATE: After doing a bit of research it looks like it would be easier to build Python without sincos...is that possible? If not, I'll need to figure out how to get bionic libm sources with a proper Makefile (instead of Android's build system) -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
In article cahu5praf_9rkgyw5yzp4tqs3dzsx0qm70esunuknodcqztm...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: On Sat, Nov 1, 2014 at 1:47 PM, Cyd Haselton chasel...@gmail.com wrote: [...] Sure enough, nm -D libm.so shows that sincos is NOT available in that library on my Android device. Now to figure out what to do about it. [...] UPDATE: After doing a bit of research it looks like it would be easier to build Python without sincos...is that possible? If not, I'll need to figure out how to get bionic libm sources with a proper Makefile (instead of Android's build system) While sin() and/or cos() are primarily used in the expected places in the Python standard library (like the math module), sin() is also used in the implementation of type complex objects, code that is part of the core interpreter. I see that there is an old, undocumented, and unsupported macro to disable building of complex number support. But more hacking is needed to even get a somewhat working build with it these days and then, without complex support, many tests in the standard library fail and what you end up isn't really Python. So, I think rather than hacking up Python, you should try to fix the broken platform. There seem to be a number of project that claim to support Python on Android. Perhaps they could be of help. FWIW, the macro is WITHOUT_COMPLEX: ./configure [...] CPPFLAGS='-DWITHOUT_COMPLEX' and Lib/optparse.py would need to be patched to comment out its use of complex. Again, good luck! -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Tue, Oct 28, 2014 at 4:01 PM, Ned Deily n...@acm.org wrote: In article cahu5pra68edgkvfyhtq+srmb6syew9a1sabb5cemluzqbs5...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: On Tue, Oct 28, 2014 at 3:11 AM, Ned Deily n...@acm.org wrote: In article CAHu5PrY-T=DT3wOg-Y+Va9WgwBV3w9d7M-tu=_n-cngwssz...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: [...] I'm building python on an Android device in the KBOX environment...which simulates a Unix type filesystem. Python isn't installed; I'm building from sources (2.7.8) with GCC 4.8.0 and make. The problem with the LDFLAGS approach is that some of the libraries that must be linked (-lc -ldl) do not need the --allow-shlib-undefined option...it's only the lpython2.7 that does. Any way to do this? Sorry, I have no experience with that environment nor an understanding of why you need to specify --allow-shlib-undefined (it seems to be the default in some versions of ld). You could look at and, if necessary, modify Lib/distutils, the part of the Python standard library that builds extension modules from setup.py. No need to apologize. Also no need to have an understanding of the environment; with a few rare exceptions it behaves just like a Unix/Linux one. The reason why I need to specify --allow-shlib-undefined is for the python library; it's throwing undefined references to sincos even though the symbol is there. Specifying that option is the only fix I've found. As mentioned earlier, i'm not familiar with python or its build system; which file in the distutils dir do I need to modify? Perhaps I should apologize for not asking earlier what you are really trying to do before suggesting heading down the path of modifying Distutils :) So the issue is with an undefined reference to sincos? It appears that that routine is often supplied in libm. Is that the case on your platform? And, if so, the right fix might be to supply it manually or, better, ensure that Python supplies it as a result of running ./configure. Also, what version of Python are you building? -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list So, after trying various ways to add that flag before lpython in setup.py, I stripped all --allow-shlib-undefined --no-allow-shlib-undefined feom the Makefile, ran make clean, and make. I still get the following error: Modules/python.o \ -lc -ldl -lm -L. -lpython2.7 -lm ./libpython2.7.so: undefined reference to `sincos' collect2: error: ld returned 1 exit status make: *** [python] Error 1 Should I repost this to the dev mailing list? -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Tue, Oct 28, 2014 at 4:01 PM, Ned Deily n...@acm.org wrote: In article cahu5pra68edgkvfyhtq+srmb6syew9a1sabb5cemluzqbs5...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: On Tue, Oct 28, 2014 at 3:11 AM, Ned Deily n...@acm.org wrote: In article CAHu5PrY-T=DT3wOg-Y+Va9WgwBV3w9d7M-tu=_n-cngwssz...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: [...] I'm building python on an Android device in the KBOX environment...which simulates a Unix type filesystem. Python isn't installed; I'm building from sources (2.7.8) with GCC 4.8.0 and make. The problem with the LDFLAGS approach is that some of the libraries that must be linked (-lc -ldl) do not need the --allow-shlib-undefined option...it's only the lpython2.7 that does. Any way to do this? Sorry, I have no experience with that environment nor an understanding of why you need to specify --allow-shlib-undefined (it seems to be the default in some versions of ld). You could look at and, if necessary, modify Lib/distutils, the part of the Python standard library that builds extension modules from setup.py. No need to apologize. Also no need to have an understanding of the environment; with a few rare exceptions it behaves just like a Unix/Linux one. The reason why I need to specify --allow-shlib-undefined is for the python library; it's throwing undefined references to sincos even though the symbol is there. Specifying that option is the only fix I've found. As mentioned earlier, i'm not familiar with python or its build system; which file in the distutils dir do I need to modify? Perhaps I should apologize for not asking earlier what you are really trying to do before suggesting heading down the path of modifying Distutils :) So the issue is with an undefined reference to sincos? It appears that that routine is often supplied in libm. Is that the case on your platform? Yes it is. And, if so, the right fix might be to supply it manually or, better, ensure that Python supplies it as a result of running ./configure. If the Makefile generated is a result of running ./configure, then Python is indeed finding the library...what it is doing with it after finding it may be something else entirely. I've tried everything from adding the -Wl,--no-allow-shlib-undefined option to adding #include math.h to the complexobject.c code...I still get an undefined reference to sincos. Running grep sincos libpython2.7.* turns up the offending symbol (function? declaration?) but re-running make after that still throws the same error. The only fix I've found is adding the -Wl,--allow-shlib-undefined before the python library...which I do by hacking the Makefile. That allows the build to continue, which it does until it hits the part where setup.py is run. Also, what version of Python are you building? 2.7.8. I thought that it would be easier to build it on Android -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
In article CAHu5PrY-T=DT3wOg-Y+Va9WgwBV3w9d7M-tu=_n-cngwssz...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: [...] I'm building python on an Android device in the KBOX environment...which simulates a Unix type filesystem. Python isn't installed; I'm building from sources (2.7.8) with GCC 4.8.0 and make. The problem with the LDFLAGS approach is that some of the libraries that must be linked (-lc -ldl) do not need the --allow-shlib-undefined option...it's only the lpython2.7 that does. Any way to do this? Sorry, I have no experience with that environment nor an understanding of why you need to specify --allow-shlib-undefined (it seems to be the default in some versions of ld). You could look at and, if necessary, modify Lib/distutils, the part of the Python standard library that builds extension modules from setup.py. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Tue, Oct 28, 2014 at 3:11 AM, Ned Deily n...@acm.org wrote: In article CAHu5PrY-T=DT3wOg-Y+Va9WgwBV3w9d7M-tu=_n-cngwssz...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: [...] I'm building python on an Android device in the KBOX environment...which simulates a Unix type filesystem. Python isn't installed; I'm building from sources (2.7.8) with GCC 4.8.0 and make. The problem with the LDFLAGS approach is that some of the libraries that must be linked (-lc -ldl) do not need the --allow-shlib-undefined option...it's only the lpython2.7 that does. Any way to do this? Sorry, I have no experience with that environment nor an understanding of why you need to specify --allow-shlib-undefined (it seems to be the default in some versions of ld). You could look at and, if necessary, modify Lib/distutils, the part of the Python standard library that builds extension modules from setup.py. -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list No need to apologize. Also no need to have an understanding of the environment; with a few rare exceptions it behaves just like a Unix/Linux one. The reason why I need to specify --allow-shlib-undefined is for the python library; it's throwing undefined references to sincos even though the symbol is there. Specifying that option is the only fix I've found. As mentioned earlier, i'm not familiar with python or its build system; which file in the distutils dir do I need to modify? -- https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
In article cahu5pra68edgkvfyhtq+srmb6syew9a1sabb5cemluzqbs5...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: On Tue, Oct 28, 2014 at 3:11 AM, Ned Deily n...@acm.org wrote: In article CAHu5PrY-T=DT3wOg-Y+Va9WgwBV3w9d7M-tu=_n-cngwssz...@mail.gmail.com, Cyd Haselton chasel...@gmail.com wrote: [...] I'm building python on an Android device in the KBOX environment...which simulates a Unix type filesystem. Python isn't installed; I'm building from sources (2.7.8) with GCC 4.8.0 and make. The problem with the LDFLAGS approach is that some of the libraries that must be linked (-lc -ldl) do not need the --allow-shlib-undefined option...it's only the lpython2.7 that does. Any way to do this? Sorry, I have no experience with that environment nor an understanding of why you need to specify --allow-shlib-undefined (it seems to be the default in some versions of ld). You could look at and, if necessary, modify Lib/distutils, the part of the Python standard library that builds extension modules from setup.py. No need to apologize. Also no need to have an understanding of the environment; with a few rare exceptions it behaves just like a Unix/Linux one. The reason why I need to specify --allow-shlib-undefined is for the python library; it's throwing undefined references to sincos even though the symbol is there. Specifying that option is the only fix I've found. As mentioned earlier, i'm not familiar with python or its build system; which file in the distutils dir do I need to modify? Perhaps I should apologize for not asking earlier what you are really trying to do before suggesting heading down the path of modifying Distutils :) So the issue is with an undefined reference to sincos? It appears that that routine is often supplied in libm. Is that the case on your platform? And, if so, the right fix might be to supply it manually or, better, ensure that Python supplies it as a result of running ./configure. Also, what version of Python are you building? -- Ned Deily, n...@acm.org -- https://mail.python.org/mailman/listinfo/python-list
Build Question: How to Add -Wl,--option Before Objects In Setup.py?
I need to add a linker option to the command(s) run by setup.py when
building various objects. I'm not familiar with Python at all, so I
basically copied and modified a line from one area of the script to
another
ext_modules=[Extension('_struct', ['_struct.c'], extra_link_args =
['Wl,--allow-shlib-undefined'])],
*snip*
Unfortunately this seems to append the option to the end of the
command line. What's the best (fastest) way to add it before the
object being built (objectname.o)?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
In article
CAHu5PrbHSob2r-zOTCtrYiqdudU_q+4mDXOXzp=_rejp7kh...@mail.gmail.com,
Cyd Haselton chasel...@gmail.com wrote:
I need to add a linker option to the command(s) run by setup.py when
building various objects. I'm not familiar with Python at all, so I
basically copied and modified a line from one area of the script to
another
ext_modules=[Extension('_struct', ['_struct.c'], extra_link_args =
['Wl,--allow-shlib-undefined'])],
*snip*
Unfortunately this seems to append the option to the end of the
command line. What's the best (fastest) way to add it before the
object being built (objectname.o)?
It depends on what system and build tools that you are using and that
the Python you are using was built with but, in general on most
POSIX-like systems, one way to do it should be to supply it via an
LDFLAGS environment variable. The safest approach would be to get the
default value of LDFLAGS for this Python instance, append your
additional values to it, and pass it back into the setup.py build. You
can do that all in one line:
LDFLAGS=$(python -c 'import
sysconfig;print(sysconfig.get_config_var(LDFLAGS))')
-Wl,--allow-shlib-undefined python setup.py build
--
Ned Deily,
n...@acm.org
--
https://mail.python.org/mailman/listinfo/python-list
Re: Build Question: How to Add -Wl, --option Before Objects In Setup.py?
On Mon, Oct 27, 2014 at 3:39 PM, Ned Deily n...@acm.org wrote:
In article
CAHu5PrbHSob2r-zOTCtrYiqdudU_q+4mDXOXzp=_rejp7kh...@mail.gmail.com,
Cyd Haselton chasel...@gmail.com wrote:
I need to add a linker option to the command(s) run by setup.py when
building various objects. I'm not familiar with Python at all, so I
basically copied and modified a line from one area of the script to
another
ext_modules=[Extension('_struct', ['_struct.c'], extra_link_args =
['Wl,--allow-shlib-undefined'])],
*snip*
Unfortunately this seems to append the option to the end of the
command line. What's the best (fastest) way to add it before the
object being built (objectname.o)?
It depends on what system and build tools that you are using and that
the Python you are using was built with but, in general on most
POSIX-like systems, one way to do it should be to supply it via an
LDFLAGS environment variable. The safest approach would be to get the
default value of LDFLAGS for this Python instance, append your
additional values to it, and pass it back into the setup.py build. You
can do that all in one line:
LDFLAGS=$(python -c 'import
sysconfig;print(sysconfig.get_config_var(LDFLAGS))')
-Wl,--allow-shlib-undefined python setup.py build
--
Ned Deily,
n...@acm.org
I'm building python on an Android device in the KBOX
environment...which simulates a Unix type filesystem.
Python isn't installed; I'm building from sources (2.7.8) with GCC
4.8.0 and make.
The problem with the LDFLAGS approach is that some of the libraries
that must be linked (-lc -ldl) do not need the --allow-shlib-undefined
option...it's only the lpython2.7 that does. Any way to do this?
Cyd Haselton
--
https://mail.python.org/mailman/listinfo/python-list
