Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
Damian, Lots of good info there. Thanks very much. -- Lou --- Damian Eads [EMAIL PROTECTED] wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). [cut] -- Lou Pecora, my views are my own. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
David Cournapeau wrote: On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). Unfortunately, this does not work. Distutils only knows how to build python extensions, not shared libraries. Depending on the platform, this is not the same thing, and mac os X is such a platform where both are not the same. cheers, David Really? distutils generates .so files for me, which I assume are shared libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not generate a shared library on a mac? Damian ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
On Feb 12, 2008 12:41 AM, Damian Eads [EMAIL PROTECTED] wrote: Robert Kern wrote: On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote: David Cournapeau wrote: On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). Unfortunately, this does not work. Distutils only knows how to build python extensions, not shared libraries. Depending on the platform, this is not the same thing, and mac os X is such a platform where both are not the same. cheers, David Really? distutils generates .so files for me, which I assume are shared libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not generate a shared library on a mac? Python extension modules are shared libraries, yes. But they must follow a particular format, namely exposing a correct initmodulename function. distutils/setuptools only maked Python extension modules, not arbitrary shared libraries. Perhaps I was a bit too liberal in my use of the term extension module. Several small libraries for a project at work do not define the standard initmodulename function, and yet they build with distutils. I can load them into ctypes without any hitches. Perhaps distutils does not check for the presence of the initmodulename function and required data structures? I'll admit I may be abusing distutils by using it for something for which it wasn't designed. Yup. It usually works on Linux because the initmodulename bit isn't checked. On Windows, it is, and the build will fail. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
Robert Kern wrote: On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote: David Cournapeau wrote: On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). Unfortunately, this does not work. Distutils only knows how to build python extensions, not shared libraries. Depending on the platform, this is not the same thing, and mac os X is such a platform where both are not the same. cheers, David Really? distutils generates .so files for me, which I assume are shared libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not generate a shared library on a mac? Python extension modules are shared libraries, yes. But they must follow a particular format, namely exposing a correct initmodulename function. distutils/setuptools only maked Python extension modules, not arbitrary shared libraries. Perhaps I was a bit too liberal in my use of the term extension module. Several small libraries for a project at work do not define the standard initmodulename function, and yet they build with distutils. I can load them into ctypes without any hitches. Perhaps distutils does not check for the presence of the initmodulename function and required data structures? I'll admit I may be abusing distutils by using it for something for which it wasn't designed. Damian ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote: David Cournapeau wrote: On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). Unfortunately, this does not work. Distutils only knows how to build python extensions, not shared libraries. Depending on the platform, this is not the same thing, and mac os X is such a platform where both are not the same. cheers, David Really? distutils generates .so files for me, which I assume are shared libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not generate a shared library on a mac? Python extension modules are shared libraries, yes. But they must follow a particular format, namely exposing a correct initmodulename function. distutils/setuptools only maked Python extension modules, not arbitrary shared libraries. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
On Feb 12, 2008 12:14 AM, Damian Eads [EMAIL PROTECTED] wrote: David Cournapeau wrote: On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). Unfortunately, this does not work. Distutils only knows how to build python extensions, not shared libraries. Depending on the platform, this is not the same thing, and mac os X is such a platform where both are not the same. cheers, David Really? distutils generates .so files for me, which I assume are shared libraries. FYI: I'm running Fedora 8 on an x86. Does distutils not generate a shared library on a mac? As to David's point, yes, distutils makes a .so shared library on Macs. This is not the same thing as a dynamic library (on Macs) which is what ctypes needs (on Macs), IIRC. There is a subtle, but important difference between the two. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
On Mon, 2008-02-11 at 22:50 -0700, Damian Eads wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). Unfortunately, this does not work. Distutils only knows how to build python extensions, not shared libraries. Depending on the platform, this is not the same thing, and mac os X is such a platform where both are not the same. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] CTypes: How to incorporate a library with shared library module?
I will be writing some C code that I will compile into a shared library (.so) on my MacOSX computer to use with ctypes. That code will be calling code from a (big) scientific numerical library (Gnu Scientific Library - GSL) to crunch the numbers. But I don't see how I incorporate that code into the .so file so my shared code can get to it when I call it from Python with ctypes. I do _not_ want to make the GSL callable from Python, only from my own C module. I suspect this isn't a ctypes question in particular. I'm hoping to avoid having to tur the whole GSL into a shared library and loading it just to use a few functions. Or avoid having to track down which functions my code will call (all the way down the trees) and rip that out to add to my own shared lib. There's got to be a better way to make use of big, useful libraries when speeding up python with shared lib extension. I hope. Maybe there are ways to do this using a gcc or g++ option. Right now my make file is simply gcc - bundle -flat_namespace -undefined suppress -o mycode.somycode.o gcc -c mycode.c -o mycode.o Any hints appreciated. I will continue googling. Nothing so far. Thanks. -- Lou Pecora, my views are my own. - Never miss a thing. Make Yahoo your homepage.___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
Dear Lou,
You may want to try using distutils or setuputils, which makes compiling
extensions much easier. It does the hard work of finding out which flags
are needed to compile extensions on the host platform. There are many
examples on the web on how to use distutils to build C extensions
(http://docs.python.org/ext/building.html).
PyGSL interfaces with numpy, and it may have what you need. The trouble
with calling GSL directly from ctypes is GSL's input and output is very
structure-oriented, which complicates the python/C logic. Like you
mentioned, GSL is pretty big, and you'd like to avoid loading it in its
entirety.
Even if you write your own shared library module that links against GSL,
GSL might still get loaded in its entirety when your own shared library
gets loaded. There is a mode argument with C's dlopen and the
ctypes.CDLL function. dlopen supports a RTLD_LAZY flag (see man dlopen),
which you can try passing to ctypes.CDLL, but I'm not sure what will
happen since the ctypes documentation makes no mention of it.
Here is a quick example of how to call a C function from python using
ctypes.
/** Sum num_elements numbers and return the result. **/
extern int myfunc(int *numbers, int num_elements) {
int i, sum = 0;
for (i = 0; i num_elements; i++) {
sum += numbers[i];
}
return sum;
}
# First compile the source
$ gcc -DNDEBUG -O2 -g -pipe -Wall -fstack-protector -D_GNU_SOURCE -fPIC
-I/usr/lib/python2.5/site-packages/numpy/core/include
-I/usr/include/python2.5 -c foo.c -o foo.o
# Now link it.
$ gcc -pthread -shared foo.o -L/usr/lib -lpython2.5 -o foo.so
# Now run it in python.
$ ipython
In [1]: import numpy, ctypes
# Load the shared library we just created
In [2]: foo=ctypes.CDLL('./foo.so')
# Create a numpy array of ints
In [3]: A=numpy.array([1,2,3],dtype='int')
# Set the return type we expect.
In [4]: foo.myfunc.restype = ctypes.c_int
# Call the C function.
In [5]: print foo.myfunc(A.ctypes.data, ctypes.c_int(3))
6
I hope this helps.
Damian
Lou Pecora wrote:
I will be writing some C code that I will compile into a shared library
(.so) on my MacOSX computer to use with ctypes. That code will be
calling code from a (big) scientific numerical library (Gnu Scientific
Library - GSL) to crunch the numbers. But I don't see how I incorporate
that code into the .so file so my shared code can get to it when I call
it from Python with ctypes. I do _not_ want to make the GSL callable
from Python, only from my own C module. I suspect this isn't a ctypes
question in particular. I'm hoping to avoid having to tur the whole GSL
into a shared library and loading it just to use a few functions. Or
avoid having to track down which functions my code will call (all the
way down the trees) and rip that out to add to my own shared lib.
There's got to be a better way to make use of big, useful libraries when
speeding up python with shared lib extension. I hope.
Maybe there are ways to do this using a gcc or g++ option. Right now my
make file is simply
gcc - bundle -flat_namespace -undefined suppress -o mycode.somycode.o
gcc -c mycode.c -o mycode.o
Any hints appreciated. I will continue googling. Nothing so far. Thanks.
-- Lou Pecora, my views are my own.
Never miss a thing. Make Yahoo your homepage.
http://us.rd.yahoo.com/evt=51438/*http://www.yahoo.com/r/hs
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
