Re: [Numpy-discussion] String sort
A Monday 11 February 2008, Charles R Harris escrigué:
On Feb 11, 2008 1:15 PM, Francesc Altet [EMAIL PROTECTED] wrote:
Here are the results of running it in several platforms:
1) My laptop: Ubuntu 7.1 (gcc 4.1.3, Pentium 4 @ 2 GHz)
Benchmark with 100 strings of size 15
C qsort with C style compare: 2.45
C qsort with Python style compare: 2.44
NumPy newqsort: 0.65
Wow, what a difference.
Yeah. This is why I got so excited initially. It's unfortunate that
most of the speed-up in newqsort in this case is probably due to a
possible flaw in qsort implementation for the combination
Ubuntu/Pentium4. On the positive side, it is nice to see that other
distros/processors have a decent performance on system qsort ;-)
* I'd definitely keep memcpy by default. From my timings, it looks
like the best option for all platforms.
OK. Was that just for the copies, or was it for the swaps also? I ran
a version of swap using memcpy on my machine and the sort was about
half as fast for 8 character strings.
No, only for the copies. For the swaps, this memcpy-based version:
#define swap_string(s1, s2, len) { \
memcpy((vp), (s2), (len)); \
memcpy((s2), (s1), (len)); \
memcpy((s1), (vp), (len)); \
}
performs extremely bad on my systems:
Pentium4/Ubuntu 7.10:
* newqsort with the loop version of swap_string: 0.65 s
* newqsort with the memcpy version of swap_string: 9.14 s
Opteron/SuSe LE 10.3:
* newqsort with the loop version of swap_string: 0.59 s
* newqsort with the memcpy version of swap_string: 8.71 s
So, it seems that the nice newqsort performance is extremely dependent
on the loop version of swap_string.
I hope the benchmark will behave well in your platform too (i.e.
newqsort will perform the best ;)
I'll check it out when I get home. As I say, it was running about 10%
slower on my machine, but if it does better on most platforms it is
probably the way to go. We can always change it in the future when
everyone is running on quantum computers.
Quantum computers? Oh, I can't wait for mine ;-)
--
0,0 Francesc Altet http://www.carabos.com/
V V Cárabos Coop. V. Enjoy Data
-
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Median advice
On 12/02/2008, Matthew Brett [EMAIL PROTECTED] wrote: Suggestion 1: def median(a, axis=0, out=None) [...] Suggestion 2: def median(a, axis=0, scratch_input=False) No reason not to combine the two. It's a pretty straightforward modification to do the sorting in place, and it could make a lot more difference to the runtime (and memory usage) than an output array. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C Extensions, CTypes and external code librarie
--- Albert Strasheim [EMAIL PROTECTED] wrote: Hello, Sounds about right. I don't know the Mac that well as far as the various types of dynamic libraries go, so just check that you're working with the right type of libraries, but you've got the right idea. Regards, Albert Thanks, Albert. I'll report back to this thread when I give it a try. -- Lou Pecora, my views are my own. Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C Extensions, CTypes and external code libraries
--- Jon Wright [EMAIL PROTECTED] wrote:
Lou Pecora wrote:
... This appears to be the way
static and shared libraries work, especially on
Mac OS
X, maybe elsewhere.
Have you tried linking against a GSL static library?
I don't have a mac,
but most linkers only pull in the routines you need.
For example, using
windows and mingw:
#include stdio.h
#include gsl/gsl_sf_bessel.h
int main (void)
{ double x = 5.0;
double y = gsl_sf_bessel_J0 (x);
printf (J0(%g) = %.18e\n, x, y);
return 0; }
...compiles to a.exe which outputs:
J0(5) = -1.775967713143382900e-001
Yes, I know about this approach if I am making an
executable. But I want to make my code into a shared
library (my code will not have a main, just the
functions I write) and, if possible, let my code call
the GSL code it needs from the C function I write
(i.e. no python interface). If what you did can be
done for a shared library, then that would be great.
However, I am ignorant of how to do this. I will try
to make my shared library using gcc and then add the
GSL library using the -l option as someone else
suggested. Maybe that will work. I'll report back.
I have been searching for info on the right approach
to this on the Mac, since, as I understand, Mac OS X
does make a distinction between shared libraries and
dynamic libraries (which I don't understand fully).
Thanks.
-- Lou Pecora, my views are my own.
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now.
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] sort method raises unexpected error with axis=None
On 12/02/2008, Matthew Brett [EMAIL PROTECTED] wrote: Is it possible, in fact, to do an inplace sort on an array with axis=None (ie flat sort)? It is, sometimes; just make an array object to point to the flattened version and sort that: In [16]: b = a[:] In [17]: b.shape = (16,) In [18]: b.sort() This is not always possible, depending on the arrangement of a in memory. An efficient way to handle in-place (or out-of-place, come to think of it) median along multiple axes is actually to take medians along all axes in succession. That saves you some sorting effort, and some programming effort, and doesn't require in-place multidimensional sorting: In [24]: def all_axes_median(a): : if len(a.shape)1: : return all_axes_median(N.median(a)) : else: : return N.median(a) : : In [26]: all_axes_median(N.reshape(N.arange(32),(2,4,2,-1))) Out[26]: 15.5 Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] sort method raises unexpected error with axis=None
On 12/02/2008, Anne Archibald [EMAIL PROTECTED] wrote: An efficient way to handle in-place (or out-of-place, come to think of it) median along multiple axes is actually to take medians along all axes in succession. That saves you some sorting effort, and some programming effort, and doesn't require in-place multidimensional sorting: Aargh. Sorry. No, that doesn't work: In [28]: all_axes_median(N.reshape([1,5,6,7],(2,2))) Out[28]: 4.75 Oops. Anne ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] sort method raises unexpected error with axis=None
Hi, To rephrase: Is it possible, in fact, to do an inplace sort on an array with axis=None (ie flat sort)? Should the sort method have its docstring changed to reflect the fact that axis=None is not valid? Matthew On Feb 10, 2008 7:50 PM, Matthew Brett [EMAIL PROTECTED] wrote: Hi, I just noticed this: From the sort method docstring: axis : integer Axis to be sorted along. None indicates that the flattened array should be used. Default is -1. In [40]: import numpy as N In [41]: a = N.arange(10) In [42]: N.sort(a, None) Out[42]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) In [43]: a.sort(None) --- TypeError Traceback (most recent call last) /home/mb312/ipython console in module() TypeError: an integer is required Perhaps the sort method is calling the c code directly, and this is not checking for axis=None? Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Median advice
Hi, On Feb 12, 2008 8:48 PM, Anne Archibald [EMAIL PROTECTED] wrote: On 12/02/2008, Matthew Brett [EMAIL PROTECTED] wrote: Suggestion 1: def median(a, axis=0, out=None) [...] Suggestion 2: def median(a, axis=0, scratch_input=False) No reason not to combine the two. It's a pretty straightforward modification to do the sorting in place, and it could make a lot more difference to the runtime (and memory usage) than an output array. Yes, true. I'll include it unless anyone pipes up to object. Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] String sort
Hi, I have a Opteron 248 (2.66GHz) that with gcc 4.1.0 (SUSE10.1?) that gives C qsort with C style compare: 0.65 C qsort with Python style compare: 0.64 NumPy newqsort: 0.36 I did notice that -O3 was essential to get the performance gain as -O2 gave: C qsort with C style compare: 0.69 C qsort with Python style compare: 0.70 NumPy newqsort: 0.61 Bruce On Feb 12, 2008 10:07 AM, Francesc Altet [EMAIL PROTECTED] wrote: A Monday 11 February 2008, Charles R Harris escrigué: I'll check it out when I get home. As I say, it was running about 10% slower on my machine, but if it does better on most platforms it is probably the way to go. We can always change it in the future when everyone is running on quantum computers. We've done some testing on newqsort in several computers in our company. Here are the results for ordering a list with 1 million of strings of length 15 filled with random information (using C rand()): 1) Ubuntu 7.1 (gcc 4.1.3, -O3, Intel Pentium 4 @ 2 GHz) C qsort with C style compare: 2.45 C qsort with Python style compare: 2.44 NumPy newqsort: 0.65 2) Windows XP (SP2) (MSVC 7.1, /Ox, Intel Pentium 4 @ 2 GHz) C qsort with C style compare: 0.971000 C qsort with Python style compare: 0.962000 NumPy newqsort: 0.921000 3) SuSe LE 10.3 (gcc 4.2.1, -O3, AMD Opteron @ 2 GHz) C qsort with C style compare: 0.64 C qsort with Python style compare: 0.60 NumPy newqsort: 0.59 4) Debian 4.2.2 (lenny) (gcc 4.2.3, -O3, Intel Pentium 4 @ 3.2 GHz) C qsort with C style compare: 1.77 C qsort with Python style compare: 1.75 NumPy newqsort: 0.44 5) Mandriva 2008.0 (gcc 4.2.2, -O3, Intel Core2 Duo @ 1.5 GHz) C qsort with C style compare: 1.59 C qsort with Python style compare: 1.55 NumPy newqsort: 0.51 6) Ubuntu 7.1 (gcc 4.1.3, -O3, Intel Pentium 4 @ 2.5 GHz) C qsort with C style compare: 1.89 C qsort with Python style compare: 1.90 NumPy newqsort: 0.50 7) Ubuntu 7.1 (gcc 4.1.2, -O3, PowerPC 3 @ 1.3 GHz) C qsort with C style compare: 3.03 C qsort with Python style compare: 2.97 NumPy newqsort: 1.04 8) MacOSX 10.4 (Tiger) (gcc 4.0.1, -O3, PowerPC 3 @ 1.3 GHz) C qsort with C style compare: 1.56 C qsort with Python style compare: 1.51 NumPy newqsort: 1.22 All benchmarks have been run using the attached benchmark (if anybody else wants to join the fiesta, please report back your feedback). Summarizing, one can say a couple of things: * Recent Debian distros and derivatives (Ubuntu) as well as Mandriva are suffering from a innefficient system qsort (at least the implementation for strings). SuSe Linux Enterprise 10.3 seems to have solved this. And Windows XP (SP2) and MacOSX (Tiger) looks like they have a relatively efficient implementation of qsort. * The newqsort performs the best on all the platforms we have checked (ranging from a 5% of improvement on Opteron/SuSe, up to 3.8x with some Pentium4/Ubuntu systems). All in all, I'd also say that newqsort would be a good candidate to be put into NumPy. Cheers, -- 0,0 Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data - ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C Extensions, CTypes and external code libraries
Lou Pecora wrote:
--- Jon Wright [EMAIL PROTECTED] wrote:
Lou Pecora wrote:
... This appears to be the way
static and shared libraries work, especially on
Mac OS
X, maybe elsewhere.
Have you tried linking against a GSL static library?
I don't have a mac,
but most linkers only pull in the routines you need.
For example, using
windows and mingw:
#include stdio.h
#include gsl/gsl_sf_bessel.h
int main (void)
{ double x = 5.0;
double y = gsl_sf_bessel_J0 (x);
printf (J0(%g) = %.18e\n, x, y);
return 0; }
...compiles to a.exe which outputs:
J0(5) = -1.775967713143382900e-001
Yes, I know about this approach if I am making an
executable. But I want to make my code into a shared
library (my code will not have a main, just the
functions I write) and, if possible, let my code call
the GSL code it needs from the C function I write
(i.e. no python interface). If what you did can be
done for a shared library, then that would be great.
However, I am ignorant of how to do this. I will try
to make my shared library using gcc and then add the
GSL library using the -l option as someone else
suggested. Maybe that will work.
Oh, I may have misunderstood what you are trying to do then. You just
want to call a shared library from another shared library ? This is
possible on any platform supporting shared library (including but not
limited to mac os x, windows, linux, most not ancient unices).
As Albert said, just do (with gcc):
gcc -shared -o mysharedlib mysharedlib.c -lgsl
This works on mac os X as well as linux (and even windows with mingw).
If you want to link the gsl statically (so that your own lib does not
depend on the gsl anymore), you have use a trick to tell gcc to link the
gsl:
gcc -shared -o mysharedlib mysharedlib.c -Wl,-Bstatic -lgsl -Wl,-Bdynamic
-Wl is used by gcc to pass option to the linker directly. -Bstatic says
that all link options after will be static. You have to use -Bdynamic
after, to avoid linking everything static (like gcc runtime, the C lib,
which are automatically linked by default by gcc: it is almost always a
bad idea to statically link those).
In the first case, mysharedlib.so will need libgsl.so:
ldd mysharedlib.so - linux-gate.so.1 = (0xe000)
libgsl.so.0 = /usr/lib/libgsl.so.0 (0xb7ddb000)
libc.so.6 = /lib/tls/i686/cmov/libc.so.6 (0xb7c91000)
libm.so.6 = /lib/tls/i686/cmov/libm.so.6 (0xb7c6b000)
/lib/ld-linux.so.2 (0x8000)
In the second case:
linux-gate.so.1 = (0xe000)
libc.so.6 = /lib/tls/i686/cmov/libc.so.6 (0xb7e6)
/lib/ld-linux.so.2 (0x8000)
I don't know if the second method works on mac os X: since it bypass gcc
and goes directly to the linker, which is notably different on mac os X,
you may have to do it differently.
I'll report back.
I have been searching for info on the right approach
to this on the Mac, since, as I understand, Mac OS X
does make a distinction between shared libraries and
dynamic libraries (which I don't understand fully).
To be over-simplistic: shared libraries are linked into the executable,
and all its symbols (function, variables) are solved when you launch the
executable. A dynamic library is not linked into the executable, and can
be loaded at anytime during the execution of the executable. Shared
library are just a way to avoid duplicate code, but are totally
transparent to the code user:
int foo()
{
return bar();
}
If bar is in a shared lib (libbar.so) or in another object code (bar.o),
it does not make a difference for you. With a dynamic lib, on most
unices, you do
hdl = dlopen(libbar.so)
((int)(*bar)()) = dlsym(hdl, bar);
That is you explicitly load the functions you want to use. Without this
scheme, you would have to link your extension to the python executable
when python is built, which is totally impractical of course. IOW,
dynamic libraries are used for plugins, things which can be added to
an executable *after* the executable is built.
On linux (and other unices using the elf binary format), both types of
libraries are built exactly the same. On mac os X (and windows as well),
they are not. Again, this is oversimplification, but you don't need to
know much more in almost all the cases.
cheers,
David
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] String sort
On Feb 12, 2008 9:07 AM, Francesc Altet [EMAIL PROTECTED] wrote: A Monday 11 February 2008, Charles R Harris escrigué: I'll check it out when I get home. As I say, it was running about 10% slower on my machine, but if it does better on most platforms it is probably the way to go. We can always change it in the future when everyone is running on quantum computers. We've done some testing on newqsort in several computers in our company. Here are the results for ordering a list with 1 million of strings of length 15 filled with random information (using C rand()): 1) Ubuntu 7.1 (gcc 4.1.3, -O3, Intel Pentium 4 @ 2 GHz) C qsort with C style compare: 2.45 C qsort with Python style compare: 2.44 NumPy newqsort: 0.65 2) Windows XP (SP2) (MSVC 7.1, /Ox, Intel Pentium 4 @ 2 GHz) C qsort with C style compare: 0.971000 C qsort with Python style compare: 0.962000 NumPy newqsort: 0.921000 3) SuSe LE 10.3 (gcc 4.2.1, -O3, AMD Opteron @ 2 GHz) C qsort with C style compare: 0.64 C qsort with Python style compare: 0.60 NumPy newqsort: 0.59 4) Debian 4.2.2 (lenny) (gcc 4.2.3, -O3, Intel Pentium 4 @ 3.2 GHz) C qsort with C style compare: 1.77 C qsort with Python style compare: 1.75 NumPy newqsort: 0.44 5) Mandriva 2008.0 (gcc 4.2.2, -O3, Intel Core2 Duo @ 1.5 GHz) C qsort with C style compare: 1.59 C qsort with Python style compare: 1.55 NumPy newqsort: 0.51 6) Ubuntu 7.1 (gcc 4.1.3, -O3, Intel Pentium 4 @ 2.5 GHz) C qsort with C style compare: 1.89 C qsort with Python style compare: 1.90 NumPy newqsort: 0.50 7) Ubuntu 7.1 (gcc 4.1.2, -O3, PowerPC 3 @ 1.3 GHz) C qsort with C style compare: 3.03 C qsort with Python style compare: 2.97 NumPy newqsort: 1.04 8) MacOSX 10.4 (Tiger) (gcc 4.0.1, -O3, PowerPC 3 @ 1.3 GHz) C qsort with C style compare: 1.56 C qsort with Python style compare: 1.51 NumPy newqsort: 1.22 All benchmarks have been run using the attached benchmark (if anybody else wants to join the fiesta, please report back your feedback). Summarizing, one can say a couple of things: * Recent Debian distros and derivatives (Ubuntu) as well as Mandriva are suffering from a innefficient system qsort (at least the implementation for strings). SuSe Linux Enterprise 10.3 seems to have solved this. And Windows XP (SP2) and MacOSX (Tiger) looks like they have a relatively efficient implementation of qsort. * The newqsort performs the best on all the platforms we have checked (ranging from a 5% of improvement on Opteron/SuSe, up to 3.8x with some Pentium4/Ubuntu systems). The 3.8 is amazing, isn't it? I've found that the performance also depends on whether I initialize the strings with random or empty. With the random initialization the new sort is ~2x faster. That's fedora 8, core duo, 32 bit OS, gcc 4.1.2. All in all, I'd also say that newqsort would be a good candidate to be put into NumPy. I've merged some sorting tests preparatory to merging the new sorts. There is a release coming up this weekend, I don't know if it is tagged yet, but in any case I plan to merge the new sorts soon. Please help with the testing when I do. Now it's off to paying work. Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] String sort
OK, The new quicksorts are in svn. Francesc, can you check them out? Chuck ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C Extensions, CTypes and external code librarie
Hello, On Feb 12, 2008 6:19 PM, Lou Pecora [EMAIL PROTECTED] wrote: Albert, Yes, I think you got the idea right. I want to call my own C code using CTypes interface, then from within my C code call GSL C code, i.e. a C function calling another C function directly. I do *not* want to go back out through the Python interface. So you are right, I do not want to wrap GSL. It sounds like I can just add something like -lnameofGSLdylib (where I put in the real name of the GSL library after the -l) in my gcc command to make my shared lib. Is that right? Sounds about right. I don't know the Mac that well as far as the various types of dynamic libraries go, so just check that you're working with the right type of libraries, but you've got the right idea. Regards, Albert ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Median advice
On Feb 12, 2008 6:33 AM, Joris De Ridder [EMAIL PROTECTED] wrote: On 12 Feb 2008, at 12:31, Matthew Brett wrote: def median(a, axis=0, out=None) (same signature as max, min etc) I would be slightly in favour of this option. Using the same signature would be convenient in code like def myfunc(myarray, somefunc): # do stuff ... x = somefunc(myarray, axis = 0, out = None) # do more stuff ... where somefunc could be median(), mean(), max(), min(), std() etc. I once wrote this kind of function to provide (small) image filtering. If the same signature is used, there is no need to special-case median(). I realise it's kind of a niche example, though. I'm happy with that use case. The docstring should mention that out= is not memory-optimized like it is for the others, though. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C Extensions, CTypes and external code libraries
Lou Pecora wrote:
... This appears to be the way
static and shared libraries work, especially on Mac OS
X, maybe elsewhere.
Have you tried linking against a GSL static library? I don't have a mac,
but most linkers only pull in the routines you need. For example, using
windows and mingw:
#include stdio.h
#include gsl/gsl_sf_bessel.h
int main (void)
{ double x = 5.0;
double y = gsl_sf_bessel_J0 (x);
printf (J0(%g) = %.18e\n, x, y);
return 0; }
...compiles to a.exe which outputs:
J0(5) = -1.775967713143382900e-001
The stripped executable is about 92 kB in comparison to the 2 mega byte
libgsl.a. Unstripped there are about 150 symbols containing gsl,
compared to 5351 symbols in the library libgsl.a. I just needed to put
-lgsl on the command line and rename $LIB/libgsl.dll.def to
something else so the shared version wasn't found.
In this case the linker has not pulled in all of the library. Presumably
just the parts it needed, including various things like error reporting,
sin, cos, exp etc. Older platforms, including vax and various unix'es
also seemed to behave in the same way in the past.
Are you saying the mac is somehow different? Perhaps they're trying to
hold people to open source ransom, where they have to give away to
source so it can be recompiled when the next OSX escapes ;-)
Cheers,
Jon
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C Extensions, CTypes and external code libraries
Albert Strasheim [EMAIL PROTECTED] wrote: Hello, I only quickly read through the previous thread, but I get that idea that what you want to do is to link your shared library against the the GSL shared library and then access your own library using ctypes. If done like this, you don't need to worry about wrapping GSL or pulling GSL code into your own library. As far as I know, this works exactly like it does when you link an executable against a shared library. If distutils doesn't allow you to do this easily, you could try using SCons's SharedLibrary builder instead. Regards, Albert ___ Albert, Yes, I think you got the idea right. I want to call my own C code using CTypes interface, then from within my C code call GSL C code, i.e. a C function calling another C function directly. I do *not* want to go back out through the Python interface. So you are right, I do not want to wrap GSL. It sounds like I can just add something like -lnameofGSLdylib (where I put in the real name of the GSL library after the -l) in my gcc command to make my shared lib. Is that right? Thanks for your help. -- Lou Pecora, my views are my own. - Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now.___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] C Extensions, CTypes and external code libraries
First, thanks to all who answered my questions about trying to use a large library with CTypes and my own shared library. The bottom line seems to be this: There is no way to incorporate code external to your own shared library. You have to either pull out the code you want from the static library's source (painful) or you must just include the whole library (huge!) and make it all one big shared library. Did I get that right? If so, it's a sad statement that makes shared libraries harder to write and works against the reuse of older established code bases. I am not criticizing CTypes. This appears to be the way static and shared libraries work, especially on Mac OS X, maybe elsewhere. I'd really like to be wrong about this and I will follow up on some of the suggested reading you all gave me. Thanks, again. -- Lou Pecora, my views are my own. Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CTypes: How to incorporate a library with shared library module?
Damian, Lots of good info there. Thanks very much. -- Lou --- Damian Eads [EMAIL PROTECTED] wrote: Dear Lou, You may want to try using distutils or setuputils, which makes compiling extensions much easier. It does the hard work of finding out which flags are needed to compile extensions on the host platform. There are many examples on the web on how to use distutils to build C extensions (http://docs.python.org/ext/building.html). [cut] -- Lou Pecora, my views are my own. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Median advice
On 12 Feb 2008, at 12:31, Matthew Brett wrote: def median(a, axis=0, out=None) (same signature as max, min etc) I would be slightly in favour of this option. Using the same signature would be convenient in code like def myfunc(myarray, somefunc): # do stuff ... x = somefunc(myarray, axis = 0, out = None) # do more stuff ... where somefunc could be median(), mean(), max(), min(), std() etc. I once wrote this kind of function to provide (small) image filtering. If the same signature is used, there is no need to special-case median(). I realise it's kind of a niche example, though. Just my 0.02 euros. Joris Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] asfarray() drops precision (float128-float64) - is it correct?
As for me, it yields lots of inconveniences (lots of my code should be rewritten, since I didn't know it before): from numpy import * a = array((1.0, 2.0), float128) b=asfarray(a) type(a[0]) #type 'numpy.float128' type(b[0]) #type 'numpy.float64' __version__ '1.0.5.dev4767' Shouldn't it be changed? (I.e. let's left 128). As for me I use asfarray() very often since I don't know does user provide arrays as numpy ndarray or matrix or Python list/tuple. D. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] f2py: sharing F90 module data between modules
On Tue, February 12, 2008 7:52 am, Garry Willgoose wrote: I have a suite of fortran modules that I want to wrap with f2py independently (so they appear to python as seperate imports) but where each module has access to another fortran module (which contains global data that is shared between the suite of fortran modules). I currently compile all the fortran modules, including the global data, in a single f2py statement so all the fortran code gets imported in a single statement The source of this issue boils down to http://bugs.python.org/issue521854 according to which makes your goal unachivable because of how Python loads shared libraries *by default*, see below. I guess the question is there any way that I can get fm3 to be shared between fm1 and fm2? The reasons for wanting to do this are because I'm developing a plug-in like architecture for environmental modelling where the user can develop new fortran modules (suitably f2py'ed) that can just be dropped into the module search path but still have access to the global data (subject to fortran module interfaces, etc). The link above also gives an hint how to resolve this issue. Try to use sys.setdlopenflags(...) before importing f2py generated extension modules and then reset the state using sys.setdlopenflags(0). See http://docs.python.org/lib/module-sys.html for more information how to find proper value for ... HTH, Pearu ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
