[Numpy-discussion] numpy on windows x64 with mingw: it (almost) works
Hi, Just a few words to mention that I've finally managed to build numpy with the mingw-w64 project (port of mingw to AMD 64 bits MS OS), and it almost run OK. By almost, I mean that numpy.test() finishes without crash, assuming a few unit tests are skipped (some long double problems). Not all unit tests pass, but almost all of them are easy to fix problems in numpy (except for the long double problem). The drawback is that you can't do that just by using the mingw-w64 binaries, you have to build your own toolchain because of some bugs/missing features in mingw-w64. I've put the gory details there: http://scipy.org/scipy/numpy/wiki/MicrosoftToolchainSupport Hopefully, this should make it easier to add fortran support with gfortran, opening the possibility to have both numpy and scipy buildable on windows x64 with free compilers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy on windows x64 with mingw: it (almost) works
On Sun, Dec 21, 2008 at 05:13:11PM +0900, David Cournapeau wrote: Just a few words to mention that I've finally managed to build numpy with the mingw-w64 project I know it was a tough task. Thanks a lot for doing this. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] numpy on windows x64 with mingw: it (almost) works
On Sun, Dec 21, 2008 at 12:13 AM, David Cournapeau da...@ar.media.kyoto-u.ac.jp wrote: Just a few words to mention that I've finally managed to build numpy with the mingw-w64 project (port of mingw to AMD 64 bits MS OS), and it almost run OK. Thanks for working on this. Jarrod ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
On Sat, Dec 20, 2008 at 3:02 PM, Pauli Virtanen p...@iki.fi wrote: Sat, 20 Dec 2008 20:15:43 +0900, David Cournapeau wrote: On Sat, Dec 20, 2008 at 7:43 PM, Ondrej Certik ond...@certik.cz wrote: Just to make it clear -- I think the docs should not be generated in the tarball -- only the sources should be there. I agree this makes more sense for you, as a packager, but I am not sure it makes much sense to put the doc sources in the tarball for users (Building numpy should only require python + a C compiler; building the doc is more difficult -you need at least sphinx and all its dependencies). For audiolab, I put the generated doc, thinking if people want to mess with the doc, they are knowledgeable enough to deal with svn - but I did not think about the packagers :) I am not sure what's the best solution: maybe put both in the (released) source tarball ? I'd say that we put the source for the documentation to the documentation tarball, and distribute the built HTML+whatever documentation in a separate package. Why not to just include the *sources* together with numpy, and possibly include html+whatever in a separate documentation package? That way everybody is happy. Ondrej ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
Ondrej Certik wrote: Why not to just include the *sources* together with numpy, and possibly include html+whatever in a separate documentation package? I don't think having separate built doc and built package is a good idea. It is confusing for the user, and I am afraid we won't alway keep everything in sync. We don't need to ship all the doc of course, but at least the pdf (or any other format, the point is to have at least one; it could be different on different platforms for the binaries for example). Specially for new comers, having everything in one place is better IMHO. I agree we should also put the doc sources together with the source tarball: that seems to be the common practice for almost every open source package out there. The only drawback is the tarball size, but since we are still talking about a couple of MB max, I don't think it is very relevant. IOW: - ship the doc sources + one built format with the released source distribution. It should not be built with sdist, but with another mean (so that one can easily generate a source distribution). - ship the built doc with every binary installer. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
Sun, 21 Dec 2008 13:05:57 +0100, Ondrej Certik wrote: On Sat, Dec 20, 2008 at 3:02 PM, Pauli Virtanen p...@iki.fi wrote: Sat, 20 Dec 2008 20:15:43 +0900, David Cournapeau wrote: On Sat, Dec 20, 2008 at 7:43 PM, Ondrej Certik ond...@certik.cz wrote: Just to make it clear -- I think the docs should not be generated in the tarball -- only the sources should be there. I agree this makes more sense for you, as a packager, but I am not sure it makes much sense to put the doc sources in the tarball for users (Building numpy should only require python + a C compiler; building the doc is more difficult -you need at least sphinx and all its dependencies). For audiolab, I put the generated doc, thinking if people want to mess with the doc, they are knowledgeable enough to deal with svn - but I did not think about the packagers :) I am not sure what's the best solution: maybe put both in the (released) source tarball ? I'd say that we put the source for the documentation to the documentation tarball, and distribute the built HTML+whatever documentation in a separate package. Why not to just include the *sources* together with numpy, and possibly include html+whatever in a separate documentation package? That's what I tried to say, but mistyped source as documentation. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
On Sun, Dec 21, 2008 at 1:49 PM, Pauli Virtanen p...@iki.fi wrote: Sun, 21 Dec 2008 13:05:57 +0100, Ondrej Certik wrote: On Sat, Dec 20, 2008 at 3:02 PM, Pauli Virtanen p...@iki.fi wrote: Sat, 20 Dec 2008 20:15:43 +0900, David Cournapeau wrote: On Sat, Dec 20, 2008 at 7:43 PM, Ondrej Certik ond...@certik.cz wrote: Just to make it clear -- I think the docs should not be generated in the tarball -- only the sources should be there. I agree this makes more sense for you, as a packager, but I am not sure it makes much sense to put the doc sources in the tarball for users (Building numpy should only require python + a C compiler; building the doc is more difficult -you need at least sphinx and all its dependencies). For audiolab, I put the generated doc, thinking if people want to mess with the doc, they are knowledgeable enough to deal with svn - but I did not think about the packagers :) I am not sure what's the best solution: maybe put both in the (released) source tarball ? I'd say that we put the source for the documentation to the documentation tarball, and distribute the built HTML+whatever documentation in a separate package. Why not to just include the *sources* together with numpy, and possibly include html+whatever in a separate documentation package? That's what I tried to say, but mistyped source as documentation. Ok, so we all seem to agree that having (at least) the source of docs together with the main numpy tarball is a good thing. I'll try to have a look at this. Ondrej ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
Pauli Virtanen wrote: Sun, 21 Dec 2008 13:05:57 +0100, Ondrej Certik wrote: On Sat, Dec 20, 2008 at 3:02 PM, Pauli Virtanen p...@iki.fi wrote: Sat, 20 Dec 2008 20:15:43 +0900, David Cournapeau wrote: On Sat, Dec 20, 2008 at 7:43 PM, Ondrej Certik ond...@certik.cz wrote: Just to make it clear -- I think the docs should not be generated in the tarball -- only the sources should be there. I agree this makes more sense for you, as a packager, but I am not sure it makes much sense to put the doc sources in the tarball for users (Building numpy should only require python + a C compiler; building the doc is more difficult -you need at least sphinx and all its dependencies). For audiolab, I put the generated doc, thinking if people want to mess with the doc, they are knowledgeable enough to deal with svn - but I did not think about the packagers :) I am not sure what's the best solution: maybe put both in the (released) source tarball ? I'd say that we put the source for the documentation to the documentation tarball, and distribute the built HTML+whatever documentation in a separate package. Why not to just include the *sources* together with numpy, and possibly include html+whatever in a separate documentation package? That's what I tried to say, but mistyped source as documentation. Pauli, Is everything under trunk/doc necessary to build the doc ? Or only a subset of it ? David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
Sun, 21 Dec 2008 22:48:41 +0900, David Cournapeau wrote: [clip] Is everything under trunk/doc necessary to build the doc ? Or only a subset of it ? Only a subset: Makefile, postprocess.py, release/*, source/*, and sphinxext/*. The rest is some older material (eg. numpybook/*), stuff targeted at developers (eg. ufuncs.txt), and miscellaneous stuff for users (eg. cython/, swig/) that should eventually be added as a part of the main documentation. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
On Sun, Dec 21, 2008 at 11:42 PM, Pauli Virtanen p...@iki.fi wrote: Sun, 21 Dec 2008 22:48:41 +0900, David Cournapeau wrote: [clip] Is everything under trunk/doc necessary to build the doc ? Or only a subset of it ? Only a subset: Makefile, postprocess.py, release/*, source/*, and sphinxext/*. Ok, it should now be included in the sdist-generated tarball. I could not generate the doc from it, but I have the same problem when trying directly from the trunk, not sure what the problem is (make html hangs, no cpu consumption). Also, the increase in size is much smaller than I said previously: it looks like most of the stuff which took space in the tarball was the other directories: we go from 1.5 to 1.9 Mb, instead of 3 Mb if we added the whole doc directory. David The rest is some older material (eg. numpybook/*), stuff targeted at developers (eg. ufuncs.txt), and miscellaneous stuff for users (eg. cython/, swig/) that should eventually be added as a part of the main documentation. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
Mon, 22 Dec 2008 00:30:55 +0900, David Cournapeau wrote: On Sun, Dec 21, 2008 at 11:42 PM, Pauli Virtanen p...@iki.fi wrote: Sun, 21 Dec 2008 22:48:41 +0900, David Cournapeau wrote: [clip] Is everything under trunk/doc necessary to build the doc ? Or only a subset of it ? Only a subset: Makefile, postprocess.py, release/*, source/*, and sphinxext/*. Ok, it should now be included in the sdist-generated tarball. I could not generate the doc from it, but I have the same problem when trying directly from the trunk, not sure what the problem is (make html hangs, no cpu consumption). What platform, what does it output? What does 'make -n' say? -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] missing doc dir in the official tarball
On Mon, Dec 22, 2008 at 3:10 AM, Pauli Virtanen p...@iki.fi wrote: Mon, 22 Dec 2008 00:30:55 +0900, David Cournapeau wrote: On Sun, Dec 21, 2008 at 11:42 PM, Pauli Virtanen p...@iki.fi wrote: Sun, 21 Dec 2008 22:48:41 +0900, David Cournapeau wrote: [clip] Is everything under trunk/doc necessary to build the doc ? Or only a subset of it ? Only a subset: Makefile, postprocess.py, release/*, source/*, and sphinxext/*. Ok, it should now be included in the sdist-generated tarball. I could not generate the doc from it, but I have the same problem when trying directly from the trunk, not sure what the problem is (make html hangs, no cpu consumption). What platform, what does it output? What does 'make -n' say? I only made this comment to imply that I did not test whether I included everything needed. I don't think my problem is really relevant, it may just be a configuration problem. David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
josef.p...@gmail.com wrote: I was looking for a function that sorts a 2-dimensional array by rows. That's what I came up with, is there a more direct way? a array([[1, 2], [0, 0], [1, 0], [0, 2], [2, 1], [1, 0], [1, 0], [0, 0], [1, 0], [2, 2]]) a[np.lexsort(np.fliplr(a).T)] array([[0, 0], [0, 0], [0, 2], [1, 0], [1, 0], [1, 0], [1, 0], [1, 2], [2, 1], [2, 2]]) Note: I needed to flip and transpose, using axis didn't work a.shape (10, 2) np.lexsort(a,axis=1) Traceback (most recent call last): File pyshell#76, line 1, in module np.lexsort(a,axis=1) ValueError: axis(=1) out of bounds Specifying individual columns in argument also works, but it's a pain if I don't know how many columns there are: a[np.lexsort((a[:,1],a[:,0]))] array([[0, 0], [0, 0], [0, 2], [1, 0], [1, 0], [1, 0], [1, 0], [1, 2], [2, 1], [2, 2]]) A helper function sortrows would be helpful, I don't know what would be the higher dimensional equivalent. Or did I miss a function that I didn't find in the help file? I may miss something obvious, but why are you using lexsort at all ? At leat, the first example is easily achieved with sort(x, axis=0) - but maybe you have more complicated examples in mind where you need actual lexical sort: David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
On Sun, Dec 21, 2008 at 20:53, David Cournapeau da...@ar.media.kyoto-u.ac.jp wrote: josef.p...@gmail.com wrote: I was looking for a function that sorts a 2-dimensional array by rows. That's what I came up with, is there a more direct way? a array([[1, 2], [0, 0], [1, 0], [0, 2], [2, 1], [1, 0], [1, 0], [0, 0], [1, 0], [2, 2]]) a[np.lexsort(np.fliplr(a).T)] array([[0, 0], [0, 0], [0, 2], [1, 0], [1, 0], [1, 0], [1, 0], [1, 2], [2, 1], [2, 2]]) Note: I needed to flip and transpose, using axis didn't work a.shape (10, 2) np.lexsort(a,axis=1) Traceback (most recent call last): File pyshell#76, line 1, in module np.lexsort(a,axis=1) ValueError: axis(=1) out of bounds Specifying individual columns in argument also works, but it's a pain if I don't know how many columns there are: a[np.lexsort((a[:,1],a[:,0]))] array([[0, 0], [0, 0], [0, 2], [1, 0], [1, 0], [1, 0], [1, 0], [1, 2], [2, 1], [2, 2]]) A helper function sortrows would be helpful, I don't know what would be the higher dimensional equivalent. Or did I miss a function that I didn't find in the help file? I may miss something obvious, but why are you using lexsort at all ? At leat, the first example is easily achieved with sort(x, axis=0) No, it isn't. In [4]: sort(a, axis=0) Out[4]: array([[0, 0], [0, 0], [0, 0], [1, 0], [1, 0], [1, 0], [1, 1], [1, 2], [2, 2], [2, 2]]) Compare to his desired result: array([[0, 0], [0, 0], [0, 2], [1, 0], [1, 0], [1, 0], [1, 0], [1, 2], [2, 1], [2, 2]]) -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
I may miss something obvious, but why are you using lexsort at all ? At leat, the first example is easily achieved with sort(x, axis=0) - but maybe you have more complicated examples in mind where you need actual lexical sort: David From the examples that I tried out np.sort, sorts each column separately (with axis = 0). If the elements of a row is supposed to stay together, then np.sort doesn't work arr array([[ 1, 14], [ 4, 12], [ 3, 11], [ 2, 14]]) np.sort(arr,axis=0) array([[ 1, 11], [ 2, 12], [ 3, 14], [ 4, 14]]) Josef ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
josef.p...@gmail.com wrote: I may miss something obvious, but why are you using lexsort at all ? At leat, the first example is easily achieved with sort(x, axis=0) - but maybe you have more complicated examples in mind where you need actual lexical sort: David From the examples that I tried out np.sort, sorts each column separately (with axis = 0). If the elements of a row is supposed to stay together, then np.sort doesn't work. You're right, as Robert just mentioned, I totally missed the point of your example... David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
On Dec 21, 2008, at 10:19 PM, josef.p...@gmail.com wrote:
From the examples that I tried out np.sort, sorts each column
separately (with axis = 0). If the elements of a row is supposed to
stay together, then np.sort doesn't work
Well, if the elements are supposed to stay together, why wouldn't you
tie them first, sort, and then untie them ?
np.sort(a.view([('',int),('',int)]),0).view(int)
The first view transforms your 2D array into a 1D array of tuples, the
second one retransforms the 1D array to 2D.
Not sure it's better than your lexsort, haven't timed it.
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
On Sun, Dec 21, 2008 at 11:10 PM, Pierre GM pgmdevl...@gmail.com wrote:
On Dec 21, 2008, at 10:19 PM, josef.p...@gmail.com wrote:
From the examples that I tried out np.sort, sorts each column
separately (with axis = 0). If the elements of a row is supposed to
stay together, then np.sort doesn't work
Well, if the elements are supposed to stay together, why wouldn't you
tie them first, sort, and then untie them ?
np.sort(a.view([('',int),('',int)]),0).view(int)
The first view transforms your 2D array into a 1D array of tuples, the
second one retransforms the 1D array to 2D.
Not sure it's better than your lexsort, haven't timed it.
That's very helpful, not so much about the sort but it's a good
example to move back and forth between structured and regular arrays.
My help search for this was not successful enough to figure this out
by myself. Several functions require structured arrays but I didn't
know how to get them without specifying everything by hand. And when I
have a structured array, I didn't know how to call var or mean on
them.
Your suggestion also works with automatic adjustment for number of columns.
np.sort(a.view([('','i4')]*a.shape[1]),0).view(int)
Thanks,
Josef
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] is there a sortrows
On Sun, Dec 21, 2008 at 11:37 PM, josef.p...@gmail.com wrote:
On Sun, Dec 21, 2008 at 11:10 PM, Pierre GM pgmdevl...@gmail.com wrote:
On Dec 21, 2008, at 10:19 PM, josef.p...@gmail.com wrote:
From the examples that I tried out np.sort, sorts each column
separately (with axis = 0). If the elements of a row is supposed to
stay together, then np.sort doesn't work
Well, if the elements are supposed to stay together, why wouldn't you
tie them first, sort, and then untie them ?
np.sort(a.view([('',int),('',int)]),0).view(int)
The first view transforms your 2D array into a 1D array of tuples, the
second one retransforms the 1D array to 2D.
Not sure it's better than your lexsort, haven't timed it.
That's very helpful, not so much about the sort but it's a good
example to move back and forth between structured and regular arrays.
My help search for this was not successful enough to figure this out
by myself. Several functions require structured arrays but I didn't
know how to get them without specifying everything by hand. And when I
have a structured array, I didn't know how to call var or mean on
them.
Your suggestion also works with automatic adjustment for number of columns.
np.sort(a.view([('','i4')]*a.shape[1]),0).view(int)
Thanks,
Josef
Version with fully automatic conversion, I don't even have to know the dtype
np.sort(a.view([('',a.dtype)]*a.shape[1]),0).view(a.dtype)
(this is for future Google searches)
Josef
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
