Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Simon Palmer
Pretty simple. I don't do any transformations. It is a euclidean distance matrix between n vectors in my data space, so I use it for lookup of minima and I recalculate portions of it during the processing of my algorithm. It is the single largest limitation of my code, both in terms of

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Simon Palmer
clustering yes, hierarchical no. On Wed, Jun 11, 2008 at 4:55 PM, Charles R Harris [EMAIL PROTECTED] wrote: On Wed, Jun 11, 2008 at 9:33 AM, Simon Palmer [EMAIL PROTECTED] wrote: Pretty simple. I don't do any transformations. It is a euclidean distance matrix between n vectors in my

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Peter Skomoroch
You could do something like this, where MyDIstanceFunc is written with weave and some intelligent caching. This only computes the upper diagonal elements in a sparse matrix: self.NumDatapoints = len(self.DataPoints) self.Distances = sparse.lil_matrix((self.NumDatapoints,self.NumDatapoints)) for

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Bruce Southey
Charles R Harris wrote: On Wed, Jun 11, 2008 at 9:33 AM, Simon Palmer [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Pretty simple. I don't do any transformations. It is a euclidean distance matrix between n vectors in my data space, so I use it for lookup of minima and I

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Keith Goodman
On Wed, Jun 11, 2008 at 9:47 AM, Simon Palmer [EMAIL PROTECTED] wrote: clustering yes, hierarchical no. I think scipy-cluster [1] only stores the upper triangle of the distance matrix. [1] http://code.google.com/p/scipy-cluster/ ___ Numpy-discussion

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Anne Archibald
2008/6/10 Simon Palmer [EMAIL PROTECTED]: Hi I have a problem which involves the creation of a large square matrix which is zero across its diagonal and symmetrical about the diagonal i.e. m[i,j] = m[j,i] and m[i,i] = 0. So, in fact, it is a large triangular matrix. I was wondering whether

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Simon Palmer
Actually that is very close to what I currently do, which is why I want to throw it away. The factor of two memory hit is not really the problem, it is the scaling O(N^2) which is the limitation I suspect I may end up using the 1-d array plus arithmetic as it is at least efficient for retrieval.

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Hoyt Koepke
The factor of two memory hit is not really the problem, it is the scaling O(N^2) which is the limitation Have you thought about using kd-trees or similar structures? If you are only concerned about the minimum distances between two points, that is the ideal data structure. It would reduce

Re: [Numpy-discussion] Large symmetrical matrix

2008-06-11 Thread Robert Kern
On Wed, Jun 11, 2008 at 15:14, Simon Palmer [EMAIL PROTECTED] wrote: Actually that is very close to what I currently do, which is why I want to throw it away. The factor of two memory hit is not really the problem, it is the scaling O(N^2) which is the limitation I suspect I may end up

Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Barry Wark
Could this also be added as a comment in the module docstring for test modules? It seems that anyone hacking on test code would be most likely to see it there...? Barry On Tue, Jun 10, 2008 at 2:57 AM, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/10 Alan McIntyre [EMAIL PROTECTED]: Is

Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Robert Kern
On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/10 Alan McIntyre [EMAIL PROTECTED]: Is the stuff Robert pointed out on a wiki page somewhere? It would be nice to have a Welcome noob NumPy developer, here's how to do NumPy-specific development things, page.

Re: [Numpy-discussion] array vs matrix

2008-06-11 Thread Ondrej Certik
On Sun, Jun 8, 2008 at 3:54 AM, Anne Archibald [EMAIL PROTECTED] wrote: 2008/6/7 Robert Kern [EMAIL PROTECTED]: On Sat, Jun 7, 2008 at 14:37, Ondrej Certik [EMAIL PROTECTED] wrote: Hi, what is the current plan with array and matrix with regard of calculating sin(A) ? I.e. elementwise vs

Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Alan McIntyre
On Wed, Jun 11, 2008 at 6:09 PM, Robert Kern [EMAIL PROTECTED] wrote: On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/10 Alan McIntyre [EMAIL PROTECTED]: Is the stuff Robert pointed out on a wiki page somewhere? It would be nice to have a Welcome noob NumPy

Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Stéfan van der Walt
2008/6/12 Robert Kern [EMAIL PROTECTED]: On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/10 Alan McIntyre [EMAIL PROTECTED]: Is the stuff Robert pointed out on a wiki page somewhere? It would be nice to have a Welcome noob NumPy developer, here's how to do

Re: [Numpy-discussion] Switching to nose test framework (was: NumpyTest problem)

2008-06-11 Thread Robert Kern
On Wed, Jun 11, 2008 at 19:51, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/12 Robert Kern [EMAIL PROTECTED]: On Tue, Jun 10, 2008 at 04:57, Stéfan van der Walt [EMAIL PROTECTED] wrote: 2008/6/10 Alan McIntyre [EMAIL PROTECTED]: Is the stuff Robert pointed out on a wiki page somewhere?

[Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread David Cournapeau
Hi, Short story: it would make my life simpler to have two mandatory scons scripts per package instead of only one. Long story: for some reasons linked to scons' notion of build directory, having two scons scripts files per package (instead of one) would simplify quite a bit some parts of

Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread Robert Kern
On Wed, Jun 11, 2008 at 23:35, David Cournapeau [EMAIL PROTECTED] wrote: Hi, Short story: it would make my life simpler to have two mandatory scons scripts per package instead of only one. Long story: for some reasons linked to scons' notion of build directory, having two scons scripts

Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread David Cournapeau
Robert Kern wrote: Can you give me a longer story? What are the three lines? Why are they necessary in a separate, boilerplate file? Sure: scons builds everything from the sources in the source directory, and build everything in the build directory. By default, both directories are the same

Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread Robert Kern
On Thu, Jun 12, 2008 at 00:08, David Cournapeau [EMAIL PROTECTED] wrote: Robert Kern wrote: Can you give me a longer story? What are the three lines? Why are they necessary in a separate, boilerplate file? Sure: scons builds everything from the sources in the source directory, and build

Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread David Cournapeau
Robert Kern wrote: Works for me. Do you mean having two files/packages is ok ? cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] [numscons] Would having two mandatory scons script per package be acceptable ?

2008-06-11 Thread Robert Kern
On Thu, Jun 12, 2008 at 00:28, David Cournapeau [EMAIL PROTECTED] wrote: Robert Kern wrote: Works for me. Do you mean having two files/packages is ok ? Yes. You had me at The recommended way to do it in scons -- Robert Kern I have come to believe that the whole world is an enigma, a

[Numpy-discussion] [numscons] numscons build for numpy will be broken for a few hours

2008-06-11 Thread David Cournapeau
Hi, Just to mention that building numpy and scipy with numscons will be broken for a few hours, the time to change some organizations. Sorry for the trouble, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org