[Numpy-discussion] recarray slow?

2010-07-21 Thread wheres pythonmonks
I have an recarray -- the first column is date. I have the following function to compute the number of unique dates in my data set: def byName(): return(len(list(set(d['Date'])) )) Question: is the string 'Date' looked up at each iteration? If so, this is dumb, but explains my horrible

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread Robert Kern
On Wed, Jul 21, 2010 at 15:12, wheres pythonmonks wherespythonmo...@gmail.com wrote: I have an recarray -- the first column is date. I have the following function to compute the number of unique dates in my data set: def byName(): return(len(list(set(d['Date'])) )) Question:  is the

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread Pauli Virtanen
Wed, 21 Jul 2010 15:12:14 -0400, wheres pythonmonks wrote: I have an recarray -- the first column is date. I have the following function to compute the number of unique dates in my data set: def byName(): return(len(list(set(d['Date'])) )) What this code does is: 1. d['Date']

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread wheres pythonmonks
Thank you very much better crack open a numpy reference manual instead of relying on my python intuition. On Wed, Jul 21, 2010 at 3:44 PM, Pauli Virtanen p...@iki.fi wrote: Wed, 21 Jul 2010 15:12:14 -0400, wheres pythonmonks wrote: I have an recarray -- the first column is date. I have

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread wheres pythonmonks
However: is there an automatic way to convert a named index to a position? What about looping over tuples of my recarray: for t in d: date = t['Date'] I guess that the above does have to lookup 'Date' each time. But the following does not need the hash lookup for each tuple: for t

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread Pierre GM
On Jul 21, 2010, at 4:22 PM, wheres pythonmonks wrote: However: is there an automatic way to convert a named index to a position? What about looping over tuples of my recarray: for t in d: date = t['Date'] Why don't you use zip ? for (date, t) in (d['Date'], d) That way,

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread wheres pythonmonks
What about: idx_by_name = dict(enumerate(d.dtype.names)) Then I can look up the index of the columns I want before the loop, and then access by the index during the loop. - W On Wed, Jul 21, 2010 at 4:29 PM, Pierre GM pgmdevl...@gmail.com wrote: On Jul 21, 2010, at 4:22 PM, wheres

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread Pierre GM
On Jul 21, 2010, at 4:35 PM, wheres pythonmonks wrote: What about: idx_by_name = dict(enumerate(d.dtype.names)) Then I can look up the index of the columns I want before the loop, and then access by the index during the loop. Sure. Why don't you try both approaches, time them and

Re: [Numpy-discussion] recarray slow?

2010-07-21 Thread wheres pythonmonks
My code had a bug: idx_by_name = dict((n,i) for i,n in enumerate(d.dtype.names)) On Wed, Jul 21, 2010 at 4:49 PM, Pauli Virtanen p...@iki.fi wrote: Wed, 21 Jul 2010 16:22:37 -0400, wheres pythonmonks wrote: However: is there an automatic way to convert a named index to a position? It's