Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Alvaro Tejero Cantero
Thank you Josh, that is representative enough. In my system the speedup of structured arrays is ~30x. A copy of the whole array is still ~6x faster. -á. On Thu, Jun 28, 2012 at 10:13 PM, Josh Ayers wrote: > import time > import numpy as np > > dtype = np.format_parser(['i4', 'i4'], [], []) > N

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
That is reason enough for me really. If someone really wants a recarray, they could always convert an ndarray to this. I think it is still worth asking the numpy list what the status is... Be Well Anthony On Thu, Jun 28, 2012 at 4:13 PM, Josh Ayers wrote: > There is a big difference in speed

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Josh Ayers
There is a big difference in speed when iterating over the rows. Possibly that was the reason structured arrays were chosen? The issue is mentioned here: http://www.scipy.org/Cookbook/Recarray In a simple test, I get a difference of about 15x, so it is significant. Iterating over a recarray with

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
On Thu, Jun 28, 2012 at 3:23 PM, Francesc Alted wrote: > Yes, I think it would make more sense to return a recarray too. However, > I remember many time ago (3, 4 years?) that NumPy developers were > recommending using structured arrays instead of recarrays. I don't > remember exactly the argu

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Francesc Alted
Yes, I think it would make more sense to return a recarray too. However, I remember many time ago (3, 4 years?) that NumPy developers were recommending using structured arrays instead of recarrays. I don't remember exactly the arguments, but I think that was the reason why the structured arra

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
Hmmm Ok. Maybe there needs to be a recarray flavor. I kind of like just returning a normal ndarray, though I see your argument for returning a recarray. Maybe some of the other devs can jump in here with an opinion. Be Well Anthony On Thu, Jun 28, 2012 at 12:37 PM, Alvaro Tejero Cantero wrote:

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Alvaro Tejero Cantero
I just tested: passing an object of type numpy.core.records.recarray to the constructor of createTable and then reading back it into memory via slicing (h5f.root.myobj[:] ) returns to me a numpy.ndarray. Best, -á. On Thu, Jun 28, 2012 at 5:30 PM, Anthony Scopatz wrote: > Hi Alvaro, > > I think

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
Hi Alvaro, I think if you save the table as a record array, it should return you a record array. Or does it return a structured array? Have you tried this? Be Well Anthony On Thu, Jun 28, 2012 at 11:22 AM, Alvaro Tejero Cantero wrote: > Hi, > > I've noticed that tables are loaded in memory as