Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Alvaro Tejero Cantero
Thank you Josh, that is representative enough. In my system the speedup of structured arrays is ~30x. A copy of the whole array is still ~6x faster. -á. On Thu, Jun 28, 2012 at 10:13 PM, Josh Ayers wrote: > import time > import numpy as np > > dtype = np.format_parser(['i4', 'i4'], [], []) > N

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
That is reason enough for me really. If someone really wants a recarray, they could always convert an ndarray to this. I think it is still worth asking the numpy list what the status is... Be Well Anthony On Thu, Jun 28, 2012 at 4:13 PM, Josh Ayers wrote: > There is a big difference in speed

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Josh Ayers
There is a big difference in speed when iterating over the rows. Possibly that was the reason structured arrays were chosen? The issue is mentioned here: http://www.scipy.org/Cookbook/Recarray In a simple test, I get a difference of about 15x, so it is significant. Iterating over a recarray with

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
On Thu, Jun 28, 2012 at 3:23 PM, Francesc Alted wrote: > Yes, I think it would make more sense to return a recarray too. However, > I remember many time ago (3, 4 years?) that NumPy developers were > recommending using structured arrays instead of recarrays. I don't > remember exactly the argu

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Francesc Alted
Yes, I think it would make more sense to return a recarray too. However, I remember many time ago (3, 4 years?) that NumPy developers were recommending using structured arrays instead of recarrays. I don't remember exactly the arguments, but I think that was the reason why the structured arra

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
Hmmm Ok. Maybe there needs to be a recarray flavor. I kind of like just returning a normal ndarray, though I see your argument for returning a recarray. Maybe some of the other devs can jump in here with an opinion. Be Well Anthony On Thu, Jun 28, 2012 at 12:37 PM, Alvaro Tejero Cantero wrote:

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Alvaro Tejero Cantero
I just tested: passing an object of type numpy.core.records.recarray to the constructor of createTable and then reading back it into memory via slicing (h5f.root.myobj[:] ) returns to me a numpy.ndarray. Best, -á. On Thu, Jun 28, 2012 at 5:30 PM, Anthony Scopatz wrote: > Hi Alvaro, > > I think

Re: [Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Anthony Scopatz
Hi Alvaro, I think if you save the table as a record array, it should return you a record array. Or does it return a structured array? Have you tried this? Be Well Anthony On Thu, Jun 28, 2012 at 11:22 AM, Alvaro Tejero Cantero wrote: > Hi, > > I've noticed that tables are loaded in memory as

[Pytables-users] Use of recarrays as representation for Tables in memory

2012-06-28 Thread Alvaro Tejero Cantero
Hi, I've noticed that tables are loaded in memory as structured arrays. It seems that returning recarrays by default would be much in the spirit of the natural naming preferences of PyTables. Is there a reason not to do so? Cheers, Álvaro. -