Re: [Pytables-users] how to speed up attributes access

2010-11-12 Thread Francesc Alted
A Friday 12 November 2010 11:23:56 Francesc Alted escrigué: > [And hopefully the last of pending discarded messages (there are > others, but they are just too old). Again, please remember to send > messages to the list from a *subscribed* mail address] > >

Re: [Pytables-users] how to speed up attributes access

2010-11-12 Thread Francesc Alted
ot; etc) in memory in the medium to long term as I get more tables? -Original Message- From: Francesc Alted [mailto:fal...@pytables.org] Sent: 26 October 2010 09:29 To: pytables-users@lists.sourceforge.net Subject: Re: [Pytables-users] how to speed up attributes access A Tuesday 26 October

Re: [Pytables-users] how to speed up attributes access

2010-11-10 Thread david.briant
riginal Message- From: Francesc Alted [mailto:fal...@pytables.org] Sent: Tue 26-Oct-2010 09:29 To: pytables-users@lists.sourceforge.net Subject: Re: [Pytables-users] how to speed up attributes access A Tuesday 26 October 2010 10:20:00 David Briant escrigué: > I'd like to have all the

Re: [Pytables-users] how to speed up attributes access

2010-10-26 Thread Francesc Alted
A Tuesday 26 October 2010 10:20:00 David Briant escrigué: > I'd like to have all the attributes in memory (because I assume they > are small and I need to search them a lot) but have the data in the > tables paged in and out of memory as I need it. > > Is that a realistic expectation? Yes, that m

Re: [Pytables-users] how to speed up attributes access

2010-10-26 Thread David Briant
I'd like to have all the attributes in memory (because I assume they are small and I need to search them a lot) but have the data in the tables paged in and out of memory as I need it. Is that a realistic expectation? Sent from my iPhone On 26 Oct 2010, at 08:23, Francesc Alted wrote: > A

Re: [Pytables-users] how to speed up attributes access

2010-10-26 Thread Francesc Alted
A Monday 25 October 2010 22:16:30 David Briant escrigué: > I'm wondering if it might be better creating a representation of the > attributes in memory using dicts. The code I posted takes about a > second at the moment and I'll need to do lots of lookups by > attributes and I'll end up with thousan

Re: [Pytables-users] how to speed up attributes access

2010-10-25 Thread David Briant
I'm wondering if it might be better creating a representation of the attributes in memory using dicts. The code I posted takes about a second at the moment and I'll need to do lots of lookups by attributes and I'll end up with thousands of tables. I'll try those changes when I'm back at work

Re: [Pytables-users] how to speed up attributes access

2010-10-25 Thread David E. Sallis
David E. Sallis said the following on 10/25/2010 12:54 PM: > 3) Next, the so-called 'double under' methods (__contains__(), __getattr__()) > are generally intended for internal use only by their containing objects. A > better way would be to use the 'in' operator; to wit (note also the binding o

Re: [Pytables-users] how to speed up attributes access

2010-10-25 Thread David E. Sallis
david.bri...@ubs.com said the following on 10/25/2010 12:19 PM: > I currently have 160 or so tables with attributes on. I wish to find the > all the tables with a certain set of attributes. This code is running a > little slow: > > def getTables(tsdb, attributes): > answer = [] > for node

[Pytables-users] how to speed up attributes access

2010-10-25 Thread david.briant
Hi I currently have 160 or so tables with attributes on. I wish to find the all the tables with a certain set of attributes. This code is running a little slow: def getTables(tsdb, attributes): answer = [] for node in tsdb: if isinstance(node, pytables.Leaf): matches =