[And hopefully the last of pending discarded messages (there are others, but they are just too old). Again, please remember to send messages to the list from a *subscribed* mail address]
-------------------------------------------------------------------- De: "David Briant" <david.bri...@tradingroomassociates.com> A: pytables-users@lists.sourceforge.net Data: 26-10-10 12:28 I'm expecting several thousand tables in the end. Individual tables might be 20-40 MB of doubles. Should I be thinking of building my own table lookup class (I used a random seed to name each table, e.g. "_1", "_2" etc) in memory in the medium to long term as I get more tables? -----Original Message----- From: Francesc Alted [mailto:fal...@pytables.org] Sent: 26 October 2010 09:29 To: pytables-users@lists.sourceforge.net Subject: Re: [Pytables-users] how to speed up attributes access A Tuesday 26 October 2010 10:20:00 David Briant escrigué: > I'd like to have all the attributes in memory (because I assume they > are small and I need to search them a lot) but have the data in the > tables paged in and out of memory as I need it. > > Is that a realistic expectation? Yes, that may be realistic as the number of tables in your file is not high (160). My recommendation then is to set `NODE_CACHE_SLOTS` to a negative value (say -512). This way the cache will behave as a dictionary (i.e. you don't need to implement your own), which is the fastest operation mode for the node cache. -- Francesc Alted ------------------------------------------------------------------------------ Centralized Desktop Delivery: Dell and VMware Reference Architecture Simplifying enterprise desktop deployment and management using Dell EqualLogic storage and VMware View: A highly scalable, end-to-end client virtualization framework. Read more! http://p.sf.net/sfu/dell-eql-dev2dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
