Hi
Thx. It's working faster.
I'm opening a table like this:
import tables as pytables
tsdb = pytables.openFile("c:\\data\\tsdb.hd5", mode="a", NODE_CACHE_SLOTS=-1)
I then list all the attribute names thus:
def getAllAttributeNames(tsdb):
answer = set()
for node in tsdb:
if isinstance(node, pytables.Leaf):
answer.update(node._v_attrs._f_list())
return answer
It takes a little longer than I think it would take if I replicated the
behaviour using dictionaries but it's fast enough at the moment.
Regards,
David
-----Original Message-----
From: Francesc Alted [mailto:fal...@pytables.org]
Sent: Tue 26-Oct-2010 09:29
To: pytables-users@lists.sourceforge.net
Subject: Re: [Pytables-users] how to speed up attributes access
A Tuesday 26 October 2010 10:20:00 David Briant escrigué:
> I'd like to have all the attributes in memory (because I assume they
> are small and I need to search them a lot) but have the data in the
> tables paged in and out of memory as I need it.
>
> Is that a realistic expectation?
Yes, that may be realistic as the number of tables in your file is not
high (160). My recommendation then is to set `NODE_CACHE_SLOTS` to a
negative value (say -512). This way the cache will behave as a
dictionary (i.e. you don't need to implement your own), which is the
fastest operation mode for the node cache.
--
Francesc Alted
------------------------------------------------------------------------------
Nokia and AT&T present the 2010 Calling All Innovators-North America contest
Create new apps & games for the Nokia N8 for consumers in U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
Visit our website at http://www.ubs.com
This message contains confidential information and is intended only
for the individual named. If you are not the named addressee you
should not disseminate, distribute or copy this e-mail. Please
notify the sender immediately by e-mail if you have received this
e-mail by mistake and delete this e-mail from your system.
E-mails are not encrypted and cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The sender
therefore does not accept liability for any errors or omissions in the
contents of this message which arise as a result of e-mail transmission.
If verification is required please request a hard-copy version. This
message is provided for informational purposes and should not be
construed as a solicitation or offer to buy or sell any securities
or related financial instruments.
UBS Limited is a company limited by shares incorporated in the United
Kingdom registered in England and Wales with number 2035362.
Registered office: 1 Finsbury Avenue, London EC2M 2PP. UBS Limited
is authorised and regulated by the Financial Services Authority.
UBS AG is a public company incorporated with limited liability in
Switzerland domiciled in the Canton of Basel-City and the Canton of
Zurich respectively registered at the Commercial Registry offices in
those Cantons with Identification No: CH-270.3.004.646-4 and having
respective head offices at Aeschenvorstadt 1, 4051 Basel and
Bahnhofstrasse 45, 8001 Zurich, Switzerland. Registered in the
United Kingdom as a foreign company with No: FC021146 and having a
UK Establishment registered at Companies House, Cardiff, with No:
BR 004507. The principal office of UK Establishment: 1 Finsbury Avenue,
London EC2M 2PP. In the United Kingdom, UBS AG is authorised and
regulated by the Financial Services Authority.
UBS reserves the right to retain all messages. Messages are protected
and accessed only in legally justified cases.
------------------------------------------------------------------------------
The Next 800 Companies to Lead America's Growth: New Video Whitepaper
David G. Thomson, author of the best-selling book "Blueprint to a
Billion" shares his insights and actions to help propel your
business during the next growth cycle. Listen Now!
http://p.sf.net/sfu/SAP-dev2dev
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
