El dc 11 de 10 del 2006 a les 15:24 -0500, en/na Jun Li va escriure:
>
> Thanks for your response!
>
> I tried your suggestion. tbl_T.cols.tmax[index1:index2].
> The performance is much worse (180 s vs 60 s).
> That makes sense to me. Since my program will access the table more
> than 9000 times instead of once using different indexes for different
> small pieces of data in the table.
> my way: src_rows = tbl_T.read(start=0, stop=tbl_T.nrows)
> txd = src_rows.field('tmax')[index1:index2]
> Only access the table once, for the following 9000 times data
> retrieval, the program only need to using different indexes to access
> the numarray in the memory, that makes it faster.
> My logic correct?
Yes, I wasn't aware that you wanted to read the table so many times. So,
yes, *if* the table fits well in your system memory, your approach is
good.
>
> However, when I run the program in your way, the performance looks
> consistent( always around 180s) .That makes me wonder the memory plays
> the role to make the program run in my way inconsistent.
> Any ideas to solve the problem?
I don't know for sure. Perhaps the table is very big and your system is
starting to swap. Depending on the part of the recarray that you are
accessing, it can be in physical memory (faster access) or swapped
(slower access). This might explain why you are observing these
differences in performance.
Try to compute the size of your table in order to check whether your
system might be swapping or not. The next formula will tell you the
extent of the recarray in MB:
(tbl_T.nrows * tbl_T.rowsize) / (1024*1024.)
If the resulting figure is bigger (or near to) your available system
memory, then your system is swapping and this can explain the difference
in performance that you are observing.
HTH,
--
>0,0< Francesc Altet http://www.carabos.com/
V V Cárabos Coop. V. Enjoy Data
"-"
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
