A Thursday 11 March 2010 10:12:04 Francesc Alted escrigué: > [This is a message that was sent from an unsubscribed address. David, > please subscribe if you want you messages injected to the list.] > > --------------------------------------------------------------------------- > ---- De: David Carmean <dlc-...@halibut.com> > A: "P.U." <pytables-users@lists.sourceforge.net> > Data: Diumenge 03:28:34 > > Greetings, > > This may seem counter to the purpose of pytables/hdf5, but I have some use > cases where I'd like to just use RAM for one of my tables, with cStringIO. > Does the underlying HDF5 library prevent using filehandle instead of a > filename string? > > > [1]: This is mostly for unit testing but there are some use cases where > I could gain a lot of speed in a pre-processing step by not using the > filesystem for a temporary table that I build. For large datasets, > it's worth saving this intermediate table so it still pays to use > a pytables Table object.
Why do you think you will be getting a big speed-up by using pure memory? When datasets are small enough, the OS filesystem cache normally does an excellent job by caching I/O requests to/from disk in memory. In general, bottlenecks are in other places (unless you have a clear case of the contrary). My advice is that you should not try to optimize things too much by yourself and trust the OS facilities better (most specially if you are just doing unit testing). Cheers, -- Francesc Alted ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
