Re: [Pytables-users] Simple fast array I/O with Pytables

2012-01-02 Thread Gael Varoquaux
On Mon, Jan 02, 2012 at 07:18:24PM +0100, Gael Varoquaux wrote: > > BTW, another cleaner, faster way to empty the OS filesystem cache is this: > > sudo echo 3 | sudo tee /proc/sys/vm/drop_caches > > [ from http://ubuntuforums.org/showthread.php?t=589975 ] > That's useful! And it works much bett

Re: [Pytables-users] Simple fast array I/O with Pytables

2012-01-02 Thread Gael Varoquaux
On Mon, Jan 02, 2012 at 06:44:54PM +0100, Francesc Alted wrote: > Perhaps you may get a bit more performance if you use the > `[read,write]_vl_blosc2_hdf` functions that I have sent in my earlier > post, but that adds the python-blosc dependency (available at > http://pypi.python.org/pypi/blosc/1.0

Re: [Pytables-users] Simple fast array I/O with Pytables

2012-01-02 Thread Francesc Alted
2012/1/2 Gael Varoquaux : > On Fri, Dec 30, 2011 at 02:16:51PM -0800, Dav Clark wrote: >> I'll add that it would be cool to see what numbers you come up with >> (maybe with some loose specs on the machine CPU and disk you used). > > Yes, I have been working on this for a little while, and it is act

Re: [Pytables-users] Simple fast array I/O with Pytables

2012-01-02 Thread Gael Varoquaux
On Fri, Dec 30, 2011 at 02:16:51PM -0800, Dav Clark wrote: > I'll add that it would be cool to see what numbers you come up with > (maybe with some loose specs on the machine CPU and disk you used). Yes, I have been working on this for a little while, and it is actually full of surprises. I have p