Re: [Pytables-users] Pytables-users Digest, Vol 86, Issue 8

Both loadtxt and genfromtxt read the entire data into memory which is not desirable. Is there a way to achieve streaming writes? Thanks, Pushkar On Wed, Jul 17, 2013 at 7:04 PM, Pushkar Raj Pande topgun...@gmail.comwrote: Thanks Antonio and Anthony. I will give this a try. -Pushkar On

Re: [Pytables-users] Pytables-users Digest, Vol 86, Issue 8

On 18.07.2013 08:45, Pushkar Raj Pande wrote: Both loadtxt and genfromtxt read the entire data into memory which is not desirable. Is there a way to achieve streaming writes? Thanks, Pushkar On Wed, Jul 17, 2013 at 7:04 PM, Pushkar Raj Pande topgun...@gmail.com

Re: [Pytables-users] Pytables-users Digest, Vol 86, Issue 8

Hi Pushkar, Il 18/07/2013 08:45, Pushkar Raj Pande ha scritto: Both loadtxt and genfromtxt read the entire data into memory which is not desirable. Is there a way to achieve streaming writes? OK, probably fromfile [1] can help you to cook something that works without loading the entire file

Re: [Pytables-users] Storing small files in the filenodes

Thank you, Anthony, I will try VLArray as you suggested =) On Thu, Jul 18, 2013 at 3:39 AM, Anthony Scopatz scop...@gmail.com wrote: Hello Valeriy, For better or worse, the is exactly the performance I would expect. The thing that you are running up against is that every HDF5 data set has

Re: [Pytables-users] Pytables bulk loading data

Thanks. I will try it out and post any findings. Pushkar On Thu, Jul 18, 2013 at 12:36 AM, Andreas Hilboll li...@hilboll.de wrote: You could use pandas_ and the read_table function. There, you have nrows and skiprows parameters with which you can easily do your own 'streaming'. ..