[Pytables-users] Pytables bulk loading data

2013-07-17 Thread Pushkar Raj Pande
Hi all, I am trying to figure out the best way to bulk load data into pytables. This question may have been already answered but I couldn't find what I was looking for. The source data is in form of csv which may require parsing, type checking and setting default values if it doesn't conform to

Re: [Pytables-users] Pytables-users Digest, Vol 86, Issue 8

2013-07-18 Thread Pushkar Raj Pande
Both loadtxt and genfromtxt read the entire data into memory which is not desirable. Is there a way to achieve streaming writes? Thanks, Pushkar On Wed, Jul 17, 2013 at 7:04 PM, Pushkar Raj Pande topgun...@gmail.comwrote: Thanks Antonio and Anthony. I will give this a try. -Pushkar

Re: [Pytables-users] Pytables bulk loading data

2013-07-18 Thread Pushkar Raj Pande
'. .. _pandas: http://pandas.pydata.org/ On Thu, Jul 18, 2013 at 1:00 AM, Antonio Valentino antonio.valent...@tiscali.it wrote: Hi Pushkar, Il 18/07/2013 08:45, Pushkar Raj Pande ha scritto: Both loadtxt and genfromtxt read the entire data into memory which is not desirable. Is there a way