Re: [Pytables-users] Pytables bulk loading data

2013-07-18 Thread Pushkar Raj Pande
Thanks. I will try it out and post any findings. Pushkar On Thu, Jul 18, 2013 at 12:36 AM, Andreas Hilboll wrote: > > > > You could use pandas_ and the read_table function. There, you have nrows > and skiprows parameters with which you can easily do your own 'streaming'. > > .. _pandas: http://

Re: [Pytables-users] Pytables bulk loading data

2013-07-17 Thread Anthony Scopatz
Hi Pushkar, I agree with Antonio. You should load your data with NumPy functions and then write back out to PyTables. This is the fastest way to do things. Be Well Anthony On Wed, Jul 17, 2013 at 2:12 PM, Antonio Valentino < antonio.valent...@tiscali.it> wrote: > Hi Pushkar, > > Il 17/07/201

Re: [Pytables-users] Pytables bulk loading data

2013-07-17 Thread Antonio Valentino
Hi Pushkar, Il 17/07/2013 19:28, Pushkar Raj Pande ha scritto: > Hi all, > > I am trying to figure out the best way to bulk load data into pytables. > This question may have been already answered but I couldn't find what I was > looking for. > > The source data is in form of csv which may requir

[Pytables-users] Pytables bulk loading data

2013-07-17 Thread Pushkar Raj Pande
Hi all, I am trying to figure out the best way to bulk load data into pytables. This question may have been already answered but I couldn't find what I was looking for. The source data is in form of csv which may require parsing, type checking and setting default values if it doesn't conform to t