Re: [Pytables-users] Improving my write speed

2012-03-19 Thread sreeaurovindh viswanathan
Thanks for your clarification and immense help Regards Sree aurovindh V On Mon, Mar 19, 2012 at 11:58 PM, Anthony Scopatz wrote: > On Mon, Mar 19, 2012 at 12:08 PM, sreeaurovindh viswanathan < > sreeaurovi...@gmail.com> wrote: > > [snip] > > >> 2) Can you please point out to an example where i

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread Anthony Scopatz
On Mon, Mar 19, 2012 at 12:08 PM, sreeaurovindh viswanathan < sreeaurovi...@gmail.com> wrote: [snip] > 2) Can you please point out to an example where i can do block hdf5 file > write using pytables (sorry for this naive question) > The Table.append() method ( http://pytables.github.com/usersgu

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread sreeaurovindh viswanathan
Thanks Jarrod Roberson for your suggestions.I could understand the problem .Will incorporate them in my solution.. Thanks On Mon, Mar 19, 2012 at 10:45 PM, Jarrod Roberson wrote: > On Mon, Mar 19, 2012 at 1:08 PM, sreeaurovindh viswanathan > wrote: >> >> Sorry to misphrase my question.But by qu

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread Jarrod Roberson
On Mon, Mar 19, 2012 at 1:08 PM, sreeaurovindh viswanathan wrote: > > Sorry to misphrase my question.But by querying speed i meant the speed of >  "pytable querying and not the postgresql querying.To rephrase, > 1) Will i be able to query(using kernel queries) a single HDF5 file using > pytables p

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread sreeaurovindh viswanathan
hi, Thanks for your suggestions. Sorry to misphrase my question.But by querying speed i meant the speed of "pytable querying and not the postgresql querying.To rephrase, 1) Will i be able to query(using kernel queries) a single HDF5 file using pytables parallely with five different programs? How

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread Anthony Scopatz
What Francesc said ;) On Mon, Mar 19, 2012 at 11:43 AM, Francesc Alted wrote: > My advice regarding parallelization is: do not worry about this *at all* > unless you already spent long time profiling your problem and you are sure > that parallelizing could be of help. 99% of the time is much mo

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread Francesc Alted
My advice regarding parallelization is: do not worry about this *at all* unless you already spent long time profiling your problem and you are sure that parallelizing could be of help. 99% of the time is much more productive focusing on improving serial speed. Please, try to follow Anthony's s

Re: [Pytables-users] Improving my write speed

2012-03-19 Thread Anthony Scopatz
Is there any way that you can query and write in much larger chunks that 6? I don't know much about postgresql in specific, but in general HDF5 does much better if you can take larger chunks. Perhaps you could at least do the postgresql in parallel. Be Well Anthony On Mon, Mar 19, 2012 at 11:23