Hi Tyler,
A Monday 09 February 2009, Tyler Brough escrigué:
> Hello All,
>
> I have a question about speeding up file io. I am building an HDF5
> database from extremely large tick by tick stock datasets. Reading
> them in line by line and appending them to a table is very slow
> going, as you can imagine. Does anyone have any suggestions for
> speeding up the process? I am wondering about using cython with c
> FILE pointers. Has anyone tried this? Your advice is much
> appreciated. I am using pytables on Ubuntu 8.10.
Which procedure are you using for feeding the table? Using the next
idiom:
row = your_table.row
for nrow in xrange(nrows):
row['field0'] = value0
row['field1'] = value1
row.append()
table.flush()
should be pretty fast because you are making use of the internal buffer
for I/O.
Cheers,
--
Francesc Alted
------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
