hi Anthony,
thanks for you reply. My comparison is straight forward. Right now we are
using C code and native C HDF5 (ver 1.6.5) routines to create datasets, groups
and organize our sampling data in HDF5 file. In this aspect we tend to create
numerous tables and datasets ( single dimensions ) in the HDF5 file. After going
through the documentation of PyTables i wrote a simple python script to create 3
groups and around 10,000 datasets in each group.
I am not expecting pytables to be as fast as C. But what i am seeing is a
huge difference on the performance front. here you go with script..
import warnings
warnings.simplefilter("ignore")
import numpy
from tables import *
import time
tstart = time.clock()
h5file = openFile('test5.h5', mode='w')
groups = h5file.createGroup("/", 'Parameters_test', 'Parameters Group') # create
groups
atom = Float32Atom()
cond_name = [ "cond1", "cond2", "cond3" ]
#filters = Filters(complevel=1, complib='zlib' )
data_array = numpy.arange(2000) # dummy data array
for counter in range(len(cond_name)):
get_gp = h5file.createGroup( groups, str(cond_name[counter]),
str(cond_name[counter]) )
for i in range(10000):
arr = h5file.createEArray(get_gp, str(i), atom, (0,),"E
array",chunkshape=(1000,) )
arr.append(data_array)
h5file.close()
print time.clock() - tstart, "seconds"
Regards
Dhananjaya
------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
Pytables-users mailing list
Pytables-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/pytables-users
