Hello All, I was glad to have found Pytables. Thanks to all involved. I hope someone more experienced than I can offer some advice. My Setup: on OSX 10.4.10, Pytables 2.0, python 2.4.
I'm getting a Segmentation Fault. I'll outline the structure FYI and perhaps someone can suggest a better approach: DB Structure: Parallel (3500+) group nodes off the root node, each with 10 sub-group nodes from which hang data leaves defined with Class (tables.IsDescription) as per the tutorial. The data (time ordered & multiplexed) is read sequentially from a flat file.The 3500+ nodes created on-the-fly and before any leaf data is appended. These nodes are created without error. The Seg. Fault occurs at some point in the leaf creation process. I'm using table = self.h5fileID.getNode() and then table.row['xxx']=data. table.row.append() and table.flush() are executed after the addition of each row. Here's some things I've tried: [1] Save and close the .h5 file after the groups were created. Then instantiate a new python and openFile( ".h5", mode='a') before populating with leaf data. The Seg. Fault still occurs. [2] As [1] but restricting the leaf data to a subset (only 5 of the 10-sub-groups.) Result: No Seg. Fault. [3] Taking the file created by [2] and adding the 2nd 5 of 10 sub- groups. Result: No Seg. Fault. Stats: Group structure .h5 file w . no leaf data: 95 MB Data for 1/65th of leaf data attached as [2]+[3] above: 223MB. (On this estimate final filesize will be 8.4 GB but I haven't tried that much leaf data yet.) I presume that there's something about trying to 'parallel' populate data down the 3500+ root nodes that causes pytables to blow the memory out. Is that possible? and isn't it bug if such action doesn't cause an exception to be generated before a Seg.Fault? Given the above, can anyone suggest a way of proceeding? I can think of 2 alternatives: (a) add the two generated DBs together, or (b) keep the DBs separate and open them both before acting on them. I haven't tried this - is it possible? Any suggestions/feedback most welcome. Sorry this post is a bit long, David _________________________________________________ experimental polymedia: www.avatar.com.au Sonic Communications Research Group, University of Canberra: creative.canberra.edu.au/scrg ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Pytables-users mailing list Pytables-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pytables-users
