I've been working with large amounts of data and creating very large tree structures that I was hoping to pickle but I've run across two apparent bugs and I was wondering if anyone else had come across any of these (I can't seem to find any mention of them in the archives or the Oz Limitations page).
First, it's probably important to note that all of these tests have been run on a machine with 4GB of memory and I'm aware of the worst-case 2GB limitation due to garbage collection. The first bug is that the 'gc.threshold' and 'gc.size' values seemed to be stored as signed ints as you can see in here: http://nlp.cs.byu.edu/size_neg.png. Initially I believed this to be simply a bug in the Oz Panel but I later realized that if you did {Show {Property.get 'gc.size'}} you could still get a negative number. Of course, if you plan on garbage collecting, you can't use that much memory anyway but its odd to see a negative memory size. Secondly, I've been pickling very large trees and don't seem to always be able pickle these trees. I've been trying to track down exactly what the limitations are but have not been very successful. One thing that I thought I had discovered but haven't been able to duplicate was the ability to call Pickle.pack on a tree that I couldn't call Pickle.save on. So far I've been able to create a tree (nested record) that has 121 million nodes, returns a byte string of 95 million bytes from Pickle.pack and which can be pickled using {Pickle.saveCompressed Tree 'file.txt' 9} into a file of about 30 MB but anything bigger than this and I get an 'Aborted' error (not virtual memory exhausted). Are there inherent limitations on how large a byte string can be? Or perhaps I'm using up all the available memory between the tree and pickle. Its just that this is the only time I ever get 'Aborted' as an error and I was hoping to track down exactly what limitations I'm running up against. Rob Van Dam _________________________________________________________________________________ mozart-users mailing list [email protected] http://www.mozart-oz.org/mailman/listinfo/mozart-users
