Hello, After running a simulation that took 6 days to complete, my script proceeded to attempt to write the results out to a file, pickled.
The operation failed even though there was 1G of RAM free (4G machine). I've since reconsidered using the pickle format for storing data sets that include large numpy arrays. However, somehow I assumed that one would be able to pickle anything that you already had in memory, but I see now that this was a rash assumption. Ought there to be a way to do this, or should I forget about being able to bundle large numpy arrays and other objects in a single pickle? Thanks, Glen (these commands performed on a different machine with 1G RAM) In [10]: za = numpy.zeros( (100000000,), dtype=numpy.float32 ) In [11]: import cPickle In [12]: zfile = file( '/tmp/zfile', 'w' ) In [13]: cPickle.dump( za, zfile ) --------------------------------------------------------------------------- exceptions.MemoryError Traceback (most recent call last) _______________________________________________ Numpy-discussion mailing list [email protected] http://projects.scipy.org/mailman/listinfo/numpy-discussion
