Mike C. Fletcher wrote: > Mathias Waack wrote: > ... > >>My python script allocates a bunch of strings each of 1024 >>characters and writes it in a cStringIO. And it fails after writing >>512K of strings. Don't know how python restricts the heap size - >>but I'm fairly sure its not a restriction of the OS. >> >> > Does cStringIO require contiguous blocks of memory? I'm wondering > if maybe there's something going on where you just can't allocate > more than > a GB of contiguous space?
That was the problem. I've used cStringIO to reduce the overhead by the python memory management. But now I know - that was a bad idea in this case. > What happens if you just store the > strings in > a list (or use StringIO instead of cStringIO)? It consumed 10 GB really fast, another GB really slow, some hundred MB much slower and then it died;) Mathias -- http://mail.python.org/mailman/listinfo/python-list