On 10/19/2012 10:08 AM, Pradipto Banerjee wrote:
Hi,
I am trying to read a file into memory. The size of the file is around 1
GB. I have a 3GB memory PC and the Windows Task Manager shows 2.3 GB
available physical memory when I was trying to read the file. I tried to
read the file as follows:
fdata = open(filename, ‘r’).read()
I got a “MemoryError”. I was watching the Windows Task Manager while I
run the python command, and it appears that python **perhaps** never
even attempted to use more memory but gave me this error.
Is there any reason why python can’t read a 1GB file in memory even when
a 2.3 GB physical memory is available?
The real issue is likely that there is more than one copy of the file in
memory somewhere. I had a similar issue years back that I resolved by
using numeric (now numpy?) as it had a more efficient method of
importing content from disk.
Also realize that windows may not allow the full memory to user space.
I'm not sure what exactly the restrictions are, but a 4Gb windows box
doesn't always get you 4Gb of memory.
Emile
--
http://mail.python.org/mailman/listinfo/python-list
