On Wed, Apr 02, 2008 at 10:59:57AM -0400, Derek Tracy wrote:
> I generated code that works wonderfully for files under 2Gb in size
> but the majority of the files I am dealing with are over the 2Gb
> limit
>
> ary = array.array('H', INPUT.read())
You're trying to read the file all at once. You need to break your
reads up into smaller chunks, in a loop. You're essentially trying to
store more data in memory than your OS can actually access in a single
process...
Something like this (off the top of my head, I may have overlooked
some detail, but it should at least illustrate the idea):
# read a meg at a time
buffsize = 1048576
while true:
buff = INPUT.read(buffsize)
OUTPUT.write(buff)
if len(buff) != buffsize:
break
--
Derek D. Martin
http://www.pizzashack.org/
GPG Key ID: 0x81CFE75D
pgp80hee5vxgO.pgp
Description: PGP signature
-- http://mail.python.org/mailman/listinfo/python-list
