sargrigory:
I have a simple program that first generates a large (~ 500 mb) file
of random numbers and then reads the numbers back to find their sum.
It uses Data.Binary and Data.ByteString.Lazy.
The problem is when the program tries to read the data back it quickly
(really quickly)
I have tweaked this program a few ways for you.
The big mistake (and why it runs out of space) is that you take
ByteString.Lazy.length to compute the block size. This forces the entire
file into memory -- so no benefits of lazy IO.
As a separate matter, calling 'appendFile . encode'