@andrea has a [spills 
module](http://forum.nim-lang.org///andreaferretti.github.io/spills/) that 
pages a seq to/from disk (IIRC). This may help or give you some ideas.

I don't agree with @Krux02 because the memfiles reads in multiple (one or more) 
block of the file, where each block is the``PAGE SIZE`` of your OS (usually 4K 
or 8K). If you are on a 32bit OS, you only have a 2g memory space to play with 
(in practice it is less than that due to whatever else is running in your 
programs mem space). The issue is when you ask for too many blocks at one time 
and exceed your free address space.

Can you process your data as a single running calculation (rather than having 
to calculate by parsing all the data multiple times)? 

Reply via email to