@jlp965 Even the blocks that are not "asked for" use up virtual address space.
You can already see that in the api of memfiles. You get a pointer and a size.
You don't get blocks that you then can put manually at an address you like. The
memory mapped file uses one continuous block of memory in
You just need **8192 B** (64 bit CPU) for you large files. Whenever you
actually does not need all 16GB data. Try:
import os, posix
const
BufSize = 8192
filename = "/home/king/test.txt"
var
pos = 0
size = 0
f: File
buf:
@andrea has a [spills
module](http://forum.nim-lang.org///andreaferretti.github.io/spills/) that
pages a seq to/from disk (IIRC). This may help or give you some ideas.
I don't agree with @Krux02 because the memfiles reads in multiple (one or more)
block of the file, where each block is
To tell you the difference. Memfiles does not actually put the entire file in
memory, but makes you belive it is. It uses the memory mapping unit that is
normally used for ram swapping, but in this context it reads the context of the
file as soon as the file is accessed. This needs the size of
There's
[FileStream](http://forum.nim-lang.org///nim-lang.org/docs/streams.html#newFileStream,string,FileMode)
that can be used to read chunks of a file using
[readData](http://forum.nim-lang.org///nim-lang.org/docs/streams.html#readData,Stream,Stream,pointer,int),
for example
HI,
I have ram around 8Gb and I want to read a file which is 16gb what is best way.
I know I cant keep whole data in seq[] is there any alternate? Or it is better
to read the data from file directly rather than reading it in seq and process.
Also, is there a way such that I can keep 10% data
Maybe using
[memfiles](http://forum.nim-lang.org///nim-lang.org/docs/memfiles.html) is a
solution for your problem.
