After reading this thread, I'm a little confused on how mapped files are
really handled. Years ago I worked on CDC mainframes(205) and on them mapped
files were nothing unusual. When a file was opened as mapped it simply
became an extension of the page (swap) file and was assigned a virtual
address range to cover the file. So references to the file simply were done
as virtual addresses and were paged into and out of real memory as needed
directly from and to the file. There were no restrictions on the size of
real memory other than performance considerations and limits of the virtual
address range. I thought that mapped files on the PC (Windows, at least)
worked the same way.
It seems to me that the only real problem with large mapped files in
J should be the ease in which an entire file can be processed. We have to go
to a lot of trouble to break files into pieces. Most other languages process
files a record at a time so for them there is seldom a need of keeping an
entire mapped file in real memory.
Are you saying that mapping a file on a PC copies the entire file into real
memory then to the swap file as needed? If so then it must make a second
write to the file itself for updates. Sounds pretty awkward.
On Tue, Nov 18, 2008 at 12:17 AM, Chris Burke <[EMAIL PROTECTED]> wrote:
> Jack Andrews wrote:
> >> When a file is mapped, the view (size) of the map must fit in RAM.
> >
> > i'm sure this is not right. i created a small ubuntu guest vm with
> > 512MB ram and here's a python program that succeeds.
> >
> > import mmap,os
> >
> > SIZE=600*1024*1024
> > f=os.tmpfile()
> > print dir(f)
> > f.truncate(SIZE)
> > mm=mmap.mmap(f.fileno(),SIZE)
> > mm.seek(0)
> > mm.write("abc")
> > mm.seek(SIZE-5)
> > mm.write("012")
>
> Correct, I had noticed this recently, and forgot! But the same problem
> remains - that map does map all the file and this won't work for an
> arbitrary large file. You would need to fix the map function for that.
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
