Max
You are right (as usual) that the VM manager would be the best solution as
it gets the
first bite of the cherry. It does of course mean you are a bit at the mercy
of the OS,
something most of us would hate to admit (let alone acknowledge)
Just suck it up and behave like it aint Win32 :-)
Neven
----- Original Message -----
From: Max Nilson <[EMAIL PROTECTED]>
To: Multiple recipients of list delphi <[EMAIL PROTECTED]>
Sent: Tuesday, 20 March 2001 17:57
Subject: RE: [DUG]: Number of lines in text file
> Sergei Stenkov asked of me:
>
> > Max Nilson wrote:
> >
> > > this works nicely for high speed data importing. Biggest file I have
> > > tested with was 10MB worth of stock item data, and that only
> > > took about three and four seconds to walk.
> >
> > I too believe that memory access is the fastest way, but the
> > problem is that i have to run files of up to 1GB with a norm
> > of about 100Mb. Shurely that would drain the memory in no time?
>
> Not at all! That's the joy and wonder of memory mapped files. What you are
> really doing is reserving a chunk of virtual memory, and then telling the
> memory manager layer that it can page the contents in from disk into that
> memory from the file you created the memory mapping with. If RAM gets low
> all the normal memory management stuff happens and bits of your file can
> be paged out, and you never notice.
>
> You can also use the CreateHandle flags to hint on your access patterns to
> the file, and the memory manage can do read ahead and more optimal RAM
> page reuse based on this hint.
>
> And this is faster than a block read loop because you have very tight
> locality in your code and all the file read stuff is handled by the OS, so
> in theory your entire loop can sit in the cache and the only slow bit is
> the paging in of chunks of your file, which should be happening ahead of
> your reading is the file system is doing its job properly. That's a big
> ask on Win9x, but the NT level code does this very nicely.
>
> Cheers, Max.
>
>
> --------------------------------------------------------------------------
-
> New Zealand Delphi Users group - Delphi List - [EMAIL PROTECTED]
> Website: http://www.delphi.org.nz
> To UnSub, send email to: [EMAIL PROTECTED]
> with body of "unsubscribe delphi"
>
---------------------------------------------------------------------------
New Zealand Delphi Users group - Delphi List - [EMAIL PROTECTED]
Website: http://www.delphi.org.nz
To UnSub, send email to: [EMAIL PROTECTED]
with body of "unsubscribe delphi"
